View in EDS PDF Full Text

Inter‐laboratory comparison of channelized hotelling observer computation.

Saved in:
Bibliographic Details
Title: Inter‐laboratory comparison of channelized hotelling observer computation.
Authors: Ba, Alexandre1, Abbey, Craig K.2, Baek, Jongduk3, Han, Minah3, Bouwman, Ramona W.4, Balta, Christiana4, Brankov, Jovan5, Massanes, Francesc5, Gifford, Howard C.6, Hernandez‐Giron, Irene7, Veldkamp, Wouter J. H.7, Petrov, Dimitar8, Marshall, Nicholas8,9, Samuelson, Frank W.10, Zeng, Rongping10, Solomon, Justin B.11, Samei, Ehsan11, Timberg, Pontus12, Förnvik, Hannie12, Reiser, Ingrid13
Source: Medical Physics. Jul2018, Vol. 45 Issue 7, p3019-3030. 12p.
Subjects: Image quality analysis, Computed tomography, Diagnostic imaging, Laboratories, Estimation theory
Abstract: Purpose: The task‐based assessment of image quality using model observers is increasingly used for the assessment of different imaging modalities. However, the performance computation of model observers needs standardization as well as a well‐established trust in its implementation methodology and uncertainty estimation. The purpose of this work was to determine the degree of equivalence of the channelized Hotelling observer performance and uncertainty estimation using an intercomparison exercise. Materials and Methods: Image samples to estimate model observer performance for detection tasks were generated from two‐dimensional CT image slices of a uniform water phantom. A common set of images was sent to participating laboratories to perform and document the following tasks: (a) estimate the detectability index of a well‐defined CHO and its uncertainty in three conditions involving different sized targets all at the same dose, and (b) apply this CHO to an image set where ground truth was unknown to participants (lower image dose). In addition, and on an optional basis, we asked the participating laboratories to (c) estimate the performance of real human observers from a psychophysical experiment of their choice. Each of the 13 participating laboratories was confidentially assigned a participant number and image sets could be downloaded through a secure server. Results were distributed with each participant recognizable by its number and then each laboratory was able to modify their results with justification as model observer calculation are not yet a routine and potentially error prone. Results: Detectability index increased with signal size for all participants and was very consistent for 6 mm sized target while showing higher variability for 8 and 10 mm sized target. There was one order of magnitude between the lowest and the largest uncertainty estimation. Conclusions: This intercomparison helped define the state of the art of model observer performance computation and with thirteen participants, reflects openness and trust within the medical imaging community. The performance of a CHO with explicitly defined channels and a relatively large number of test images was consistently estimated by all participants. In contrast, the paper demonstrates that there is no agreement on estimating the variance of detectability in the training and testing setting. [ABSTRACT FROM AUTHOR]
Copyright of Medical Physics is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 130628534
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Inter‐laboratory comparison of channelized hotelling observer computation.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Ba%2C+Alexandre%22">Ba, Alexandre</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Abbey%2C+Craig+K%2E%22">Abbey, Craig K.</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Baek%2C+Jongduk%22">Baek, Jongduk</searchLink><relatesTo>3</relatesTo><br /><searchLink fieldCode="AR" term="%22Han%2C+Minah%22">Han, Minah</searchLink><relatesTo>3</relatesTo><br /><searchLink fieldCode="AR" term="%22Bouwman%2C+Ramona+W%2E%22">Bouwman, Ramona W.</searchLink><relatesTo>4</relatesTo><br /><searchLink fieldCode="AR" term="%22Balta%2C+Christiana%22">Balta, Christiana</searchLink><relatesTo>4</relatesTo><br /><searchLink fieldCode="AR" term="%22Brankov%2C+Jovan%22">Brankov, Jovan</searchLink><relatesTo>5</relatesTo><br /><searchLink fieldCode="AR" term="%22Massanes%2C+Francesc%22">Massanes, Francesc</searchLink><relatesTo>5</relatesTo><br /><searchLink fieldCode="AR" term="%22Gifford%2C+Howard+C%2E%22">Gifford, Howard C.</searchLink><relatesTo>6</relatesTo><br /><searchLink fieldCode="AR" term="%22Hernandez‐Giron%2C+Irene%22">Hernandez‐Giron, Irene</searchLink><relatesTo>7</relatesTo><br /><searchLink fieldCode="AR" term="%22Veldkamp%2C+Wouter+J%2E+H%2E%22">Veldkamp, Wouter J. H.</searchLink><relatesTo>7</relatesTo><br /><searchLink fieldCode="AR" term="%22Petrov%2C+Dimitar%22">Petrov, Dimitar</searchLink><relatesTo>8</relatesTo><br /><searchLink fieldCode="AR" term="%22Marshall%2C+Nicholas%22">Marshall, Nicholas</searchLink><relatesTo>8,9</relatesTo><br /><searchLink fieldCode="AR" term="%22Samuelson%2C+Frank+W%2E%22">Samuelson, Frank W.</searchLink><relatesTo>10</relatesTo><br /><searchLink fieldCode="AR" term="%22Zeng%2C+Rongping%22">Zeng, Rongping</searchLink><relatesTo>10</relatesTo><br /><searchLink fieldCode="AR" term="%22Solomon%2C+Justin+B%2E%22">Solomon, Justin B.</searchLink><relatesTo>11</relatesTo><br /><searchLink fieldCode="AR" term="%22Samei%2C+Ehsan%22">Samei, Ehsan</searchLink><relatesTo>11</relatesTo><br /><searchLink fieldCode="AR" term="%22Timberg%2C+Pontus%22">Timberg, Pontus</searchLink><relatesTo>12</relatesTo><br /><searchLink fieldCode="AR" term="%22Förnvik%2C+Hannie%22">Förnvik, Hannie</searchLink><relatesTo>12</relatesTo><br /><searchLink fieldCode="AR" term="%22Reiser%2C+Ingrid%22">Reiser, Ingrid</searchLink><relatesTo>13</relatesTo>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Medical+Physics%22">Medical Physics</searchLink>. Jul2018, Vol. 45 Issue 7, p3019-3030. 12p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Image+quality+analysis%22">Image quality analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Computed+tomography%22">Computed tomography</searchLink><br /><searchLink fieldCode="DE" term="%22Diagnostic+imaging%22">Diagnostic imaging</searchLink><br /><searchLink fieldCode="DE" term="%22Laboratories%22">Laboratories</searchLink><br /><searchLink fieldCode="DE" term="%22Estimation+theory%22">Estimation theory</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Purpose: The task‐based assessment of image quality using model observers is increasingly used for the assessment of different imaging modalities. However, the performance computation of model observers needs standardization as well as a well‐established trust in its implementation methodology and uncertainty estimation. The purpose of this work was to determine the degree of equivalence of the channelized Hotelling observer performance and uncertainty estimation using an intercomparison exercise. Materials and Methods: Image samples to estimate model observer performance for detection tasks were generated from two‐dimensional CT image slices of a uniform water phantom. A common set of images was sent to participating laboratories to perform and document the following tasks: (a) estimate the detectability index of a well‐defined CHO and its uncertainty in three conditions involving different sized targets all at the same dose, and (b) apply this CHO to an image set where ground truth was unknown to participants (lower image dose). In addition, and on an optional basis, we asked the participating laboratories to (c) estimate the performance of real human observers from a psychophysical experiment of their choice. Each of the 13 participating laboratories was confidentially assigned a participant number and image sets could be downloaded through a secure server. Results were distributed with each participant recognizable by its number and then each laboratory was able to modify their results with justification as model observer calculation are not yet a routine and potentially error prone. Results: Detectability index increased with signal size for all participants and was very consistent for 6 mm sized target while showing higher variability for 8 and 10 mm sized target. There was one order of magnitude between the lowest and the largest uncertainty estimation. Conclusions: This intercomparison helped define the state of the art of model observer performance computation and with thirteen participants, reflects openness and trust within the medical imaging community. The performance of a CHO with explicitly defined channels and a relatively large number of test images was consistently estimated by all participants. In contrast, the paper demonstrates that there is no agreement on estimating the variance of detectability in the training and testing setting. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Medical Physics is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=130628534
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/mp.12940
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 3019
    Subjects:
      – SubjectFull: Image quality analysis
        Type: general
      – SubjectFull: Computed tomography
        Type: general
      – SubjectFull: Diagnostic imaging
        Type: general
      – SubjectFull: Laboratories
        Type: general
      – SubjectFull: Estimation theory
        Type: general
    Titles:
      – TitleFull: Inter‐laboratory comparison of channelized hotelling observer computation.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Ba, Alexandre
      – PersonEntity:
          Name:
            NameFull: Abbey, Craig K.
      – PersonEntity:
          Name:
            NameFull: Baek, Jongduk
      – PersonEntity:
          Name:
            NameFull: Han, Minah
      – PersonEntity:
          Name:
            NameFull: Bouwman, Ramona W.
      – PersonEntity:
          Name:
            NameFull: Balta, Christiana
      – PersonEntity:
          Name:
            NameFull: Brankov, Jovan
      – PersonEntity:
          Name:
            NameFull: Massanes, Francesc
      – PersonEntity:
          Name:
            NameFull: Gifford, Howard C.
      – PersonEntity:
          Name:
            NameFull: Hernandez‐Giron, Irene
      – PersonEntity:
          Name:
            NameFull: Veldkamp, Wouter J. H.
      – PersonEntity:
          Name:
            NameFull: Petrov, Dimitar
      – PersonEntity:
          Name:
            NameFull: Marshall, Nicholas
      – PersonEntity:
          Name:
            NameFull: Samuelson, Frank W.
      – PersonEntity:
          Name:
            NameFull: Zeng, Rongping
      – PersonEntity:
          Name:
            NameFull: Solomon, Justin B.
      – PersonEntity:
          Name:
            NameFull: Samei, Ehsan
      – PersonEntity:
          Name:
            NameFull: Timberg, Pontus
      – PersonEntity:
          Name:
            NameFull: Förnvik, Hannie
      – PersonEntity:
          Name:
            NameFull: Reiser, Ingrid
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Text: Jul2018
              Type: published
              Y: 2018
          Identifiers:
            – Type: issn-print
              Value: 00942405
          Numbering:
            – Type: volume
              Value: 45
            – Type: issue
              Value: 7
          Titles:
            – TitleFull: Medical Physics
              Type: main
ResultId 1