View in EDS HTML Full Text PDF Full Text

Meta-research: How many diagnostic or prognostic models published in radiological journals are evaluated externally?

Saved in:
Bibliographic Details
Title: Meta-research: How many diagnostic or prognostic models published in radiological journals are evaluated externally?
Authors: Hameed, Maira1 (AUTHOR), Yeung, Jason1 (AUTHOR), Boone, Darren1 (AUTHOR), Mallett, Sue1 (AUTHOR), Halligan, Steve1 (AUTHOR) s.halligan@ucl.ac.uk
Source: European Radiology. Apr2024, Vol. 34 Issue 4, p2524-2533. 10p.
Subjects: Meta-research, Prognostic models, Proportional hazards models, Periodical publishing, Research personnel
Abstract: Objectives: Prognostic and diagnostic models must work in their intended clinical setting, proven via "external evaluation", preferably by authors uninvolved with model development. By systematic review, we determined the proportion of models published in high-impact radiological journals that are evaluated subsequently. Methods: We hand-searched three radiological journals for multivariable diagnostic/prognostic models 2013–2015 inclusive, developed using regression. We assessed completeness of data presentation to allow subsequent external evaluation. We then searched literature to August 2022 to identify external evaluations of these index models. Results: We identified 98 index studies (73 prognostic; 25 diagnostic) describing 145 models. Only 15 (15%) index studies presented an evaluation (two external). No model was updated. Only 20 (20%) studies presented a model equation. Just 7 (15%) studies developing Cox models presented a risk table, and just 4 (9%) presented the baseline hazard. Two (4%) studies developing non-Cox models presented the intercept. Just 20 (20%) articles presented a Kaplan–Meier curve of the final model. The 98 index studies attracted 4224 citations (including 559 self-citations), median 28 per study. We identified just six (6%) subsequent external evaluations of an index model, five of which were external evaluations by researchers uninvolved with model development, and from a different institution. Conclusions: Very few prognostic or diagnostic models published in radiological literature are evaluated externally, suggesting wasted research effort and resources. Authors' published models should present data sufficient to allow external evaluation by others. To achieve clinical utility, researchers should concentrate on model evaluation and updating rather than continual redevelopment. Clinical relevance statement: The large majority of prognostic and diagnostic models published in high-impact radiological journals are never evaluated. It would be more efficient for researchers to evaluate existing models rather than practice continual redevelopment. Key Points: • Systematic review of highly cited radiological literature identified few diagnostic or prognostic models that were evaluated subsequently by researchers uninvolved with the original model. • Published radiological models frequently omit important information necessary for others to perform an external evaluation: Only 20% of studies presented a model equation or nomogram. • A large proportion of research citing published models focuses on redevelopment and ignores evaluation and updating, which would be a more efficient use of research resources. [ABSTRACT FROM AUTHOR]
Copyright of European Radiology is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: egs
DbLabel: Engineering Source
An: 176221673
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Meta-research: How many diagnostic or prognostic models published in radiological journals are evaluated externally?
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Hameed%2C+Maira%22">Hameed, Maira</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Yeung%2C+Jason%22">Yeung, Jason</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Boone%2C+Darren%22">Boone, Darren</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Mallett%2C+Sue%22">Mallett, Sue</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Halligan%2C+Steve%22">Halligan, Steve</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> s.halligan@ucl.ac.uk</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22European+Radiology%22">European Radiology</searchLink>. Apr2024, Vol. 34 Issue 4, p2524-2533. 10p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Meta-research%22">Meta-research</searchLink><br /><searchLink fieldCode="DE" term="%22Prognostic+models%22">Prognostic models</searchLink><br /><searchLink fieldCode="DE" term="%22Proportional+hazards+models%22">Proportional hazards models</searchLink><br /><searchLink fieldCode="DE" term="%22Periodical+publishing%22">Periodical publishing</searchLink><br /><searchLink fieldCode="DE" term="%22Research+personnel%22">Research personnel</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Objectives: Prognostic and diagnostic models must work in their intended clinical setting, proven via "external evaluation", preferably by authors uninvolved with model development. By systematic review, we determined the proportion of models published in high-impact radiological journals that are evaluated subsequently. Methods: We hand-searched three radiological journals for multivariable diagnostic/prognostic models 2013–2015 inclusive, developed using regression. We assessed completeness of data presentation to allow subsequent external evaluation. We then searched literature to August 2022 to identify external evaluations of these index models. Results: We identified 98 index studies (73 prognostic; 25 diagnostic) describing 145 models. Only 15 (15%) index studies presented an evaluation (two external). No model was updated. Only 20 (20%) studies presented a model equation. Just 7 (15%) studies developing Cox models presented a risk table, and just 4 (9%) presented the baseline hazard. Two (4%) studies developing non-Cox models presented the intercept. Just 20 (20%) articles presented a Kaplan–Meier curve of the final model. The 98 index studies attracted 4224 citations (including 559 self-citations), median 28 per study. We identified just six (6%) subsequent external evaluations of an index model, five of which were external evaluations by researchers uninvolved with model development, and from a different institution. Conclusions: Very few prognostic or diagnostic models published in radiological literature are evaluated externally, suggesting wasted research effort and resources. Authors' published models should present data sufficient to allow external evaluation by others. To achieve clinical utility, researchers should concentrate on model evaluation and updating rather than continual redevelopment. Clinical relevance statement: The large majority of prognostic and diagnostic models published in high-impact radiological journals are never evaluated. It would be more efficient for researchers to evaluate existing models rather than practice continual redevelopment. Key Points: • Systematic review of highly cited radiological literature identified few diagnostic or prognostic models that were evaluated subsequently by researchers uninvolved with the original model. • Published radiological models frequently omit important information necessary for others to perform an external evaluation: Only 20% of studies presented a model equation or nomogram. • A large proportion of research citing published models focuses on redevelopment and ignores evaluation and updating, which would be a more efficient use of research resources. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of European Radiology is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=176221673
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s00330-023-10168-3
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 10
        StartPage: 2524
    Subjects:
      – SubjectFull: Meta-research
        Type: general
      – SubjectFull: Prognostic models
        Type: general
      – SubjectFull: Proportional hazards models
        Type: general
      – SubjectFull: Periodical publishing
        Type: general
      – SubjectFull: Research personnel
        Type: general
    Titles:
      – TitleFull: Meta-research: How many diagnostic or prognostic models published in radiological journals are evaluated externally?
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Hameed, Maira
      – PersonEntity:
          Name:
            NameFull: Yeung, Jason
      – PersonEntity:
          Name:
            NameFull: Boone, Darren
      – PersonEntity:
          Name:
            NameFull: Mallett, Sue
      – PersonEntity:
          Name:
            NameFull: Halligan, Steve
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 04
              Text: Apr2024
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 09387994
          Numbering:
            – Type: volume
              Value: 34
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: European Radiology
              Type: main
ResultId 1