View in EDS HTML Full Text PDF Full Text

Impact of hospital-specific domain adaptation on BERT-based models to classify neuroradiology reports.

Saved in:
Bibliographic Details
Title: Impact of hospital-specific domain adaptation on BERT-based models to classify neuroradiology reports.
Authors: Agarwal, Siddharth1 (AUTHOR), Wood, David1 (AUTHOR), Murray, Benjamin A. K.1 (AUTHOR), Wei, Yiran1 (AUTHOR), Busaidi, Ayisha Al2 (AUTHOR), Kafiabadi, Sina2 (AUTHOR), Guilhem, Emily2 (AUTHOR), Lynch, Jeremy2 (AUTHOR), Townend, Matthew1 (AUTHOR), Mazumder, Asif3 (AUTHOR), Barker, Gareth J.4 (AUTHOR), Cole, James H.5 (AUTHOR), Sasieni, Peter6 (AUTHOR), Ourselin, Sebastien1 (AUTHOR), Modat, Marc1 (AUTHOR), Booth, Thomas C.1,2 (AUTHOR) thomas.booth@kcl.ac.uk
Source: European Radiology. Sep2025, Vol. 35 Issue 9, p5299-5313. 15p.
Subjects: Neuroradiology, Classification, Language models, Brain imaging, Health facilities
Abstract: Objectives: To determine the effectiveness of hospital-specific domain adaptation through masked language modelling (MLM) on BERT-based models' performance in classifying neuroradiology reports, and to compare these models with open-source large language models (LLMs). Materials and methods: This retrospective study (2008–2019) utilised 126,556 and 86,032 MRI brain reports from two tertiary hospitals—King's College Hospital (KCH) and Guys and St Thomas' Trust (GSTT). Various BERT-based models, including RoBERTa, BioBERT and RadBERT, underwent MLM on unlabelled reports from these centres. The downstream tasks were binary abnormality classification and multi-label classification. Performances of models with and without hospital-specific domain adaptation were compared against each other and LLMs on internal (KCH) and external (GSTT) hold-out test sets. Model performances for binary classification were compared using 2-way and 1-way ANOVA. Results: All models that underwent hospital-specific domain adaptation performed better than their baseline counterparts (all p-values < 0.001). For binary classification, MLM on all available unlabelled reports (194,467 reports) yielded the highest balanced accuracies (KCH: mean 97.0 ± 0.4% (standard deviation), GSTT: 95.5 ± 1.0%), after which no differences between BERT-based models remained (1-way ANOVA, p-values > 0.05). There was a log-linear relationship between the number of reports and performance. LLama-3.0 70B was the best-performing LLM (KCH: 97.1%, GSTT: 94.0%). Multi-label classification demonstrated consistent performance improvements from MLM for all abnormality categories. Conclusion: Hospital-specific domain adaptation should be considered best practice when deploying BERT-based models in new clinical settings. When labelled data is scarce or unavailable, LLMs can serve as a viable alternative, assuming adequate computational power is accessible. Key Points: QuestionBERT-based models can classify radiology reports, but it is unclear if there is any incremental benefit from additional hospital-specific domain adaptation. FindingsHospital-specific domain adaptation resulted in the highest BERT-based model accuracies and performance scaled log-linearly with the number of reports. Clinical relevanceBERT-based models after hospital-specific domain adaptation achieve the best classification results provided sufficient high-quality training labels. When labelled data is scarce, LLMs such as Llama-3.0 70B are a viable alternative provided there are sufficient computational resources. [ABSTRACT FROM AUTHOR]
Copyright of European Radiology is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: egs
DbLabel: Engineering Source
An: 187309367
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Impact of hospital-specific domain adaptation on BERT-based models to classify neuroradiology reports.
– Name: Author
  Label: Authors
  Group: Au
  Data: &lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Agarwal%2C+Siddharth%22&quot;&gt;Agarwal, Siddharth&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Wood%2C+David%22&quot;&gt;Wood, David&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Murray%2C+Benjamin+A%2E+K%2E%22&quot;&gt;Murray, Benjamin A. K.&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Wei%2C+Yiran%22&quot;&gt;Wei, Yiran&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Busaidi%2C+Ayisha+Al%22&quot;&gt;Busaidi, Ayisha Al&lt;/searchLink&gt;&lt;relatesTo&gt;2&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Kafiabadi%2C+Sina%22&quot;&gt;Kafiabadi, Sina&lt;/searchLink&gt;&lt;relatesTo&gt;2&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Guilhem%2C+Emily%22&quot;&gt;Guilhem, Emily&lt;/searchLink&gt;&lt;relatesTo&gt;2&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Lynch%2C+Jeremy%22&quot;&gt;Lynch, Jeremy&lt;/searchLink&gt;&lt;relatesTo&gt;2&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Townend%2C+Matthew%22&quot;&gt;Townend, Matthew&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Mazumder%2C+Asif%22&quot;&gt;Mazumder, Asif&lt;/searchLink&gt;&lt;relatesTo&gt;3&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Barker%2C+Gareth+J%2E%22&quot;&gt;Barker, Gareth J.&lt;/searchLink&gt;&lt;relatesTo&gt;4&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Cole%2C+James+H%2E%22&quot;&gt;Cole, James H.&lt;/searchLink&gt;&lt;relatesTo&gt;5&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Sasieni%2C+Peter%22&quot;&gt;Sasieni, Peter&lt;/searchLink&gt;&lt;relatesTo&gt;6&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Ourselin%2C+Sebastien%22&quot;&gt;Ourselin, Sebastien&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Modat%2C+Marc%22&quot;&gt;Modat, Marc&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt; (AUTHOR)&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Booth%2C+Thomas+C%2E%22&quot;&gt;Booth, Thomas C.&lt;/searchLink&gt;&lt;relatesTo&gt;1,2&lt;/relatesTo&gt; (AUTHOR)&lt;i&gt; thomas.booth@kcl.ac.uk&lt;/i&gt;
– Name: TitleSource
  Label: Source
  Group: Src
  Data: &lt;searchLink fieldCode=&quot;JN&quot; term=&quot;%22European+Radiology%22&quot;&gt;European Radiology&lt;/searchLink&gt;. Sep2025, Vol. 35 Issue 9, p5299-5313. 15p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: &lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Neuroradiology%22&quot;&gt;Neuroradiology&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Classification%22&quot;&gt;Classification&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Language+models%22&quot;&gt;Language models&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Brain+imaging%22&quot;&gt;Brain imaging&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Health+facilities%22&quot;&gt;Health facilities&lt;/searchLink&gt;
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Objectives: To determine the effectiveness of hospital-specific domain adaptation through masked language modelling (MLM) on BERT-based models&#39; performance in classifying neuroradiology reports, and to compare these models with open-source large language models (LLMs). Materials and methods: This retrospective study (2008–2019) utilised 126,556 and 86,032 MRI brain reports from two tertiary hospitals—King&#39;s College Hospital (KCH) and Guys and St Thomas&#39; Trust (GSTT). Various BERT-based models, including RoBERTa, BioBERT and RadBERT, underwent MLM on unlabelled reports from these centres. The downstream tasks were binary abnormality classification and multi-label classification. Performances of models with and without hospital-specific domain adaptation were compared against each other and LLMs on internal (KCH) and external (GSTT) hold-out test sets. Model performances for binary classification were compared using 2-way and 1-way ANOVA. Results: All models that underwent hospital-specific domain adaptation performed better than their baseline counterparts (all p-values &lt; 0.001). For binary classification, MLM on all available unlabelled reports (194,467 reports) yielded the highest balanced accuracies (KCH: mean 97.0 &#177; 0.4% (standard deviation), GSTT: 95.5 &#177; 1.0%), after which no differences between BERT-based models remained (1-way ANOVA, p-values &gt; 0.05). There was a log-linear relationship between the number of reports and performance. LLama-3.0 70B was the best-performing LLM (KCH: 97.1%, GSTT: 94.0%). Multi-label classification demonstrated consistent performance improvements from MLM for all abnormality categories. Conclusion: Hospital-specific domain adaptation should be considered best practice when deploying BERT-based models in new clinical settings. When labelled data is scarce or unavailable, LLMs can serve as a viable alternative, assuming adequate computational power is accessible. Key Points: QuestionBERT-based models can classify radiology reports, but it is unclear if there is any incremental benefit from additional hospital-specific domain adaptation. FindingsHospital-specific domain adaptation resulted in the highest BERT-based model accuracies and performance scaled log-linearly with the number of reports. Clinical relevanceBERT-based models after hospital-specific domain adaptation achieve the best classification results provided sufficient high-quality training labels. When labelled data is scarce, LLMs such as Llama-3.0 70B are a viable alternative provided there are sufficient computational resources. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: &lt;i&gt;Copyright of European Radiology is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder&#39;s express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.&lt;/i&gt; (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=187309367
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s00330-025-11500-9
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 15
        StartPage: 5299
    Subjects:
      – SubjectFull: Neuroradiology
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Language models
        Type: general
      – SubjectFull: Brain imaging
        Type: general
      – SubjectFull: Health facilities
        Type: general
    Titles:
      – TitleFull: Impact of hospital-specific domain adaptation on BERT-based models to classify neuroradiology reports.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Agarwal, Siddharth
      – PersonEntity:
          Name:
            NameFull: Wood, David
      – PersonEntity:
          Name:
            NameFull: Murray, Benjamin A. K.
      – PersonEntity:
          Name:
            NameFull: Wei, Yiran
      – PersonEntity:
          Name:
            NameFull: Busaidi, Ayisha Al
      – PersonEntity:
          Name:
            NameFull: Kafiabadi, Sina
      – PersonEntity:
          Name:
            NameFull: Guilhem, Emily
      – PersonEntity:
          Name:
            NameFull: Lynch, Jeremy
      – PersonEntity:
          Name:
            NameFull: Townend, Matthew
      – PersonEntity:
          Name:
            NameFull: Mazumder, Asif
      – PersonEntity:
          Name:
            NameFull: Barker, Gareth J.
      – PersonEntity:
          Name:
            NameFull: Cole, James H.
      – PersonEntity:
          Name:
            NameFull: Sasieni, Peter
      – PersonEntity:
          Name:
            NameFull: Ourselin, Sebastien
      – PersonEntity:
          Name:
            NameFull: Modat, Marc
      – PersonEntity:
          Name:
            NameFull: Booth, Thomas C.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 09
              Text: Sep2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 09387994
          Numbering:
            – Type: volume
              Value: 35
            – Type: issue
              Value: 9
          Titles:
            – TitleFull: European Radiology
              Type: main
ResultId 1