View in EDS HTML Full Text PDF Full Text

Evaluating the Performance of ChatGPT on Board-Style Examination Questions in Ophthalmology: A Meta-Analysis.

Saved in:
Bibliographic Details
Title: Evaluating the Performance of ChatGPT on Board-Style Examination Questions in Ophthalmology: A Meta-Analysis.
Authors: Wei, Jiawen1, Wang, Xiaoyan1, Huang, Mingxue1, Xu, Yanwu2,3, Yang, Weihua4 benben0606@139.com
Source: Journal of Medical Systems. 7/5/2025, Vol. 49 Issue 1, p1-13. 13p.
Subjects: Generative artificial intelligence, Medical information storage & retrieval systems, Language & languages, Optics, Family medicine, Medical education, Professional licensure examinations, Meta-analysis, Descriptive statistics, Natural language processing, Educational tests & measurements, Ophthalmology, Systematic reviews, MEDLINE, Medical databases, Online information services, Quality assurance, Digital image processing, Evidence-based medicine, Confidence intervals, Communication barriers
Abstract (English): To review empirical research on ChatGPT's accuracy in answering ophthalmology board-style examination questions up to March 2025 and to analyze the effects of GPT versions, question types, language differences, and ophthalmology topics on accuracy. A search was conducted in PubMed, Web of Science, Embase, Scopus, and the Cochrane Library in March 2025. Two authors extracted data and independently assessed study quality. Accuracy rates were calculated with Stata 17.0. GPT-4 had an integrated accuracy of 73%, higher than GPT-3.5's 54%. It scored 77% in text and 55% in image tasks. GPT-4's accuracy was 73% in English-speaking countries and 71% in non-English ones. In ophthalmology, General Medicine achieved the highest accuracy (80%), while Clinical Optics had the lowest performance (55%). GPT-4 outperforms GPT-3.5, but its image processing capability needs further validation. Performance varies by language and topic, suggesting the need for more research on cross-linguistic efficacy and error analysis. [ABSTRACT FROM AUTHOR]
Abstract (German): Clinical Trial Number: Not applicable. [ABSTRACT FROM AUTHOR]
Copyright of Journal of Medical Systems is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: egs
DbLabel: Engineering Source
An: 186463869
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Evaluating the Performance of ChatGPT on Board-Style Examination Questions in Ophthalmology: A Meta-Analysis.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Wei%2C+Jiawen%22">Wei, Jiawen</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Wang%2C+Xiaoyan%22">Wang, Xiaoyan</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Huang%2C+Mingxue%22">Huang, Mingxue</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Xu%2C+Yanwu%22">Xu, Yanwu</searchLink><relatesTo>2,3</relatesTo><br /><searchLink fieldCode="AR" term="%22Yang%2C+Weihua%22">Yang, Weihua</searchLink><relatesTo>4</relatesTo><i> benben0606@139.com</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Journal+of+Medical+Systems%22">Journal of Medical Systems</searchLink>. 7/5/2025, Vol. 49 Issue 1, p1-13. 13p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Generative+artificial+intelligence%22">Generative artificial intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+information+storage+%26+retrieval+systems%22">Medical information storage & retrieval systems</searchLink><br /><searchLink fieldCode="DE" term="%22Language+%26+languages%22">Language & languages</searchLink><br /><searchLink fieldCode="DE" term="%22Optics%22">Optics</searchLink><br /><searchLink fieldCode="DE" term="%22Family+medicine%22">Family medicine</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+education%22">Medical education</searchLink><br /><searchLink fieldCode="DE" term="%22Professional+licensure+examinations%22">Professional licensure examinations</searchLink><br /><searchLink fieldCode="DE" term="%22Meta-analysis%22">Meta-analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Descriptive+statistics%22">Descriptive statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+language+processing%22">Natural language processing</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+tests+%26+measurements%22">Educational tests & measurements</searchLink><br /><searchLink fieldCode="DE" term="%22Ophthalmology%22">Ophthalmology</searchLink><br /><searchLink fieldCode="DE" term="%22Systematic+reviews%22">Systematic reviews</searchLink><br /><searchLink fieldCode="DE" term="%22MEDLINE%22">MEDLINE</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+databases%22">Medical databases</searchLink><br /><searchLink fieldCode="DE" term="%22Online+information+services%22">Online information services</searchLink><br /><searchLink fieldCode="DE" term="%22Quality+assurance%22">Quality assurance</searchLink><br /><searchLink fieldCode="DE" term="%22Digital+image+processing%22">Digital image processing</searchLink><br /><searchLink fieldCode="DE" term="%22Evidence-based+medicine%22">Evidence-based medicine</searchLink><br /><searchLink fieldCode="DE" term="%22Confidence+intervals%22">Confidence intervals</searchLink><br /><searchLink fieldCode="DE" term="%22Communication+barriers%22">Communication barriers</searchLink>
– Name: Abstract
  Label: Abstract (English)
  Group: Ab
  Data: To review empirical research on ChatGPT's accuracy in answering ophthalmology board-style examination questions up to March 2025 and to analyze the effects of GPT versions, question types, language differences, and ophthalmology topics on accuracy. A search was conducted in PubMed, Web of Science, Embase, Scopus, and the Cochrane Library in March 2025. Two authors extracted data and independently assessed study quality. Accuracy rates were calculated with Stata 17.0. GPT-4 had an integrated accuracy of 73%, higher than GPT-3.5's 54%. It scored 77% in text and 55% in image tasks. GPT-4's accuracy was 73% in English-speaking countries and 71% in non-English ones. In ophthalmology, General Medicine achieved the highest accuracy (80%), while Clinical Optics had the lowest performance (55%). GPT-4 outperforms GPT-3.5, but its image processing capability needs further validation. Performance varies by language and topic, suggesting the need for more research on cross-linguistic efficacy and error analysis. [ABSTRACT FROM AUTHOR]
– Name: Abstract
  Label: Abstract (German)
  Group: Ab
  Data: Clinical Trial Number: Not applicable. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Journal of Medical Systems is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=186463869
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10916-025-02227-7
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 13
        StartPage: 1
    Subjects:
      – SubjectFull: Generative artificial intelligence
        Type: general
      – SubjectFull: Medical information storage & retrieval systems
        Type: general
      – SubjectFull: Language & languages
        Type: general
      – SubjectFull: Optics
        Type: general
      – SubjectFull: Family medicine
        Type: general
      – SubjectFull: Medical education
        Type: general
      – SubjectFull: Professional licensure examinations
        Type: general
      – SubjectFull: Meta-analysis
        Type: general
      – SubjectFull: Descriptive statistics
        Type: general
      – SubjectFull: Natural language processing
        Type: general
      – SubjectFull: Educational tests & measurements
        Type: general
      – SubjectFull: Ophthalmology
        Type: general
      – SubjectFull: Systematic reviews
        Type: general
      – SubjectFull: MEDLINE
        Type: general
      – SubjectFull: Medical databases
        Type: general
      – SubjectFull: Online information services
        Type: general
      – SubjectFull: Quality assurance
        Type: general
      – SubjectFull: Digital image processing
        Type: general
      – SubjectFull: Evidence-based medicine
        Type: general
      – SubjectFull: Confidence intervals
        Type: general
      – SubjectFull: Communication barriers
        Type: general
    Titles:
      – TitleFull: Evaluating the Performance of ChatGPT on Board-Style Examination Questions in Ophthalmology: A Meta-Analysis.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Wei, Jiawen
      – PersonEntity:
          Name:
            NameFull: Wang, Xiaoyan
      – PersonEntity:
          Name:
            NameFull: Huang, Mingxue
      – PersonEntity:
          Name:
            NameFull: Xu, Yanwu
      – PersonEntity:
          Name:
            NameFull: Yang, Weihua
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 05
              M: 07
              Text: 7/5/2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 01485598
          Numbering:
            – Type: volume
              Value: 49
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Journal of Medical Systems
              Type: main
ResultId 1