Evaluating the Performance of ChatGPT on Board-Style Examination Questions in Ophthalmology: A Meta-Analysis.
Saved in:
| Title: | Evaluating the Performance of ChatGPT on Board-Style Examination Questions in Ophthalmology: A Meta-Analysis. |
|---|---|
| Authors: | Wei, Jiawen1, Wang, Xiaoyan1, Huang, Mingxue1, Xu, Yanwu2,3, Yang, Weihua4 benben0606@139.com |
| Source: | Journal of Medical Systems. 7/5/2025, Vol. 49 Issue 1, p1-13. 13p. |
| Subjects: | Generative artificial intelligence, Medical information storage & retrieval systems, Language & languages, Optics, Family medicine, Medical education, Professional licensure examinations, Meta-analysis, Descriptive statistics, Natural language processing, Educational tests & measurements, Ophthalmology, Systematic reviews, MEDLINE, Medical databases, Online information services, Quality assurance, Digital image processing, Evidence-based medicine, Confidence intervals, Communication barriers |
| Abstract (English): | To review empirical research on ChatGPT's accuracy in answering ophthalmology board-style examination questions up to March 2025 and to analyze the effects of GPT versions, question types, language differences, and ophthalmology topics on accuracy. A search was conducted in PubMed, Web of Science, Embase, Scopus, and the Cochrane Library in March 2025. Two authors extracted data and independently assessed study quality. Accuracy rates were calculated with Stata 17.0. GPT-4 had an integrated accuracy of 73%, higher than GPT-3.5's 54%. It scored 77% in text and 55% in image tasks. GPT-4's accuracy was 73% in English-speaking countries and 71% in non-English ones. In ophthalmology, General Medicine achieved the highest accuracy (80%), while Clinical Optics had the lowest performance (55%). GPT-4 outperforms GPT-3.5, but its image processing capability needs further validation. Performance varies by language and topic, suggesting the need for more research on cross-linguistic efficacy and error analysis. [ABSTRACT FROM AUTHOR] |
| Abstract (German): | Clinical Trial Number: Not applicable. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Medical Systems is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | To review empirical research on ChatGPT's accuracy in answering ophthalmology board-style examination questions up to March 2025 and to analyze the effects of GPT versions, question types, language differences, and ophthalmology topics on accuracy. A search was conducted in PubMed, Web of Science, Embase, Scopus, and the Cochrane Library in March 2025. Two authors extracted data and independently assessed study quality. Accuracy rates were calculated with Stata 17.0. GPT-4 had an integrated accuracy of 73%, higher than GPT-3.5's 54%. It scored 77% in text and 55% in image tasks. GPT-4's accuracy was 73% in English-speaking countries and 71% in non-English ones. In ophthalmology, General Medicine achieved the highest accuracy (80%), while Clinical Optics had the lowest performance (55%). GPT-4 outperforms GPT-3.5, but its image processing capability needs further validation. Performance varies by language and topic, suggesting the need for more research on cross-linguistic efficacy and error analysis. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 01485598 |
| DOI: | 10.1007/s10916-025-02227-7 |
