Expert Evaluation of Artificial Intelligence Chatbots for Central Auditory Processing Disorder Information.

Saved in:

Bibliographic Details
Title:	Expert Evaluation of Artificial Intelligence Chatbots for Central Auditory Processing Disorder Information.
Authors:	Davidson, Alyssa J.¹ ajeverett9917@gmail.com, Jedrzejczak, W. Wiktor^2,3, McCullagh, Jennifer⁴, Ferre, Jeanane M.⁵, Bradbury, Amy⁶, Palmer, Shannon B.⁷, Iliadou, Vasiliki M.⁸, Musiek, Frank E.⁹
Source:	American Journal of Audiology. Jun2026, Vol. 35 Issue 2, p668-679. 12p.
Subject Terms:	Generative artificial intelligence, Information resources, Expertise, Comparative studies, *Information-seeking behavior, Word deafness, Medical personnel, Health, Research evaluation, Descriptive statistics, Attitudes of medical personnel, Analysis of variance, Data analysis software
Geographic Terms:	United States, Europe
Abstract:	Purpose: Artificial intelligence (AI) chatbots based on large language models (LLMs) can deliver medical information, but their performance on specialized topics such as central auditory processing disorder (CAPD) remains unexplored. This study evaluated the accuracy and completeness of three AI chatbots (ChatGPT, Gemini, and Claude) in providing CAPD-related information across varying levels of question complexity. Method: Forty-four questions, categorized into four difficulty levels (patient level, easy, intermediate, and specialized; n = 11 each), were submitted to each chatbot, generating 132 responses. Seven clinical experts, blinded to chatbot identity, independently rated accuracy and completeness on a 1-5 Likert scale. Data were analyzed with analyses of variance, correlations, and interrater comparisons. Results: Chatbot performance was similar, with mean accuracy below 4.0 and completeness about 3.5. Complex questions often scored below 3.0 across experts. Only three of the 44 questions, primarily patient level or relatively simple, received consistently high expert ratings (≥ 4 for both accuracy and completeness) across all three chatbots. Performance declined with question difficulty, although differences were not statistically significant. Accuracy and completeness were correlated across chatbots. Conclusions: Current AI chatbots provided generally accurate CAPD information but fell short of clinical standards, particularly on specialized questions. Their limited performance underscores the need for clinician oversight in CAPD assessment and management. Chatbots may serve as helpful adjuncts but should not replace expert evaluation and guidance in clinical settings. [ABSTRACT FROM AUTHOR]
	Copyright of American Journal of Audiology is the property of American Speech-Language-Hearing Association and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Education Research Complete

Description
Abstract:	Purpose: Artificial intelligence (AI) chatbots based on large language models (LLMs) can deliver medical information, but their performance on specialized topics such as central auditory processing disorder (CAPD) remains unexplored. This study evaluated the accuracy and completeness of three AI chatbots (ChatGPT, Gemini, and Claude) in providing CAPD-related information across varying levels of question complexity. Method: Forty-four questions, categorized into four difficulty levels (patient level, easy, intermediate, and specialized; n = 11 each), were submitted to each chatbot, generating 132 responses. Seven clinical experts, blinded to chatbot identity, independently rated accuracy and completeness on a 1-5 Likert scale. Data were analyzed with analyses of variance, correlations, and interrater comparisons. Results: Chatbot performance was similar, with mean accuracy below 4.0 and completeness about 3.5. Complex questions often scored below 3.0 across experts. Only three of the 44 questions, primarily patient level or relatively simple, received consistently high expert ratings (≥ 4 for both accuracy and completeness) across all three chatbots. Performance declined with question difficulty, although differences were not statistically significant. Accuracy and completeness were correlated across chatbots. Conclusions: Current AI chatbots provided generally accurate CAPD information but fell short of clinical standards, particularly on specialized questions. Their limited performance underscores the need for clinician oversight in CAPD assessment and management. Chatbots may serve as helpful adjuncts but should not replace expert evaluation and guidance in clinical settings. [ABSTRACT FROM AUTHOR]
ISSN:	10590889
DOI:	10.1044/2026_AJA-25-00224