View in EDS PDF Full Text

Expert Evaluation of Artificial Intelligence Chatbots for Central Auditory Processing Disorder Information.

Saved in:
Bibliographic Details
Title: Expert Evaluation of Artificial Intelligence Chatbots for Central Auditory Processing Disorder Information.
Authors: Davidson, Alyssa J.1 ajeverett9917@gmail.com, Jedrzejczak, W. Wiktor2,3, McCullagh, Jennifer4, Ferre, Jeanane M.5, Bradbury, Amy6, Palmer, Shannon B.7, Iliadou, Vasiliki M.8, Musiek, Frank E.9
Source: American Journal of Audiology. Jun2026, Vol. 35 Issue 2, p668-679. 12p.
Subject Terms: *Generative artificial intelligence, *Information resources, *Expertise, *Comparative studies, *Information-seeking behavior, Word deafness, Medical personnel, Health, Research evaluation, Descriptive statistics, Attitudes of medical personnel, Analysis of variance, Data analysis software
Geographic Terms: United States, Europe
Abstract: Purpose: Artificial intelligence (AI) chatbots based on large language models (LLMs) can deliver medical information, but their performance on specialized topics such as central auditory processing disorder (CAPD) remains unexplored. This study evaluated the accuracy and completeness of three AI chatbots (ChatGPT, Gemini, and Claude) in providing CAPD-related information across varying levels of question complexity. Method: Forty-four questions, categorized into four difficulty levels (patient level, easy, intermediate, and specialized; n = 11 each), were submitted to each chatbot, generating 132 responses. Seven clinical experts, blinded to chatbot identity, independently rated accuracy and completeness on a 1-5 Likert scale. Data were analyzed with analyses of variance, correlations, and interrater comparisons. Results: Chatbot performance was similar, with mean accuracy below 4.0 and completeness about 3.5. Complex questions often scored below 3.0 across experts. Only three of the 44 questions, primarily patient level or relatively simple, received consistently high expert ratings (≥ 4 for both accuracy and completeness) across all three chatbots. Performance declined with question difficulty, although differences were not statistically significant. Accuracy and completeness were correlated across chatbots. Conclusions: Current AI chatbots provided generally accurate CAPD information but fell short of clinical standards, particularly on specialized questions. Their limited performance underscores the need for clinician oversight in CAPD assessment and management. Chatbots may serve as helpful adjuncts but should not replace expert evaluation and guidance in clinical settings. [ABSTRACT FROM AUTHOR]
Copyright of American Journal of Audiology is the property of American Speech-Language-Hearing Association and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: ehh
DbLabel: Education Research Complete
An: 194359739
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Expert Evaluation of Artificial Intelligence Chatbots for Central Auditory Processing Disorder Information.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Davidson%2C+Alyssa+J%2E%22">Davidson, Alyssa J.</searchLink><relatesTo>1</relatesTo><i> ajeverett9917@gmail.com</i><br /><searchLink fieldCode="AR" term="%22Jedrzejczak%2C+W%2E+Wiktor%22">Jedrzejczak, W. Wiktor</searchLink><relatesTo>2,3</relatesTo><br /><searchLink fieldCode="AR" term="%22McCullagh%2C+Jennifer%22">McCullagh, Jennifer</searchLink><relatesTo>4</relatesTo><br /><searchLink fieldCode="AR" term="%22Ferre%2C+Jeanane+M%2E%22">Ferre, Jeanane M.</searchLink><relatesTo>5</relatesTo><br /><searchLink fieldCode="AR" term="%22Bradbury%2C+Amy%22">Bradbury, Amy</searchLink><relatesTo>6</relatesTo><br /><searchLink fieldCode="AR" term="%22Palmer%2C+Shannon+B%2E%22">Palmer, Shannon B.</searchLink><relatesTo>7</relatesTo><br /><searchLink fieldCode="AR" term="%22Iliadou%2C+Vasiliki+M%2E%22">Iliadou, Vasiliki M.</searchLink><relatesTo>8</relatesTo><br /><searchLink fieldCode="AR" term="%22Musiek%2C+Frank+E%2E%22">Musiek, Frank E.</searchLink><relatesTo>9</relatesTo>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22American+Journal+of+Audiology%22">American Journal of Audiology</searchLink>. Jun2026, Vol. 35 Issue 2, p668-679. 12p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Generative+artificial+intelligence%22">Generative artificial intelligence</searchLink><br />*<searchLink fieldCode="DE" term="%22Information+resources%22">Information resources</searchLink><br />*<searchLink fieldCode="DE" term="%22Expertise%22">Expertise</searchLink><br />*<searchLink fieldCode="DE" term="%22Comparative+studies%22">Comparative studies</searchLink><br />*<searchLink fieldCode="DE" term="%22Information-seeking+behavior%22">Information-seeking behavior</searchLink><br /><searchLink fieldCode="DE" term="%22Word+deafness%22">Word deafness</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+personnel%22">Medical personnel</searchLink><br /><searchLink fieldCode="DE" term="%22Health%22">Health</searchLink><br /><searchLink fieldCode="DE" term="%22Research+evaluation%22">Research evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Descriptive+statistics%22">Descriptive statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Attitudes+of+medical+personnel%22">Attitudes of medical personnel</searchLink><br /><searchLink fieldCode="DE" term="%22Analysis+of+variance%22">Analysis of variance</searchLink><br /><searchLink fieldCode="DE" term="%22Data+analysis+software%22">Data analysis software</searchLink>
– Name: SubjectGeographic
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22United+States%22">United States</searchLink><br /><searchLink fieldCode="DE" term="%22Europe%22">Europe</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Purpose: Artificial intelligence (AI) chatbots based on large language models (LLMs) can deliver medical information, but their performance on specialized topics such as central auditory processing disorder (CAPD) remains unexplored. This study evaluated the accuracy and completeness of three AI chatbots (ChatGPT, Gemini, and Claude) in providing CAPD-related information across varying levels of question complexity. Method: Forty-four questions, categorized into four difficulty levels (patient level, easy, intermediate, and specialized; n = 11 each), were submitted to each chatbot, generating 132 responses. Seven clinical experts, blinded to chatbot identity, independently rated accuracy and completeness on a 1-5 Likert scale. Data were analyzed with analyses of variance, correlations, and interrater comparisons. Results: Chatbot performance was similar, with mean accuracy below 4.0 and completeness about 3.5. Complex questions often scored below 3.0 across experts. Only three of the 44 questions, primarily patient level or relatively simple, received consistently high expert ratings (≥ 4 for both accuracy and completeness) across all three chatbots. Performance declined with question difficulty, although differences were not statistically significant. Accuracy and completeness were correlated across chatbots. Conclusions: Current AI chatbots provided generally accurate CAPD information but fell short of clinical standards, particularly on specialized questions. Their limited performance underscores the need for clinician oversight in CAPD assessment and management. Chatbots may serve as helpful adjuncts but should not replace expert evaluation and guidance in clinical settings. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of American Journal of Audiology is the property of American Speech-Language-Hearing Association and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=194359739
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1044/2026_AJA-25-00224
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 668
    Subjects:
      – SubjectFull: Generative artificial intelligence
        Type: general
      – SubjectFull: Information resources
        Type: general
      – SubjectFull: Expertise
        Type: general
      – SubjectFull: Comparative studies
        Type: general
      – SubjectFull: Information-seeking behavior
        Type: general
      – SubjectFull: Word deafness
        Type: general
      – SubjectFull: Medical personnel
        Type: general
      – SubjectFull: Health
        Type: general
      – SubjectFull: Research evaluation
        Type: general
      – SubjectFull: Descriptive statistics
        Type: general
      – SubjectFull: Attitudes of medical personnel
        Type: general
      – SubjectFull: Analysis of variance
        Type: general
      – SubjectFull: Data analysis software
        Type: general
      – SubjectFull: United States
        Type: general
      – SubjectFull: Europe
        Type: general
    Titles:
      – TitleFull: Expert Evaluation of Artificial Intelligence Chatbots for Central Auditory Processing Disorder Information.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Davidson, Alyssa J.
      – PersonEntity:
          Name:
            NameFull: Jedrzejczak, W. Wiktor
      – PersonEntity:
          Name:
            NameFull: McCullagh, Jennifer
      – PersonEntity:
          Name:
            NameFull: Ferre, Jeanane M.
      – PersonEntity:
          Name:
            NameFull: Bradbury, Amy
      – PersonEntity:
          Name:
            NameFull: Palmer, Shannon B.
      – PersonEntity:
          Name:
            NameFull: Iliadou, Vasiliki M.
      – PersonEntity:
          Name:
            NameFull: Musiek, Frank E.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 06
              Text: Jun2026
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 10590889
          Numbering:
            – Type: volume
              Value: 35
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: American Journal of Audiology
              Type: main
ResultId 1