View in EDS HTML Full Text PDF Full Text

Can ChatGPT generate practice question explanations for medical students, a new faculty teaching tool?

Saved in:
Bibliographic Details
Title: Can ChatGPT generate practice question explanations for medical students, a new faculty teaching tool?
Authors: Tong, Lilin1 tonglil@bu.edu, Wang, Jennifer1, Rapaka, Srikar1, Garg, Priya S.2
Source: Medical Teacher. Mar2025, Vol. 47 Issue 3, p560-564. 5p.
Subject Terms: *Generative artificial intelligence, *Medical education, *College teachers, *Educational tests & measurements, *Educational technology, *Teaching methods, *Medical students, *Reference sources, *Learning strategies
Geographic Terms: United States
Abstract: Introduction: Multiple-choice questions (MCQs) are frequently used for formative assessment in medical school but often lack sufficient answer explanations given time-restraints of faculty. Chat Generated Pre-trained Transformer (ChatGPT) has emerged as a potential student learning aid and faculty teaching tool. This study aims to evaluate ChatGPT's performance in answering and providing explanations for MCQs. Method: Ninety-four faculty-generated MCQs were collected from the pre-clerkship curriculum at a US medical school. ChatGPT's accuracy in answering MCQ's were tracked on first attempt without an answer prompt (Pass 1) and after being given a prompt for the correct answer (Pass 2). Explanations provided by ChatGPT were compared with faculty-generated explanations, and a 3-point evaluation scale was used to assess accuracy and thoroughness compared to faculty-generated answers. Results: On first attempt, ChatGPT demonstrated a 75% accuracy in correctly answering faculty-generated MCQs. Among correctly answered questions, 66.4% of ChatGPT's explanations matched faculty explanations, and 89.1% captured some key aspects without providing inaccurate information. The amount of inaccurately generated explanations increases significantly if the questions was not answered correctly on the first pass (2.7% if correct on first pass vs. 34.6% if incorrect on first pass, p < 0.001). Conclusion: ChatGPT shows promise in assisting faculty and students with explanations for practice MCQ's but should be used with caution. Faculty should review explanations and supplement to ensure coverage of learning objectives. Students can benefit from ChatGPT for immediate feedback through explanations if ChatGPT answers the question correctly on the first try. If the question is answered incorrectly students should remain cautious of the explanation and seek clarification from instructors. [ABSTRACT FROM AUTHOR]
Copyright of Medical Teacher is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: ehh
DbLabel: Education Research Complete
An: 183197187
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Can ChatGPT generate practice question explanations for medical students, a new faculty teaching tool?
– Name: Author
  Label: Authors
  Group: Au
  Data: &lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Tong%2C+Lilin%22&quot;&gt;Tong, Lilin&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt;&lt;i&gt; tonglil@bu.edu&lt;/i&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Wang%2C+Jennifer%22&quot;&gt;Wang, Jennifer&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Rapaka%2C+Srikar%22&quot;&gt;Rapaka, Srikar&lt;/searchLink&gt;&lt;relatesTo&gt;1&lt;/relatesTo&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;AR&quot; term=&quot;%22Garg%2C+Priya+S%2E%22&quot;&gt;Garg, Priya S.&lt;/searchLink&gt;&lt;relatesTo&gt;2&lt;/relatesTo&gt;
– Name: TitleSource
  Label: Source
  Group: Src
  Data: &lt;searchLink fieldCode=&quot;JN&quot; term=&quot;%22Medical+Teacher%22&quot;&gt;Medical Teacher&lt;/searchLink&gt;. Mar2025, Vol. 47 Issue 3, p560-564. 5p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Generative+artificial+intelligence%22&quot;&gt;Generative artificial intelligence&lt;/searchLink&gt;&lt;br /&gt;*&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Medical+education%22&quot;&gt;Medical education&lt;/searchLink&gt;&lt;br /&gt;*&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22College+teachers%22&quot;&gt;College teachers&lt;/searchLink&gt;&lt;br /&gt;*&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Educational+tests+%26+measurements%22&quot;&gt;Educational tests &amp; measurements&lt;/searchLink&gt;&lt;br /&gt;*&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Educational+technology%22&quot;&gt;Educational technology&lt;/searchLink&gt;&lt;br /&gt;*&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Teaching+methods%22&quot;&gt;Teaching methods&lt;/searchLink&gt;&lt;br /&gt;*&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Medical+students%22&quot;&gt;Medical students&lt;/searchLink&gt;&lt;br /&gt;*&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Reference+sources%22&quot;&gt;Reference sources&lt;/searchLink&gt;&lt;br /&gt;*&lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22Learning+strategies%22&quot;&gt;Learning strategies&lt;/searchLink&gt;
– Name: SubjectGeographic
  Label: Geographic Terms
  Group: Su
  Data: &lt;searchLink fieldCode=&quot;DE&quot; term=&quot;%22United+States%22&quot;&gt;United States&lt;/searchLink&gt;
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Introduction: Multiple-choice questions (MCQs) are frequently used for formative assessment in medical school but often lack sufficient answer explanations given time-restraints of faculty. Chat Generated Pre-trained Transformer (ChatGPT) has emerged as a potential student learning aid and faculty teaching tool. This study aims to evaluate ChatGPT&#39;s performance in answering and providing explanations for MCQs. Method: Ninety-four faculty-generated MCQs were collected from the pre-clerkship curriculum at a US medical school. ChatGPT&#39;s accuracy in answering MCQ&#39;s were tracked on first attempt without an answer prompt (Pass 1) and after being given a prompt for the correct answer (Pass 2). Explanations provided by ChatGPT were compared with faculty-generated explanations, and a 3-point evaluation scale was used to assess accuracy and thoroughness compared to faculty-generated answers. Results: On first attempt, ChatGPT demonstrated a 75% accuracy in correctly answering faculty-generated MCQs. Among correctly answered questions, 66.4% of ChatGPT&#39;s explanations matched faculty explanations, and 89.1% captured some key aspects without providing inaccurate information. The amount of inaccurately generated explanations increases significantly if the questions was not answered correctly on the first pass (2.7% if correct on first pass vs. 34.6% if incorrect on first pass, p &lt; 0.001). Conclusion: ChatGPT shows promise in assisting faculty and students with explanations for practice MCQ&#39;s but should be used with caution. Faculty should review explanations and supplement to ensure coverage of learning objectives. Students can benefit from ChatGPT for immediate feedback through explanations if ChatGPT answers the question correctly on the first try. If the question is answered incorrectly students should remain cautious of the explanation and seek clarification from instructors. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: &lt;i&gt;Copyright of Medical Teacher is the property of Taylor &amp; Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder&#39;s express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.&lt;/i&gt; (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=183197187
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/0142159X.2024.2363486
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 5
        StartPage: 560
    Subjects:
      – SubjectFull: Generative artificial intelligence
        Type: general
      – SubjectFull: Medical education
        Type: general
      – SubjectFull: College teachers
        Type: general
      – SubjectFull: Educational tests & measurements
        Type: general
      – SubjectFull: Educational technology
        Type: general
      – SubjectFull: Teaching methods
        Type: general
      – SubjectFull: Medical students
        Type: general
      – SubjectFull: Reference sources
        Type: general
      – SubjectFull: Learning strategies
        Type: general
      – SubjectFull: United States
        Type: general
    Titles:
      – TitleFull: Can ChatGPT generate practice question explanations for medical students, a new faculty teaching tool?
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Tong, Lilin
      – PersonEntity:
          Name:
            NameFull: Wang, Jennifer
      – PersonEntity:
          Name:
            NameFull: Rapaka, Srikar
      – PersonEntity:
          Name:
            NameFull: Garg, Priya S.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Text: Mar2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0142159X
          Numbering:
            – Type: volume
              Value: 47
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Medical Teacher
              Type: main
ResultId 1