View in EDS HTML Full Text PDF Full Text

Evaluating the application of ChatGPT in China's residency training education: An exploratory study.

Saved in:
Bibliographic Details
Title: Evaluating the application of ChatGPT in China's residency training education: An exploratory study.
Authors: Shang, Luxiang1,2, Li, Rui3, Xue, Mingyue4, Guo, Qilong5, Hou, Yinglong1 yinglonghou@hotmail.com
Source: Medical Teacher. May2025, Vol. 47 Issue 5, p858-864. 7p.
Subject Terms: *Generative artificial intelligence, *Medical education, *Internship programs, *Health occupations students, *Information resources, *Teaching methods, *Educational tests & measurements, *Educational technology, *Students, *Research, *Student attitudes, Scale analysis (Psychology), Research funding, Satisfaction, Health, Descriptive statistics, Surveys, Simulated patients, User-centered system design, Data quality
Geographic Terms: China
Abstract: Objective: The purpose of this study was to assess the utility of information generated by ChatGPT for residency education in China. Methods: We designed a three-step survey to evaluate the performance of ChatGPT in China's residency training education including residency final examination questions, patient cases, and resident satisfaction scores. First, 204 questions from the residency final exam were input into ChatGPT's interface to obtain the percentage of correct answers. Next, ChatGPT was asked to generate 20 clinical cases, which were subsequently evaluated by three instructors using a pre-designed Likert scale with 5 points. The quality of the cases was assessed based on criteria including clarity, relevance, logicality, credibility, and comprehensiveness. Finally, interaction sessions between 31 third-year residents and ChatGPT were conducted. Residents' perceptions of ChatGPT's feedback were assessed using a Likert scale, focusing on aspects such as ease of use, accuracy and completeness of responses, and its effectiveness in enhancing understanding of medical knowledge. Results: Our results showed ChatGPT-3.5 correctly answered 45.1% of exam questions. In the virtual patient cases, ChatGPT received mean ratings of 4.57 ± 0.50, 4.68 ± 0.47, 4.77 ± 0.46, 4.60 ± 0.53, and 3.95 ± 0.59 points for clarity, relevance, logicality, credibility, and comprehensiveness from clinical instructors, respectively. Among training residents, ChatGPT scored 4.48 ± 0.70, 4.00 ± 0.82 and 4.61 ± 0.50 points for ease of use, accuracy and completeness, and usefulness, respectively. Conclusion: Our findings demonstrate ChatGPT's immense potential for personalized Chinese medical education. [ABSTRACT FROM AUTHOR]
Copyright of Medical Teacher is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: ehh
DbLabel: Education Research Complete
An: 184864309
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Evaluating the application of ChatGPT in China's residency training education: An exploratory study.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Shang%2C+Luxiang%22">Shang, Luxiang</searchLink><relatesTo>1,2</relatesTo><br /><searchLink fieldCode="AR" term="%22Li%2C+Rui%22">Li, Rui</searchLink><relatesTo>3</relatesTo><br /><searchLink fieldCode="AR" term="%22Xue%2C+Mingyue%22">Xue, Mingyue</searchLink><relatesTo>4</relatesTo><br /><searchLink fieldCode="AR" term="%22Guo%2C+Qilong%22">Guo, Qilong</searchLink><relatesTo>5</relatesTo><br /><searchLink fieldCode="AR" term="%22Hou%2C+Yinglong%22">Hou, Yinglong</searchLink><relatesTo>1</relatesTo><i> yinglonghou@hotmail.com</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Medical+Teacher%22">Medical Teacher</searchLink>. May2025, Vol. 47 Issue 5, p858-864. 7p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Generative+artificial+intelligence%22">Generative artificial intelligence</searchLink><br />*<searchLink fieldCode="DE" term="%22Medical+education%22">Medical education</searchLink><br />*<searchLink fieldCode="DE" term="%22Internship+programs%22">Internship programs</searchLink><br />*<searchLink fieldCode="DE" term="%22Health+occupations+students%22">Health occupations students</searchLink><br />*<searchLink fieldCode="DE" term="%22Information+resources%22">Information resources</searchLink><br />*<searchLink fieldCode="DE" term="%22Teaching+methods%22">Teaching methods</searchLink><br />*<searchLink fieldCode="DE" term="%22Educational+tests+%26+measurements%22">Educational tests & measurements</searchLink><br />*<searchLink fieldCode="DE" term="%22Educational+technology%22">Educational technology</searchLink><br />*<searchLink fieldCode="DE" term="%22Students%22">Students</searchLink><br />*<searchLink fieldCode="DE" term="%22Research%22">Research</searchLink><br />*<searchLink fieldCode="DE" term="%22Student+attitudes%22">Student attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Scale+analysis+%28Psychology%29%22">Scale analysis (Psychology)</searchLink><br /><searchLink fieldCode="DE" term="%22Research+funding%22">Research funding</searchLink><br /><searchLink fieldCode="DE" term="%22Satisfaction%22">Satisfaction</searchLink><br /><searchLink fieldCode="DE" term="%22Health%22">Health</searchLink><br /><searchLink fieldCode="DE" term="%22Descriptive+statistics%22">Descriptive statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Surveys%22">Surveys</searchLink><br /><searchLink fieldCode="DE" term="%22Simulated+patients%22">Simulated patients</searchLink><br /><searchLink fieldCode="DE" term="%22User-centered+system+design%22">User-centered system design</searchLink><br /><searchLink fieldCode="DE" term="%22Data+quality%22">Data quality</searchLink>
– Name: SubjectGeographic
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22China%22">China</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Objective: The purpose of this study was to assess the utility of information generated by ChatGPT for residency education in China. Methods: We designed a three-step survey to evaluate the performance of ChatGPT in China's residency training education including residency final examination questions, patient cases, and resident satisfaction scores. First, 204 questions from the residency final exam were input into ChatGPT's interface to obtain the percentage of correct answers. Next, ChatGPT was asked to generate 20 clinical cases, which were subsequently evaluated by three instructors using a pre-designed Likert scale with 5 points. The quality of the cases was assessed based on criteria including clarity, relevance, logicality, credibility, and comprehensiveness. Finally, interaction sessions between 31 third-year residents and ChatGPT were conducted. Residents' perceptions of ChatGPT's feedback were assessed using a Likert scale, focusing on aspects such as ease of use, accuracy and completeness of responses, and its effectiveness in enhancing understanding of medical knowledge. Results: Our results showed ChatGPT-3.5 correctly answered 45.1% of exam questions. In the virtual patient cases, ChatGPT received mean ratings of 4.57 ± 0.50, 4.68 ± 0.47, 4.77 ± 0.46, 4.60 ± 0.53, and 3.95 ± 0.59 points for clarity, relevance, logicality, credibility, and comprehensiveness from clinical instructors, respectively. Among training residents, ChatGPT scored 4.48 ± 0.70, 4.00 ± 0.82 and 4.61 ± 0.50 points for ease of use, accuracy and completeness, and usefulness, respectively. Conclusion: Our findings demonstrate ChatGPT's immense potential for personalized Chinese medical education. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Medical Teacher is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=184864309
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/0142159X.2024.2377808
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 7
        StartPage: 858
    Subjects:
      – SubjectFull: Generative artificial intelligence
        Type: general
      – SubjectFull: Medical education
        Type: general
      – SubjectFull: Internship programs
        Type: general
      – SubjectFull: Health occupations students
        Type: general
      – SubjectFull: Information resources
        Type: general
      – SubjectFull: Teaching methods
        Type: general
      – SubjectFull: Educational tests & measurements
        Type: general
      – SubjectFull: Educational technology
        Type: general
      – SubjectFull: Students
        Type: general
      – SubjectFull: Research
        Type: general
      – SubjectFull: Student attitudes
        Type: general
      – SubjectFull: Scale analysis (Psychology)
        Type: general
      – SubjectFull: Research funding
        Type: general
      – SubjectFull: Satisfaction
        Type: general
      – SubjectFull: Health
        Type: general
      – SubjectFull: Descriptive statistics
        Type: general
      – SubjectFull: Surveys
        Type: general
      – SubjectFull: Simulated patients
        Type: general
      – SubjectFull: User-centered system design
        Type: general
      – SubjectFull: Data quality
        Type: general
      – SubjectFull: China
        Type: general
    Titles:
      – TitleFull: Evaluating the application of ChatGPT in China's residency training education: An exploratory study.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Shang, Luxiang
      – PersonEntity:
          Name:
            NameFull: Li, Rui
      – PersonEntity:
          Name:
            NameFull: Xue, Mingyue
      – PersonEntity:
          Name:
            NameFull: Guo, Qilong
      – PersonEntity:
          Name:
            NameFull: Hou, Yinglong
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 05
              Text: May2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0142159X
          Numbering:
            – Type: volume
              Value: 47
            – Type: issue
              Value: 5
          Titles:
            – TitleFull: Medical Teacher
              Type: main
ResultId 1