View in EDS

Investigating the Efficacy of ChatGPT-3.5 for Tutoring in Chinese Elementary Education Settings

Saved in:
Bibliographic Details
Title: Investigating the Efficacy of ChatGPT-3.5 for Tutoring in Chinese Elementary Education Settings
Language: English
Authors: Yu Bai (ORCID 0009-0002-1805-1503), Jun Li (ORCID 0009-0000-9978-4690), Jun Shen (ORCID 0000-0002-9403-7140), Liang Zhao (ORCID 0000-0001-5829-6850)
Source: IEEE Transactions on Learning Technologies. 2024 17:2156-2171.
Availability: Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=4620076
Peer Reviewed: Y
Page Count: 16
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Education Level: Elementary Education
Descriptors: Instructional Effectiveness, Artificial Intelligence, Technology Uses in Education, Natural Language Processing, Tutoring, Elementary School Students, Foreign Countries, Standardized Tests, Academic Achievement, Accuracy, Teaching Methods, Study Skills, Barriers
Geographic Terms: China
DOI: 10.1109/TLT.2024.3464560
ISSN: 1939-1382
Abstract: The potential of artificial intelligence (AI) in transforming education has received considerable attention. This study aims to explore the potential of large language models (LLMs) in assisting students with studying and passing standardized exams, while many people think it is a hype situation. Using primary education as an example, this research investigates whether ChatGPT-3.5 can achieve satisfactory performance on the Chinese Primary School Exams and whether it can be used as a teaching aid or tutor. We designed an experimental framework and constructed a benchmark that comprises 4800 questions collected from 48 tasks in Chinese elementary education settings. Through automatic and manual evaluations, we observed that ChatGPT-3.5's pass rate was below the required level of accuracy for most tasks, and the correctness of ChatGPT-3.5's answer interpretation was unsatisfactory. These results revealed a discrepancy between the findings and our initial expectations. However, the comparative experiments between ChatGPT-3.5 and ChatGPT-4 indicated significant improvements in model performance, demonstrating the potential of using LLMs as a teaching aid. This article also investigates the use of the trans-prompting strategy to reduce the impact of language bias and enhance question understanding. We present a comparison of the models' performance and the improvement under the trans-lingual problem decomposition prompting mechanism. Finally, we discuss the challenges associated with the appropriate application of AI-driven language models, along with future directions and limitations in the field of AI for education.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1445362
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1445362
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Investigating the Efficacy of ChatGPT-3.5 for Tutoring in Chinese Elementary Education Settings
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Yu+Bai%22">Yu Bai</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0002-1805-1503">0009-0002-1805-1503</externalLink>)<br /><searchLink fieldCode="AR" term="%22Jun+Li%22">Jun Li</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0000-9978-4690">0009-0000-9978-4690</externalLink>)<br /><searchLink fieldCode="AR" term="%22Jun+Shen%22">Jun Shen</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-9403-7140">0000-0002-9403-7140</externalLink>)<br /><searchLink fieldCode="AR" term="%22Liang+Zhao%22">Liang Zhao</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5829-6850">0000-0001-5829-6850</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22IEEE+Transactions+on+Learning+Technologies%22"><i>IEEE Transactions on Learning Technologies</i></searchLink>. 2024 17:2156-2171.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=4620076
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 16
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Instructional+Effectiveness%22">Instructional Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Tutoring%22">Tutoring</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Standardized+Tests%22">Standardized Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Achievement%22">Academic Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Teaching+Methods%22">Teaching Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Study+Skills%22">Study Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Barriers%22">Barriers</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22China%22">China</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1109/TLT.2024.3464560
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1939-1382
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The potential of artificial intelligence (AI) in transforming education has received considerable attention. This study aims to explore the potential of large language models (LLMs) in assisting students with studying and passing standardized exams, while many people think it is a hype situation. Using primary education as an example, this research investigates whether ChatGPT-3.5 can achieve satisfactory performance on the Chinese Primary School Exams and whether it can be used as a teaching aid or tutor. We designed an experimental framework and constructed a benchmark that comprises 4800 questions collected from 48 tasks in Chinese elementary education settings. Through automatic and manual evaluations, we observed that ChatGPT-3.5's pass rate was below the required level of accuracy for most tasks, and the correctness of ChatGPT-3.5's answer interpretation was unsatisfactory. These results revealed a discrepancy between the findings and our initial expectations. However, the comparative experiments between ChatGPT-3.5 and ChatGPT-4 indicated significant improvements in model performance, demonstrating the potential of using LLMs as a teaching aid. This article also investigates the use of the trans-prompting strategy to reduce the impact of language bias and enhance question understanding. We present a comparison of the models' performance and the improvement under the trans-lingual problem decomposition prompting mechanism. Finally, we discuss the challenges associated with the appropriate application of AI-driven language models, along with future directions and limitations in the field of AI for education.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1445362
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1445362
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1109/TLT.2024.3464560
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
        StartPage: 2156
    Subjects:
      – SubjectFull: Instructional Effectiveness
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Tutoring
        Type: general
      – SubjectFull: Elementary School Students
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Standardized Tests
        Type: general
      – SubjectFull: Academic Achievement
        Type: general
      – SubjectFull: Accuracy
        Type: general
      – SubjectFull: Teaching Methods
        Type: general
      – SubjectFull: Study Skills
        Type: general
      – SubjectFull: Barriers
        Type: general
      – SubjectFull: China
        Type: general
    Titles:
      – TitleFull: Investigating the Efficacy of ChatGPT-3.5 for Tutoring in Chinese Elementary Education Settings
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Yu Bai
      – PersonEntity:
          Name:
            NameFull: Jun Li
      – PersonEntity:
          Name:
            NameFull: Jun Shen
      – PersonEntity:
          Name:
            NameFull: Liang Zhao
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-electronic
              Value: 1939-1382
          Numbering:
            – Type: volume
              Value: 17
          Titles:
            – TitleFull: IEEE Transactions on Learning Technologies
              Type: main
ResultId 1