View in EDS PDF Full Text

Assessing AI's problem solving in physics: Analyzing reasoning, false positives and negatives through the force concept inventory.

Saved in:
Bibliographic Details
Title: Assessing AI's problem solving in physics: Analyzing reasoning, false positives and negatives through the force concept inventory.
Authors: Aldazharova, Salima1, Issayeva, Gulnara1, Maxutov, Samat2, Balta, Nuri2 baltanuri@gmail.com
Source: Contemporary Educational Technology. Oct2024, Vol. 16 Issue 4, p1-16. 16p.
Subject Terms: *Physics education, *Common misconceptions, *Educational outcomes, Newton's laws of motion, Generative pre-trained transformers
Abstract: This study investigates the performance of GPT-4, an advanced AI model developed by OpenAI, on the force concept inventory (FCI) to evaluate its accuracy, reasoning patterns, and the occurrence of false positives and false negatives. GPT-4 was tasked with answering the FCI questions across multiple sessions. Key findings include GPT-4's proficiency in several FCI items, particularly those related to Newton's third law, achieving perfect scores on many items. However, it struggled significantly with questions involving the interpretation of figures and spatial reasoning, resulting in a higher occurrence of false negatives where the reasoning was correct, but the answers were incorrect. Additionally, GPT-4 displayed several conceptual errors, such as misunderstanding the effect of friction and retaining the outdated impetus theory of motion. The study's findings emphasize the importance of refining AI-driven tools to make them more effective in educational settings. Addressing both AI limitations and common misconceptions in physics can lead to improved educational outcomes. [ABSTRACT FROM AUTHOR]
Copyright of Contemporary Educational Technology is the property of Bastas Publications and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: ehh
DbLabel: Education Research Complete
An: 181418646
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Assessing AI's problem solving in physics: Analyzing reasoning, false positives and negatives through the force concept inventory.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Aldazharova%2C+Salima%22">Aldazharova, Salima</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Issayeva%2C+Gulnara%22">Issayeva, Gulnara</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Maxutov%2C+Samat%22">Maxutov, Samat</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Balta%2C+Nuri%22">Balta, Nuri</searchLink><relatesTo>2</relatesTo><i> baltanuri@gmail.com</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Contemporary+Educational+Technology%22">Contemporary Educational Technology</searchLink>. Oct2024, Vol. 16 Issue 4, p1-16. 16p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Physics+education%22">Physics education</searchLink><br />*<searchLink fieldCode="DE" term="%22Common+misconceptions%22">Common misconceptions</searchLink><br />*<searchLink fieldCode="DE" term="%22Educational+outcomes%22">Educational outcomes</searchLink><br /><searchLink fieldCode="DE" term="%22Newton's+laws+of+motion%22">Newton's laws of motion</searchLink><br /><searchLink fieldCode="DE" term="%22Generative+pre-trained+transformers%22">Generative pre-trained transformers</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study investigates the performance of GPT-4, an advanced AI model developed by OpenAI, on the force concept inventory (FCI) to evaluate its accuracy, reasoning patterns, and the occurrence of false positives and false negatives. GPT-4 was tasked with answering the FCI questions across multiple sessions. Key findings include GPT-4's proficiency in several FCI items, particularly those related to Newton's third law, achieving perfect scores on many items. However, it struggled significantly with questions involving the interpretation of figures and spatial reasoning, resulting in a higher occurrence of false negatives where the reasoning was correct, but the answers were incorrect. Additionally, GPT-4 displayed several conceptual errors, such as misunderstanding the effect of friction and retaining the outdated impetus theory of motion. The study's findings emphasize the importance of refining AI-driven tools to make them more effective in educational settings. Addressing both AI limitations and common misconceptions in physics can lead to improved educational outcomes. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Contemporary Educational Technology is the property of Bastas Publications and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=181418646
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.30935/cedtech/15592
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
        StartPage: 1
    Subjects:
      – SubjectFull: Physics education
        Type: general
      – SubjectFull: Common misconceptions
        Type: general
      – SubjectFull: Educational outcomes
        Type: general
      – SubjectFull: Newton's laws of motion
        Type: general
      – SubjectFull: Generative pre-trained transformers
        Type: general
    Titles:
      – TitleFull: Assessing AI's problem solving in physics: Analyzing reasoning, false positives and negatives through the force concept inventory.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Aldazharova, Salima
      – PersonEntity:
          Name:
            NameFull: Issayeva, Gulnara
      – PersonEntity:
          Name:
            NameFull: Maxutov, Samat
      – PersonEntity:
          Name:
            NameFull: Balta, Nuri
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Text: Oct2024
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1309517X
          Numbering:
            – Type: volume
              Value: 16
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Contemporary Educational Technology
              Type: main
ResultId 1