View in EDS

Gaze Patterns Enhance Response Prediction: More than Correct or Incorrect

Saved in:
Bibliographic Details
Title: Gaze Patterns Enhance Response Prediction: More than Correct or Incorrect
Language: English
Authors: Becker, Sebastian (ORCID 0000-0002-2461-0992), Küchemann, Stefan, Klein, Pascal (ORCID 0000-0003-3023-1478), Lichtenberger, Andreas (ORCID 0000-0001-7739-3904), Kuhn, Jochen
Source: Physical Review Physics Education Research. Jul-Dec 2022 18(2).
Availability: American Physical Society. One Physics Ellipse 4th Floor, College Park, MD 20740-3844. Tel: 301-209-3200; Fax: 301-209-0865; e-mail: assocpub@aps.org; Web site: http://prst-per.aps.org
Peer Reviewed: Y
Page Count: 11
Publication Date: 2022
Document Type: Journal Articles
Reports - Research
Education Level: High Schools
Secondary Education
Descriptors: Foreign Countries, High School Students, Eye Movements, Visual Stimuli, Attention, Predictor Variables, Responses, Accuracy, Problem Solving, Motion, Biomechanics
Geographic Terms: Germany, Switzerland
DOI: 10.1103/PhysRevPhysEducRes.18.020107
ISSN: 2469-9896
Abstract: Eye tracking enables the reconstruction of eye movements and thus the analysis of visual information selection and integration processes during problem solving. In this way, learner-specific difficulties can be identified and problem-solving process can be adapted accordingly. For such an adaptation, the prediction of response behavior plays a crucial role. To predict whether a problem is solved correctly or incorrectly, the segmentation of the visual stimulus into specific areas of interest (AOIs) is particularly crucial for the quality of a prediction based on eye-tracking data. In the study presented here, the gaze data of N=115 students were analyzed while solving the Test of Understanding Graphs in Kinematics (TUG-K), a validated test instrument whose items include graphs of position, velocity, and acceleration versus time. For selected items, response accuracy was predicted based on visual attention using multiple logistic regression analysis, examining the influence of AOI segmentation. The prediction quality could be significantly improved when the diagram was not considered as contiguous AOI, but when it was divided into solution-relevant and solution-irrelevant areas. To verify that the AOIs selected by the regression algorithm are indeed relevant to the solution process, an expert rating was performed, which showed moderate to good agreement between the AOIs rated by the experts as relevant to the correct solution and the AOIs selected by the algorithm. There are also pairs of items in the TUG-K that require the same mathematical solution procedure but differ in the physical context. This opened the possibility to investigate a new approach. Based on response accuracy and allocation of visual attention to one item, the response accuracy of the other item of the pair was predicted. It could be shown that the prediction quality based on visual attention was significantly higher than the prediction based on response accuracy. This demonstrates the added value of collecting process-based data versus product-based data for prediction and thus for learner-specific adaptation. The results of this study indicate, first, that only certain areas are crucial for a correct solution when extracting information from diagrams and, second, that the application of mathematical procedures plays a crucial role in interpreting graphs of different physics quantities. These findings thus provide insight into the visual strategies involved in interpreting kinematic diagrams and can also serve as a basis for eye tracking-based adaptation of problem-solving processes, in which adaptation can occur even before an incorrect answer is given.
Abstractor: As Provided
Entry Date: 2022
Accession Number: EJ1354783
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1354783
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Gaze Patterns Enhance Response Prediction: More than Correct or Incorrect
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Becker%2C+Sebastian%22">Becker, Sebastian</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-2461-0992">0000-0002-2461-0992</externalLink>)<br /><searchLink fieldCode="AR" term="%22Küchemann%2C+Stefan%22">Küchemann, Stefan</searchLink><br /><searchLink fieldCode="AR" term="%22Klein%2C+Pascal%22">Klein, Pascal</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3023-1478">0000-0003-3023-1478</externalLink>)<br /><searchLink fieldCode="AR" term="%22Lichtenberger%2C+Andreas%22">Lichtenberger, Andreas</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-7739-3904">0000-0001-7739-3904</externalLink>)<br /><searchLink fieldCode="AR" term="%22Kuhn%2C+Jochen%22">Kuhn, Jochen</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Physical+Review+Physics+Education+Research%22"><i>Physical Review Physics Education Research</i></searchLink>. Jul-Dec 2022 18(2).
– Name: Avail
  Label: Availability
  Group: Avail
  Data: American Physical Society. One Physics Ellipse 4th Floor, College Park, MD 20740-3844. Tel: 301-209-3200; Fax: 301-209-0865; e-mail: assocpub@aps.org; Web site: http://prst-per.aps.org
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 11
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22High+Schools%22">High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22High+School+Students%22">High School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Eye+Movements%22">Eye Movements</searchLink><br /><searchLink fieldCode="DE" term="%22Visual+Stimuli%22">Visual Stimuli</searchLink><br /><searchLink fieldCode="DE" term="%22Attention%22">Attention</searchLink><br /><searchLink fieldCode="DE" term="%22Predictor+Variables%22">Predictor Variables</searchLink><br /><searchLink fieldCode="DE" term="%22Responses%22">Responses</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Solving%22">Problem Solving</searchLink><br /><searchLink fieldCode="DE" term="%22Motion%22">Motion</searchLink><br /><searchLink fieldCode="DE" term="%22Biomechanics%22">Biomechanics</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Germany%22">Germany</searchLink><br /><searchLink fieldCode="DE" term="%22Switzerland%22">Switzerland</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1103/PhysRevPhysEducRes.18.020107
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2469-9896
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Eye tracking enables the reconstruction of eye movements and thus the analysis of visual information selection and integration processes during problem solving. In this way, learner-specific difficulties can be identified and problem-solving process can be adapted accordingly. For such an adaptation, the prediction of response behavior plays a crucial role. To predict whether a problem is solved correctly or incorrectly, the segmentation of the visual stimulus into specific areas of interest (AOIs) is particularly crucial for the quality of a prediction based on eye-tracking data. In the study presented here, the gaze data of N=115 students were analyzed while solving the Test of Understanding Graphs in Kinematics (TUG-K), a validated test instrument whose items include graphs of position, velocity, and acceleration versus time. For selected items, response accuracy was predicted based on visual attention using multiple logistic regression analysis, examining the influence of AOI segmentation. The prediction quality could be significantly improved when the diagram was not considered as contiguous AOI, but when it was divided into solution-relevant and solution-irrelevant areas. To verify that the AOIs selected by the regression algorithm are indeed relevant to the solution process, an expert rating was performed, which showed moderate to good agreement between the AOIs rated by the experts as relevant to the correct solution and the AOIs selected by the algorithm. There are also pairs of items in the TUG-K that require the same mathematical solution procedure but differ in the physical context. This opened the possibility to investigate a new approach. Based on response accuracy and allocation of visual attention to one item, the response accuracy of the other item of the pair was predicted. It could be shown that the prediction quality based on visual attention was significantly higher than the prediction based on response accuracy. This demonstrates the added value of collecting process-based data versus product-based data for prediction and thus for learner-specific adaptation. The results of this study indicate, first, that only certain areas are crucial for a correct solution when extracting information from diagrams and, second, that the application of mathematical procedures plays a crucial role in interpreting graphs of different physics quantities. These findings thus provide insight into the visual strategies involved in interpreting kinematic diagrams and can also serve as a basis for eye tracking-based adaptation of problem-solving processes, in which adaptation can occur even before an incorrect answer is given.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1354783
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1354783
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1103/PhysRevPhysEducRes.18.020107
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: High School Students
        Type: general
      – SubjectFull: Eye Movements
        Type: general
      – SubjectFull: Visual Stimuli
        Type: general
      – SubjectFull: Attention
        Type: general
      – SubjectFull: Predictor Variables
        Type: general
      – SubjectFull: Responses
        Type: general
      – SubjectFull: Accuracy
        Type: general
      – SubjectFull: Problem Solving
        Type: general
      – SubjectFull: Motion
        Type: general
      – SubjectFull: Biomechanics
        Type: general
      – SubjectFull: Germany
        Type: general
      – SubjectFull: Switzerland
        Type: general
    Titles:
      – TitleFull: Gaze Patterns Enhance Response Prediction: More than Correct or Incorrect
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Becker, Sebastian
      – PersonEntity:
          Name:
            NameFull: Küchemann, Stefan
      – PersonEntity:
          Name:
            NameFull: Klein, Pascal
      – PersonEntity:
          Name:
            NameFull: Lichtenberger, Andreas
      – PersonEntity:
          Name:
            NameFull: Kuhn, Jochen
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-electronic
              Value: 2469-9896
          Numbering:
            – Type: volume
              Value: 18
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Physical Review Physics Education Research
              Type: main
ResultId 1