View in EDS

Item Response Theory-Based Gaming Detection

Saved in:
Bibliographic Details
Title: Item Response Theory-Based Gaming Detection
Language: English
Authors: Huang, Yun, Dang, Steven, Richey, J. Elizabeth, Asher, Michael, Lobczowski, Nikki G., Chine, Danielle, McLaughlin, Elizabeth A., Harackiewicz, Judith M., Aleven, Vincent, Koedinger, Kenneth
Source: International Educational Data Mining Society. 2022.
Availability: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
Peer Reviewed: Y
Page Count: 12
Publication Date: 2022
Document Type: Speeches/Meeting Papers
Reports - Descriptive
Education Level: Junior High Schools
Middle Schools
Secondary Education
High Schools
Descriptors: Item Response Theory, Learner Engagement, Student Behavior, Student Motivation, Academic Achievement, Middle School Students, High School Students, Algebra, Learning Strategies
Abstract: Gaming the system, a behavior in which learners exploit a system's properties to make progress while avoiding learning, has frequently been shown to be associated with lower learning. However, when we applied a previously validated gaming detector across conditions in experiments with an algebra tutor, the detected gaming was not associated with learning, challenging its construct validity. Our iterative exploratory data analysis suggested that some contextual factors that varied across and within conditions might contribute to this lack of association. We present a latent variable model, "item response theory-based gaming detection" (IRT-GD), that accounts for contextual factors and estimates latent gaming tendencies as the degree of deviation from normative behaviors across contexts. Item response theory models, widely used in knowledge assessment, account for item difficulty in estimating latent student abilities: students are estimated as having higher ability when they can get harder items correct than when they only get easier items correct. Similarly, IRT-GD accounts for contextual factors in estimating latent gaming tendencies: students are estimated as having a higher gaming tendency when they game in less commonly gamed contexts than when they only game in more commonly gamed contexts. IRT-GD outperformed the original detector on three datasets in terms of the association with learning. IRT-GD also more accurately revealed intervention effects on gaming and revealed a correlation between gaming and perceived competence in math. Our approach is not only useful for others wanting to apply a gaming assessment in their context but is also generally applicable in creating robust behavioral measures. [For the full proceedings, see ED623995.]
Abstractor: As Provided
Entry Date: 2022
Accession Number: ED624075
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED624075
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED624075
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Item Response Theory-Based Gaming Detection
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Huang%2C+Yun%22">Huang, Yun</searchLink><br /><searchLink fieldCode="AR" term="%22Dang%2C+Steven%22">Dang, Steven</searchLink><br /><searchLink fieldCode="AR" term="%22Richey%2C+J%2E+Elizabeth%22">Richey, J. Elizabeth</searchLink><br /><searchLink fieldCode="AR" term="%22Asher%2C+Michael%22">Asher, Michael</searchLink><br /><searchLink fieldCode="AR" term="%22Lobczowski%2C+Nikki+G%2E%22">Lobczowski, Nikki G.</searchLink><br /><searchLink fieldCode="AR" term="%22Chine%2C+Danielle%22">Chine, Danielle</searchLink><br /><searchLink fieldCode="AR" term="%22McLaughlin%2C+Elizabeth+A%2E%22">McLaughlin, Elizabeth A.</searchLink><br /><searchLink fieldCode="AR" term="%22Harackiewicz%2C+Judith+M%2E%22">Harackiewicz, Judith M.</searchLink><br /><searchLink fieldCode="AR" term="%22Aleven%2C+Vincent%22">Aleven, Vincent</searchLink><br /><searchLink fieldCode="AR" term="%22Koedinger%2C+Kenneth%22">Koedinger, Kenneth</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Educational+Data+Mining+Society%22"><i>International Educational Data Mining Society</i></searchLink>. 2022.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 12
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Descriptive
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22High+Schools%22">High Schools</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Learner+Engagement%22">Learner Engagement</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Behavior%22">Student Behavior</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Motivation%22">Student Motivation</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Achievement%22">Academic Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+School+Students%22">Middle School Students</searchLink><br /><searchLink fieldCode="DE" term="%22High+School+Students%22">High School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Algebra%22">Algebra</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Strategies%22">Learning Strategies</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Gaming the system, a behavior in which learners exploit a system's properties to make progress while avoiding learning, has frequently been shown to be associated with lower learning. However, when we applied a previously validated gaming detector across conditions in experiments with an algebra tutor, the detected gaming was not associated with learning, challenging its construct validity. Our iterative exploratory data analysis suggested that some contextual factors that varied across and within conditions might contribute to this lack of association. We present a latent variable model, "item response theory-based gaming detection" (IRT-GD), that accounts for contextual factors and estimates latent gaming tendencies as the degree of deviation from normative behaviors across contexts. Item response theory models, widely used in knowledge assessment, account for item difficulty in estimating latent student abilities: students are estimated as having higher ability when they can get harder items correct than when they only get easier items correct. Similarly, IRT-GD accounts for contextual factors in estimating latent gaming tendencies: students are estimated as having a higher gaming tendency when they game in less commonly gamed contexts than when they only game in more commonly gamed contexts. IRT-GD outperformed the original detector on three datasets in terms of the association with learning. IRT-GD also more accurately revealed intervention effects on gaming and revealed a correlation between gaming and perceived competence in math. Our approach is not only useful for others wanting to apply a gaming assessment in their context but is also generally applicable in creating robust behavioral measures. [For the full proceedings, see ED623995.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED624075
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED624075
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
    Subjects:
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Learner Engagement
        Type: general
      – SubjectFull: Student Behavior
        Type: general
      – SubjectFull: Student Motivation
        Type: general
      – SubjectFull: Academic Achievement
        Type: general
      – SubjectFull: Middle School Students
        Type: general
      – SubjectFull: High School Students
        Type: general
      – SubjectFull: Algebra
        Type: general
      – SubjectFull: Learning Strategies
        Type: general
    Titles:
      – TitleFull: Item Response Theory-Based Gaming Detection
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Huang, Yun
      – PersonEntity:
          Name:
            NameFull: Dang, Steven
      – PersonEntity:
          Name:
            NameFull: Richey, J. Elizabeth
      – PersonEntity:
          Name:
            NameFull: Asher, Michael
      – PersonEntity:
          Name:
            NameFull: Lobczowski, Nikki G.
      – PersonEntity:
          Name:
            NameFull: Chine, Danielle
      – PersonEntity:
          Name:
            NameFull: McLaughlin, Elizabeth A.
      – PersonEntity:
          Name:
            NameFull: Harackiewicz, Judith M.
      – PersonEntity:
          Name:
            NameFull: Aleven, Vincent
      – PersonEntity:
          Name:
            NameFull: Koedinger, Kenneth
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2022
          Titles:
            – TitleFull: International Educational Data Mining Society
              Type: main
ResultId 1