View in EDS

Starting Seatwork Earlier as a Valid Measure of Student Engagement

Saved in:
Bibliographic Details
Title: Starting Seatwork Earlier as a Valid Measure of Student Engagement
Language: English
Authors: Ashish Gurung, Jionghao Lin, Zhongtian Huang, Conrad Borchers, Ryan S. Baker, Vincent Aleven, Kenneth R. Koedinger
Source: International Educational Data Mining Society. 2025.
Availability: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
Peer Reviewed: Y
Page Count: 14
Publication Date: 2025
Document Type: Speeches/Meeting Papers
Reports - Research
Education Level: High Schools
Secondary Education
Descriptors: Decision Making, Computer Software, Tutoring, Electronic Learning, Time on Task, Outcomes of Education, Student Motivation, Prediction, Models, Secondary School Students, Mathematics Instruction
Abstract: Prior work has developed a range of automated measures ("detectors") of student self-regulation and engagement from student log data. These measures have been successfully used to make discoveries about student learning. Here, we extend this line of research to an underexplored aspect of self-regulation: students' decisions about when to start and stop working on learning software during classwork. In the first of two analyses, we build on prior work on session-level measures (e.g., delayed start, early stop) to evaluate their reliability and predictive validity. We compute these measures from year-long log data from Cognitive Tutor for students in grades 8-12 (N = 222). Our findings show that these measures exhibit moderate to high month-to-month reliability (G > 0.75), comparable to or exceeding gaming-the-system behavior. Additionally, they enhance the prediction of final math scores beyond prior knowledge and gaming-the-system behaviors. The improvement in learning outcome predictions beyond time-on-task suggests they capture a broader motivational state tied to overall learning. The second analysis demonstrates the cross-system generalizability of these measures in i-Ready, where they predict state test scores for grade 7 students (N = 818). By leveraging log data, we introduce system-general naturally embedded measures that complement motivational surveys without extra instrumentation or disruption of instruction time. Our findings demonstrate the potential of session-level logs to mine valid and generalizable measures with broad applications in the predictive modeling of learning outcomes and analysis of learner self-regulation. [For the complete proceedings, see ED675583.]
Abstractor: As Provided
Entry Date: 2025
Accession Number: ED675676
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED675676
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED675676
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Starting Seatwork Earlier as a Valid Measure of Student Engagement
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Ashish+Gurung%22">Ashish Gurung</searchLink><br /><searchLink fieldCode="AR" term="%22Jionghao+Lin%22">Jionghao Lin</searchLink><br /><searchLink fieldCode="AR" term="%22Zhongtian+Huang%22">Zhongtian Huang</searchLink><br /><searchLink fieldCode="AR" term="%22Conrad+Borchers%22">Conrad Borchers</searchLink><br /><searchLink fieldCode="AR" term="%22Ryan+S%2E+Baker%22">Ryan S. Baker</searchLink><br /><searchLink fieldCode="AR" term="%22Vincent+Aleven%22">Vincent Aleven</searchLink><br /><searchLink fieldCode="AR" term="%22Kenneth+R%2E+Koedinger%22">Kenneth R. Koedinger</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Educational+Data+Mining+Society%22"><i>International Educational Data Mining Society</i></searchLink>. 2025.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 14
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22High+Schools%22">High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Decision+Making%22">Decision Making</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software%22">Computer Software</searchLink><br /><searchLink fieldCode="DE" term="%22Tutoring%22">Tutoring</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+Learning%22">Electronic Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Time+on+Task%22">Time on Task</searchLink><br /><searchLink fieldCode="DE" term="%22Outcomes+of+Education%22">Outcomes of Education</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Motivation%22">Student Motivation</searchLink><br /><searchLink fieldCode="DE" term="%22Prediction%22">Prediction</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Secondary+School+Students%22">Secondary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Instruction%22">Mathematics Instruction</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Prior work has developed a range of automated measures ("detectors") of student self-regulation and engagement from student log data. These measures have been successfully used to make discoveries about student learning. Here, we extend this line of research to an underexplored aspect of self-regulation: students' decisions about when to start and stop working on learning software during classwork. In the first of two analyses, we build on prior work on session-level measures (e.g., delayed start, early stop) to evaluate their reliability and predictive validity. We compute these measures from year-long log data from Cognitive Tutor for students in grades 8-12 (N = 222). Our findings show that these measures exhibit moderate to high month-to-month reliability (G > 0.75), comparable to or exceeding gaming-the-system behavior. Additionally, they enhance the prediction of final math scores beyond prior knowledge and gaming-the-system behaviors. The improvement in learning outcome predictions beyond time-on-task suggests they capture a broader motivational state tied to overall learning. The second analysis demonstrates the cross-system generalizability of these measures in i-Ready, where they predict state test scores for grade 7 students (N = 818). By leveraging log data, we introduce system-general naturally embedded measures that complement motivational surveys without extra instrumentation or disruption of instruction time. Our findings demonstrate the potential of session-level logs to mine valid and generalizable measures with broad applications in the predictive modeling of learning outcomes and analysis of learner self-regulation. [For the complete proceedings, see ED675583.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED675676
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED675676
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 14
    Subjects:
      – SubjectFull: Decision Making
        Type: general
      – SubjectFull: Computer Software
        Type: general
      – SubjectFull: Tutoring
        Type: general
      – SubjectFull: Electronic Learning
        Type: general
      – SubjectFull: Time on Task
        Type: general
      – SubjectFull: Outcomes of Education
        Type: general
      – SubjectFull: Student Motivation
        Type: general
      – SubjectFull: Prediction
        Type: general
      – SubjectFull: Models
        Type: general
      – SubjectFull: Secondary School Students
        Type: general
      – SubjectFull: Mathematics Instruction
        Type: general
    Titles:
      – TitleFull: Starting Seatwork Earlier as a Valid Measure of Student Engagement
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Ashish Gurung
      – PersonEntity:
          Name:
            NameFull: Jionghao Lin
      – PersonEntity:
          Name:
            NameFull: Zhongtian Huang
      – PersonEntity:
          Name:
            NameFull: Conrad Borchers
      – PersonEntity:
          Name:
            NameFull: Ryan S. Baker
      – PersonEntity:
          Name:
            NameFull: Vincent Aleven
      – PersonEntity:
          Name:
            NameFull: Kenneth R. Koedinger
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Titles:
            – TitleFull: International Educational Data Mining Society
              Type: main
ResultId 1