View in EDS

Evaluating a Data-Driven Redesign Process for Intelligent Tutoring Systems

Saved in:
Bibliographic Details
Title: Evaluating a Data-Driven Redesign Process for Intelligent Tutoring Systems
Language: English
Authors: Qianru Lyu (ORCID 0000-0003-0650-1570), Conrad Borchers (ORCID 0000-0003-3437-8979), Meng Xia (ORCID 0000-0002-2676-9032), Karen Xiao (ORCID 0009-0001-9717-6111), Paulo F. Carvalho (ORCID 0000-0002-0449-3733), Kenneth R. Koedinger (ORCID 0000-0002-5850-4768), Vincent Aleven (ORCID 0000-0002-1581-6657)
Source: Grantee Submission. 2026Paper presented at the International Conference on Artificial Intelligence in Education (AIED 2026) (27th, Seoul, Republic of Korea, 2026).
Peer Reviewed: Y
Page Count: 10
Publication Date: 2026
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305A220386
Document Type: Speeches/Meeting Papers
Reports - Research
Education Level: Junior High Schools
Middle Schools
Secondary Education
Descriptors: Intelligent Tutoring Systems, Educational Technology, Design, Middle School Mathematics, Middle School Students, Time on Task, Mastery Learning, Learning Processes, Achievement Gains
Abstract: Past research has defined a general process for the data-driven redesign of educational technologies and has shown that in carefully-selected instances, this process can help make systems more effective. In the current work, we test the generality of the approach by applying it to four units of a middle-school mathematics intelligent tutoring system that were selected not based on suitability for redesign, as in previous work, but on topic. We tested whether the redesigned system was more effective than the original in a classroom study with 123 students. Although the learning gains did not differ between the conditions, students who used the Redesigned Tutor had more productive time-on-task, a larger number of skills practiced, and greater total knowledge mastery. The findings highlight the promise of data-driven redesign even when applied to instructional units "not" selected as likely to yield improvement, as evidence of the generality and wide applicability of the method.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2026
Accession Number: ED679484
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: ED679484
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Evaluating a Data-Driven Redesign Process for Intelligent Tutoring Systems
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Qianru+Lyu%22">Qianru Lyu</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-0650-1570">0000-0003-0650-1570</externalLink>)<br /><searchLink fieldCode="AR" term="%22Conrad+Borchers%22">Conrad Borchers</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3437-8979">0000-0003-3437-8979</externalLink>)<br /><searchLink fieldCode="AR" term="%22Meng+Xia%22">Meng Xia</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-2676-9032">0000-0002-2676-9032</externalLink>)<br /><searchLink fieldCode="AR" term="%22Karen+Xiao%22">Karen Xiao</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0001-9717-6111">0009-0001-9717-6111</externalLink>)<br /><searchLink fieldCode="AR" term="%22Paulo+F%2E+Carvalho%22">Paulo F. Carvalho</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0449-3733">0000-0002-0449-3733</externalLink>)<br /><searchLink fieldCode="AR" term="%22Kenneth+R%2E+Koedinger%22">Kenneth R. Koedinger</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-5850-4768">0000-0002-5850-4768</externalLink>)<br /><searchLink fieldCode="AR" term="%22Vincent+Aleven%22">Vincent Aleven</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-1581-6657">0000-0002-1581-6657</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Grantee+Submission%22"><i>Grantee Submission</i></searchLink>. 2026Paper presented at the International Conference on Artificial Intelligence in Education (AIED 2026) (27th, Seoul, Republic of Korea, 2026).
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 10
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2026
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305A220386
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Intelligent+Tutoring+Systems%22">Intelligent Tutoring Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Technology%22">Educational Technology</searchLink><br /><searchLink fieldCode="DE" term="%22Design%22">Design</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+School+Mathematics%22">Middle School Mathematics</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+School+Students%22">Middle School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Time+on+Task%22">Time on Task</searchLink><br /><searchLink fieldCode="DE" term="%22Mastery+Learning%22">Mastery Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Processes%22">Learning Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Achievement+Gains%22">Achievement Gains</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Past research has defined a general process for the data-driven redesign of educational technologies and has shown that in carefully-selected instances, this process can help make systems more effective. In the current work, we test the generality of the approach by applying it to four units of a middle-school mathematics intelligent tutoring system that were selected not based on suitability for redesign, as in previous work, but on topic. We tested whether the redesigned system was more effective than the original in a classroom study with 123 students. Although the learning gains did not differ between the conditions, students who used the Redesigned Tutor had more productive time-on-task, a larger number of skills practiced, and greater total knowledge mastery. The findings highlight the promise of data-driven redesign even when applied to instructional units "not" selected as likely to yield improvement, as evidence of the generality and wide applicability of the method.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED679484
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED679484
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 10
    Subjects:
      – SubjectFull: Intelligent Tutoring Systems
        Type: general
      – SubjectFull: Educational Technology
        Type: general
      – SubjectFull: Design
        Type: general
      – SubjectFull: Middle School Mathematics
        Type: general
      – SubjectFull: Middle School Students
        Type: general
      – SubjectFull: Time on Task
        Type: general
      – SubjectFull: Mastery Learning
        Type: general
      – SubjectFull: Learning Processes
        Type: general
      – SubjectFull: Achievement Gains
        Type: general
    Titles:
      – TitleFull: Evaluating a Data-Driven Redesign Process for Intelligent Tutoring Systems
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Qianru Lyu
      – PersonEntity:
          Name:
            NameFull: Conrad Borchers
      – PersonEntity:
          Name:
            NameFull: Meng Xia
      – PersonEntity:
          Name:
            NameFull: Karen Xiao
      – PersonEntity:
          Name:
            NameFull: Paulo F. Carvalho
      – PersonEntity:
          Name:
            NameFull: Kenneth R. Koedinger
      – PersonEntity:
          Name:
            NameFull: Vincent Aleven
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2026
          Titles:
            – TitleFull: Grantee Submission
              Type: main
ResultId 1