View in EDS

Using Learning Progressions to Guide AI Feedback for Science Learning

Saved in:
Bibliographic Details
Title: Using Learning Progressions to Guide AI Feedback for Science Learning
Language: English
Authors: Xin Xia (ORCID 0009-0009-1717-8511), Nejla Yuruk (ORCID 0000-0001-9240-750X), Yun Wang (ORCID 0009-0004-6611-0752), Xiaoming Zhai (ORCID 0000-0003-4519-1931)
Source: Grantee Submission. 2026.
Peer Reviewed: Y
Page Count: 16
Publication Date: 2026
Sponsoring Agency: Institute of Education Sciences (ED)
National Science Foundation (NSF)
Contract Number: R305C240010
2101104
Document Type: Speeches/Meeting Papers
Reports - Research
Education Level: Junior High Schools
Middle Schools
Secondary Education
Descriptors: Artificial Intelligence, Technology Uses in Education, Learning Trajectories, Feedback (Response), Chemistry, Middle School Students, Scoring Rubrics, Interrater Reliability, Science Instruction, Formative Evaluation
DOI: 10.48550/arXiv.2603.03249
Abstract: Generative artificial intelligence (AI) offers scalable support for formative feedback, yet most AI-generated feedback relies on task-specific rubrics authored by domain experts. While effective, rubric authoring is time-consuming and limits scalability across instructional contexts. Learning progressions (LP) provide a theoretically grounded representation of students' developing understanding and may offer an alternative solution. This study examines whether an LP-driven rubric generation pipeline can produce AI-generated feedback comparable in quality to feedback guided by expert-authored task rubrics. We analyzed AI-generated feedback for written scientific explanations produced by 207 middle school students in a chemistry task. Two pipelines were compared: (a) feedback guided by a human expert-designed, task-specific rubric, and (b) feedback guided by a task-specific rubric automatically derived from a learning progression prior to grading and feedback generation. Two human coders evaluated feedback quality using a multi-dimensional rubric assessing "Clarity," "Accuracy," "Relevance," "Engagement and Motivation," and "Reflectiveness" (10 sub-dimensions). Inter-rater reliability was high, with percent agreement ranging from 89% to 100% and Cohen's κ values for estimable dimensions (κ = 0.66 to 0.88). Paired t-tests revealed no statistically significant differences between the two pipelines for "Clarity" (t₁ = 0.00, p₁ = 1.000; t₂ = 0.84, p₂ = 0.399), "Relevance" (t₁ = 0.28, p₁ = 0.782; t₂ = -0.58, p₂ = 0.565), "Engagement and Motivation" (t₁ = 0.50, p₁ = 0.618; t₂ = -0.58, p₂ = 0.565), or "Reflectiveness" (t = -0.45, p = 0.656). These findings suggest that the LP-driven rubric pipeline can serve as an alternative solution.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2026
Accession Number: ED681007
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: ED681007
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Using Learning Progressions to Guide AI Feedback for Science Learning
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Xin+Xia%22">Xin Xia</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0009-1717-8511">0009-0009-1717-8511</externalLink>)<br /><searchLink fieldCode="AR" term="%22Nejla+Yuruk%22">Nejla Yuruk</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-9240-750X">0000-0001-9240-750X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Yun+Wang%22">Yun Wang</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0004-6611-0752">0009-0004-6611-0752</externalLink>)<br /><searchLink fieldCode="AR" term="%22Xiaoming+Zhai%22">Xiaoming Zhai</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-4519-1931">0000-0003-4519-1931</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Grantee+Submission%22"><i>Grantee Submission</i></searchLink>. 2026.
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 16
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2026
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)<br />National Science Foundation (NSF)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305C240010<br />2101104
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Trajectories%22">Learning Trajectories</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Chemistry%22">Chemistry</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+School+Students%22">Middle School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring+Rubrics%22">Scoring Rubrics</searchLink><br /><searchLink fieldCode="DE" term="%22Interrater+Reliability%22">Interrater Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Instruction%22">Science Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Formative+Evaluation%22">Formative Evaluation</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.48550/arXiv.2603.03249
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Generative artificial intelligence (AI) offers scalable support for formative feedback, yet most AI-generated feedback relies on task-specific rubrics authored by domain experts. While effective, rubric authoring is time-consuming and limits scalability across instructional contexts. Learning progressions (LP) provide a theoretically grounded representation of students' developing understanding and may offer an alternative solution. This study examines whether an LP-driven rubric generation pipeline can produce AI-generated feedback comparable in quality to feedback guided by expert-authored task rubrics. We analyzed AI-generated feedback for written scientific explanations produced by 207 middle school students in a chemistry task. Two pipelines were compared: (a) feedback guided by a human expert-designed, task-specific rubric, and (b) feedback guided by a task-specific rubric automatically derived from a learning progression prior to grading and feedback generation. Two human coders evaluated feedback quality using a multi-dimensional rubric assessing "Clarity," "Accuracy," "Relevance," "Engagement and Motivation," and "Reflectiveness" (10 sub-dimensions). Inter-rater reliability was high, with percent agreement ranging from 89% to 100% and Cohen's κ values for estimable dimensions (κ = 0.66 to 0.88). Paired t-tests revealed no statistically significant differences between the two pipelines for "Clarity" (t₁ = 0.00, p₁ = 1.000; t₂ = 0.84, p₂ = 0.399), "Relevance" (t₁ = 0.28, p₁ = 0.782; t₂ = -0.58, p₂ = 0.565), "Engagement and Motivation" (t₁ = 0.50, p₁ = 0.618; t₂ = -0.58, p₂ = 0.565), or "Reflectiveness" (t = -0.45, p = 0.656). These findings suggest that the LP-driven rubric pipeline can serve as an alternative solution.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED681007
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED681007
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.48550/arXiv.2603.03249
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 16
    Subjects:
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
      – SubjectFull: Learning Trajectories
        Type: general
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Chemistry
        Type: general
      – SubjectFull: Middle School Students
        Type: general
      – SubjectFull: Scoring Rubrics
        Type: general
      – SubjectFull: Interrater Reliability
        Type: general
      – SubjectFull: Science Instruction
        Type: general
      – SubjectFull: Formative Evaluation
        Type: general
    Titles:
      – TitleFull: Using Learning Progressions to Guide AI Feedback for Science Learning
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Xin Xia
      – PersonEntity:
          Name:
            NameFull: Nejla Yuruk
      – PersonEntity:
          Name:
            NameFull: Yun Wang
      – PersonEntity:
          Name:
            NameFull: Xiaoming Zhai
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2026
          Titles:
            – TitleFull: Grantee Submission
              Type: main
ResultId 1