Using Learning Progressions to Guide AI Feedback for Science Learning
Saved in:
| Title: | Using Learning Progressions to Guide AI Feedback for Science Learning |
|---|---|
| Language: | English |
| Authors: | Xin Xia (ORCID |
| Source: | Grantee Submission. 2026. |
| Peer Reviewed: | Y |
| Page Count: | 16 |
| Publication Date: | 2026 |
| Sponsoring Agency: | Institute of Education Sciences (ED) National Science Foundation (NSF) |
| Contract Number: | R305C240010 2101104 |
| Document Type: | Speeches/Meeting Papers Reports - Research |
| Education Level: | Junior High Schools Middle Schools Secondary Education |
| Descriptors: | Artificial Intelligence, Technology Uses in Education, Learning Trajectories, Feedback (Response), Chemistry, Middle School Students, Scoring Rubrics, Interrater Reliability, Science Instruction, Formative Evaluation |
| DOI: | 10.48550/arXiv.2603.03249 |
| Abstract: | Generative artificial intelligence (AI) offers scalable support for formative feedback, yet most AI-generated feedback relies on task-specific rubrics authored by domain experts. While effective, rubric authoring is time-consuming and limits scalability across instructional contexts. Learning progressions (LP) provide a theoretically grounded representation of students' developing understanding and may offer an alternative solution. This study examines whether an LP-driven rubric generation pipeline can produce AI-generated feedback comparable in quality to feedback guided by expert-authored task rubrics. We analyzed AI-generated feedback for written scientific explanations produced by 207 middle school students in a chemistry task. Two pipelines were compared: (a) feedback guided by a human expert-designed, task-specific rubric, and (b) feedback guided by a task-specific rubric automatically derived from a learning progression prior to grading and feedback generation. Two human coders evaluated feedback quality using a multi-dimensional rubric assessing "Clarity," "Accuracy," "Relevance," "Engagement and Motivation," and "Reflectiveness" (10 sub-dimensions). Inter-rater reliability was high, with percent agreement ranging from 89% to 100% and Cohen's κ values for estimable dimensions (κ = 0.66 to 0.88). Paired t-tests revealed no statistically significant differences between the two pipelines for "Clarity" (t₁ = 0.00, p₁ = 1.000; t₂ = 0.84, p₂ = 0.399), "Relevance" (t₁ = 0.28, p₁ = 0.782; t₂ = -0.58, p₂ = 0.565), "Engagement and Motivation" (t₁ = 0.50, p₁ = 0.618; t₂ = -0.58, p₂ = 0.565), or "Reflectiveness" (t = -0.45, p = 0.656). These findings suggest that the LP-driven rubric pipeline can serve as an alternative solution. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2026 |
| Accession Number: | ED681007 |
| Database: | ERIC |
| Abstract: | Generative artificial intelligence (AI) offers scalable support for formative feedback, yet most AI-generated feedback relies on task-specific rubrics authored by domain experts. While effective, rubric authoring is time-consuming and limits scalability across instructional contexts. Learning progressions (LP) provide a theoretically grounded representation of students' developing understanding and may offer an alternative solution. This study examines whether an LP-driven rubric generation pipeline can produce AI-generated feedback comparable in quality to feedback guided by expert-authored task rubrics. We analyzed AI-generated feedback for written scientific explanations produced by 207 middle school students in a chemistry task. Two pipelines were compared: (a) feedback guided by a human expert-designed, task-specific rubric, and (b) feedback guided by a task-specific rubric automatically derived from a learning progression prior to grading and feedback generation. Two human coders evaluated feedback quality using a multi-dimensional rubric assessing "Clarity," "Accuracy," "Relevance," "Engagement and Motivation," and "Reflectiveness" (10 sub-dimensions). Inter-rater reliability was high, with percent agreement ranging from 89% to 100% and Cohen's κ values for estimable dimensions (κ = 0.66 to 0.88). Paired t-tests revealed no statistically significant differences between the two pipelines for "Clarity" (t₁ = 0.00, p₁ = 1.000; t₂ = 0.84, p₂ = 0.399), "Relevance" (t₁ = 0.28, p₁ = 0.782; t₂ = -0.58, p₂ = 0.565), "Engagement and Motivation" (t₁ = 0.50, p₁ = 0.618; t₂ = -0.58, p₂ = 0.565), or "Reflectiveness" (t = -0.45, p = 0.656). These findings suggest that the LP-driven rubric pipeline can serve as an alternative solution. |
|---|---|
| DOI: | 10.48550/arXiv.2603.03249 |
