Classroom-Based Experimentation in a Digital Learning Platform
Saved in:
| Title: | Classroom-Based Experimentation in a Digital Learning Platform |
|---|---|
| Language: | English |
| Authors: | April Murphy, Digital Promise, Empirical Education Inc., Institute of Education Sciences (ED) |
| Source: | Digital Promise. 2026. |
| Availability: | Digital Promise. 1001 Connecticut Avenue NW Suite 935, Washington DC 20036. Tel: 202-450-3675; e-mail: contact@digitalpromise.org; Web site: https://digitalpromise.org/ |
| Peer Reviewed: | N |
| Page Count: | 13 |
| Publication Date: | 2026 |
| Document Type: | Reports - Evaluative |
| Education Level: | Elementary Education Grade 6 Intermediate Grades Middle Schools Grade 7 Junior High Schools Secondary Education Grade 8 |
| Descriptors: | Electronic Learning, Educational Technology, Technology Uses in Education, Educational Research, Mathematics Education, Grade 6, Grade 7, Grade 8, Mathematics Instruction, Classroom Environment, Intervention, Educational Experiments |
| Geographic Terms: | Delaware, South Dakota, Texas |
| Abstract: | Educational research often struggles to balance laboratory-style control with the real-world scale of authentic classrooms. This paper explores how classroom-embedded, digitally integrated experimentation addresses this tension by generating actionable evidence directly within digital learning environments. Using Carnegie Learning's MATHia intelligent tutoring system alongside UpGrade--an open-source A/B testing platform--the author highlights three diverse field trials encompassing nearly 100 experiments and hundreds of thousands of students. These include an XPRIZE study on localized personalization, an IES-funded reading readability initiative leveraging Large Language Models (LLMs), and an ongoing project evaluating targeted metacognitive prompts. From these large-scale applications, three foundational insights emerge for optimizing digital education research: seamlessly weaving interventions into daily instruction to eliminate teacher burden, capturing real-time learning metrics through intrinsic system-generated data, and prioritizing small, theory-driven manipulations to isolate specific cognitive mechanisms. While logistical constraints like sequence-dependent timing and varying state curricula present ongoing challenges, the paper concludes that embedding rigorous, focused experiments into adaptive software offers a scalable, non-disruptive framework. This approach simultaneously advances learning science theory and drives iterative, evidence-based development in educational technology. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2026 |
| Accession Number: | ED682372 |
| Database: | ERIC |
| Abstract: | Educational research often struggles to balance laboratory-style control with the real-world scale of authentic classrooms. This paper explores how classroom-embedded, digitally integrated experimentation addresses this tension by generating actionable evidence directly within digital learning environments. Using Carnegie Learning's MATHia intelligent tutoring system alongside UpGrade--an open-source A/B testing platform--the author highlights three diverse field trials encompassing nearly 100 experiments and hundreds of thousands of students. These include an XPRIZE study on localized personalization, an IES-funded reading readability initiative leveraging Large Language Models (LLMs), and an ongoing project evaluating targeted metacognitive prompts. From these large-scale applications, three foundational insights emerge for optimizing digital education research: seamlessly weaving interventions into daily instruction to eliminate teacher burden, capturing real-time learning metrics through intrinsic system-generated data, and prioritizing small, theory-driven manipulations to isolate specific cognitive mechanisms. While logistical constraints like sequence-dependent timing and varying state curricula present ongoing challenges, the paper concludes that embedding rigorous, focused experiments into adaptive software offers a scalable, non-disruptive framework. This approach simultaneously advances learning science theory and drives iterative, evidence-based development in educational technology. |
|---|
