Impact Evaluation of Progress Learning in the Douglas County School System
Saved in:
| Title: | Impact Evaluation of Progress Learning in the Douglas County School System |
|---|---|
| Authors: | Michael A. Cook, Jane Eisinger, Steven M. Ross, Johns Hopkins University, Center for Research and Reform in Education (CRRE) |
| Source: | Center for Research and Reform in Education. 2024. |
| Availability: | Johns Hopkins Center for Research and Reform in Education, 300 East Joppa Road Suite 500, Baltimore, MD 21286. Tel: 410-616-2338; Fax: 410-324-4444; Web site: https://education.jhu.edu/crre/ |
| Peer Reviewed: | Y |
| Page Count: | 34 |
| Publication Date: | 2024 |
| Document Type: | Reports - Research Tests/Questionnaires Numerical/Quantitative Data |
| Education Level: | Elementary Education Grade 6 Intermediate Grades Middle Schools Grade 7 Junior High Schools Secondary Education Grade 8 |
| Descriptors: | Evaluation, Academic Achievement, Elementary School Mathematics, Language Arts, English Curriculum, Program Effectiveness, Teacher Attitudes, Grade 6, Grade 7, Grade 8, Influences, Learner Engagement, Program Implementation, Scores |
| Geographic Terms: | Georgia |
| Abstract: | The current study was a retrospective mixed-methods quasi-experimental design (QED) study to determine the effects of Progress Learning on Grades 6-8 mathematics and ELA achievement by comparison growth on the Georgia Milestones Mathematics and ELA assessments of students who received Progress Learning services, in relation to students that did not receive Progress Learning. Supplementary analyses examining the associations between Progress Learning usage metrics and achievement gains are also performed in this study. The results of the main impact analyses showed a positive and statistically significant impact of Progress Learning on student mathematics achievement, with treatment students outgaining comparison students by more than 4 points. The results of the main ELA impact analysis showed a directionally positive, though not statistically significant, impact on ELA achievement, with treatment students outgaining comparison students by more than 3 points. Effect sizes of these analyses ranged between 0.06 to 0.09 SDs, indicating small, though practically meaningful, program impacts of Progress Learning on student achievement, especially in mathematics. Usage analyses showed significant positive associations between student-level Progress Learning usage metrics and achievement gains. Correlations between average Progress Learning activity scores and achievement gains were of particular note, with observed correlations of magnitude above .4 in ELA and above .6 in mathematics. This gives preliminary evidence supporting modest to moderate predictive validity of Progress Learning activity scores in relation to Georgia Milestones scores. These associations remained significant and positive when controlling for prior achievement and demographics, using HLMs similar to those used in the main impact analyses. |
| Abstractor: | As Provided |
| Entry Date: | 2024 |
| Access URL: | https://jscholarship.library.jhu.edu/handle/1774.2/69242 |
| Accession Number: | ED642765 |
| Database: | ERIC |
| Abstract: | The current study was a retrospective mixed-methods quasi-experimental design (QED) study to determine the effects of Progress Learning on Grades 6-8 mathematics and ELA achievement by comparison growth on the Georgia Milestones Mathematics and ELA assessments of students who received Progress Learning services, in relation to students that did not receive Progress Learning. Supplementary analyses examining the associations between Progress Learning usage metrics and achievement gains are also performed in this study. The results of the main impact analyses showed a positive and statistically significant impact of Progress Learning on student mathematics achievement, with treatment students outgaining comparison students by more than 4 points. The results of the main ELA impact analysis showed a directionally positive, though not statistically significant, impact on ELA achievement, with treatment students outgaining comparison students by more than 3 points. Effect sizes of these analyses ranged between 0.06 to 0.09 SDs, indicating small, though practically meaningful, program impacts of Progress Learning on student achievement, especially in mathematics. Usage analyses showed significant positive associations between student-level Progress Learning usage metrics and achievement gains. Correlations between average Progress Learning activity scores and achievement gains were of particular note, with observed correlations of magnitude above .4 in ELA and above .6 in mathematics. This gives preliminary evidence supporting modest to moderate predictive validity of Progress Learning activity scores in relation to Georgia Milestones scores. These associations remained significant and positive when controlling for prior achievement and demographics, using HLMs similar to those used in the main impact analyses. |
|---|
