View in EDS

Impact Evaluation of Progress Learning in the Douglas County School System

Saved in:
Bibliographic Details
Title: Impact Evaluation of Progress Learning in the Douglas County School System
Authors: Michael A. Cook, Jane Eisinger, Steven M. Ross, Johns Hopkins University, Center for Research and Reform in Education (CRRE)
Source: Center for Research and Reform in Education. 2024.
Availability: Johns Hopkins Center for Research and Reform in Education, 300 East Joppa Road Suite 500, Baltimore, MD 21286. Tel: 410-616-2338; Fax: 410-324-4444; Web site: https://education.jhu.edu/crre/
Peer Reviewed: Y
Page Count: 34
Publication Date: 2024
Document Type: Reports - Research
Tests/Questionnaires
Numerical/Quantitative Data
Education Level: Elementary Education
Grade 6
Intermediate Grades
Middle Schools
Grade 7
Junior High Schools
Secondary Education
Grade 8
Descriptors: Evaluation, Academic Achievement, Elementary School Mathematics, Language Arts, English Curriculum, Program Effectiveness, Teacher Attitudes, Grade 6, Grade 7, Grade 8, Influences, Learner Engagement, Program Implementation, Scores
Geographic Terms: Georgia
Abstract: The current study was a retrospective mixed-methods quasi-experimental design (QED) study to determine the effects of Progress Learning on Grades 6-8 mathematics and ELA achievement by comparison growth on the Georgia Milestones Mathematics and ELA assessments of students who received Progress Learning services, in relation to students that did not receive Progress Learning. Supplementary analyses examining the associations between Progress Learning usage metrics and achievement gains are also performed in this study. The results of the main impact analyses showed a positive and statistically significant impact of Progress Learning on student mathematics achievement, with treatment students outgaining comparison students by more than 4 points. The results of the main ELA impact analysis showed a directionally positive, though not statistically significant, impact on ELA achievement, with treatment students outgaining comparison students by more than 3 points. Effect sizes of these analyses ranged between 0.06 to 0.09 SDs, indicating small, though practically meaningful, program impacts of Progress Learning on student achievement, especially in mathematics. Usage analyses showed significant positive associations between student-level Progress Learning usage metrics and achievement gains. Correlations between average Progress Learning activity scores and achievement gains were of particular note, with observed correlations of magnitude above .4 in ELA and above .6 in mathematics. This gives preliminary evidence supporting modest to moderate predictive validity of Progress Learning activity scores in relation to Georgia Milestones scores. These associations remained significant and positive when controlling for prior achievement and demographics, using HLMs similar to those used in the main impact analyses.
Abstractor: As Provided
Entry Date: 2024
Access URL: https://jscholarship.library.jhu.edu/handle/1774.2/69242
Accession Number: ED642765
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED642765
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED642765
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Impact Evaluation of Progress Learning in the Douglas County School System
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Michael+A%2E+Cook%22">Michael A. Cook</searchLink><br /><searchLink fieldCode="AR" term="%22Jane+Eisinger%22">Jane Eisinger</searchLink><br /><searchLink fieldCode="AR" term="%22Steven+M%2E+Ross%22">Steven M. Ross</searchLink><br /><searchLink fieldCode="AR" term="%22Johns+Hopkins+University%2C+Center+for+Research+and+Reform+in+Education+%28CRRE%29%22">Johns Hopkins University, Center for Research and Reform in Education (CRRE)</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Center+for+Research+and+Reform+in+Education%22"><i>Center for Research and Reform in Education</i></searchLink>. 2024.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Johns Hopkins Center for Research and Reform in Education, 300 East Joppa Road Suite 500, Baltimore, MD 21286. Tel: 410-616-2338; Fax: 410-324-4444; Web site: https://education.jhu.edu/crre/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 34
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research<br />Tests/Questionnaires<br />Numerical/Quantitative Data
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+6%22">Grade 6</searchLink><br /><searchLink fieldCode="EL" term="%22Intermediate+Grades%22">Intermediate Grades</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+7%22">Grade 7</searchLink><br /><searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+8%22">Grade 8</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Evaluation%22">Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Achievement%22">Academic Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Mathematics%22">Elementary School Mathematics</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Arts%22">Language Arts</searchLink><br /><searchLink fieldCode="DE" term="%22English+Curriculum%22">English Curriculum</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Effectiveness%22">Program Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Attitudes%22">Teacher Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+6%22">Grade 6</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+7%22">Grade 7</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+8%22">Grade 8</searchLink><br /><searchLink fieldCode="DE" term="%22Influences%22">Influences</searchLink><br /><searchLink fieldCode="DE" term="%22Learner+Engagement%22">Learner Engagement</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Implementation%22">Program Implementation</searchLink><br /><searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Georgia%22">Georgia</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The current study was a retrospective mixed-methods quasi-experimental design (QED) study to determine the effects of Progress Learning on Grades 6-8 mathematics and ELA achievement by comparison growth on the Georgia Milestones Mathematics and ELA assessments of students who received Progress Learning services, in relation to students that did not receive Progress Learning. Supplementary analyses examining the associations between Progress Learning usage metrics and achievement gains are also performed in this study. The results of the main impact analyses showed a positive and statistically significant impact of Progress Learning on student mathematics achievement, with treatment students outgaining comparison students by more than 4 points. The results of the main ELA impact analysis showed a directionally positive, though not statistically significant, impact on ELA achievement, with treatment students outgaining comparison students by more than 3 points. Effect sizes of these analyses ranged between 0.06 to 0.09 SDs, indicating small, though practically meaningful, program impacts of Progress Learning on student achievement, especially in mathematics. Usage analyses showed significant positive associations between student-level Progress Learning usage metrics and achievement gains. Correlations between average Progress Learning activity scores and achievement gains were of particular note, with observed correlations of magnitude above .4 in ELA and above .6 in mathematics. This gives preliminary evidence supporting modest to moderate predictive validity of Progress Learning activity scores in relation to Georgia Milestones scores. These associations remained significant and positive when controlling for prior achievement and demographics, using HLMs similar to those used in the main impact analyses.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: URL
  Label: Access URL
  Group: URL
  Data: <link linkTarget="URL" linkTerm="https://jscholarship.library.jhu.edu/handle/1774.2/69242" linkWindow="_blank">https://jscholarship.library.jhu.edu/handle/1774.2/69242</link>
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED642765
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED642765
RecordInfo BibRecord:
  BibEntity:
    PhysicalDescription:
      Pagination:
        PageCount: 34
    Subjects:
      – SubjectFull: Evaluation
        Type: general
      – SubjectFull: Academic Achievement
        Type: general
      – SubjectFull: Elementary School Mathematics
        Type: general
      – SubjectFull: Language Arts
        Type: general
      – SubjectFull: English Curriculum
        Type: general
      – SubjectFull: Program Effectiveness
        Type: general
      – SubjectFull: Teacher Attitudes
        Type: general
      – SubjectFull: Grade 6
        Type: general
      – SubjectFull: Grade 7
        Type: general
      – SubjectFull: Grade 8
        Type: general
      – SubjectFull: Influences
        Type: general
      – SubjectFull: Learner Engagement
        Type: general
      – SubjectFull: Program Implementation
        Type: general
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Georgia
        Type: general
    Titles:
      – TitleFull: Impact Evaluation of Progress Learning in the Douglas County School System
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Johns Hopkins University, Center for Research and Reform in Education (CRRE)
      – PersonEntity:
          Name:
            NameFull: Michael A. Cook
      – PersonEntity:
          Name:
            NameFull: Jane Eisinger
      – PersonEntity:
          Name:
            NameFull: Steven M. Ross
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Titles:
            – TitleFull: Center for Research and Reform in Education
              Type: main
ResultId 1