View in EDS

Comparing Performance of Methods to Deal with Differential Attrition in Lottery Based Evaluations

Saved in:
Bibliographic Details
Title: Comparing Performance of Methods to Deal with Differential Attrition in Lottery Based Evaluations
Language: English
Authors: Zamarro, Gema, Anderson, Kaitlin, Steele, Jennifer, Miller, Trey, Society for Research on Educational Effectiveness (SREE)
Source: Society for Research on Educational Effectiveness. 2016.
Availability: Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
Peer Reviewed: Y
Page Count: 6
Publication Date: 2016
Document Type: Reports - Research
Descriptors: Data Analysis, Student Attrition, Evaluation Methods, Evaluation Research, Probability, Weighted Scores, Randomized Controlled Trials, Control Groups, Experimental Groups, School Districts, School Statistics, Computation, Comparative Analysis, Scoring Rubrics, State Standards, Benchmarking, Statistical Studies, Predictor Variables
Geographic Terms: Oregon
Abstract: The purpose of this study is to study the performance of different methods (inverse probability weighting and estimation of informative bounds) to control for differential attrition by comparing the results of different methods using two datasets: an original dataset from Portland Public Schools (PPS) subject to high rates of differential attrition, and the expanded PPS and state level dataset that does not suffer as much from differential attrition. The main research questions are: (1) Do various methods (inverse probability weighting or estimation of informative bounds) adequately compensate for differential attrition in a random assignment evaluation; and (2) How do various assumptions within these methods affect the results? This study utilizes two datasets which will be compared using various methods: (1) PPS school district data: this data set experienced high levels of differential attrition. Attrition in the control group was about 24 percentage points higher than attrition in the treatment group; and (2) PPS school district data supplemented with Oregon Department of Education (ODE) State level data. This will serve as the benchmark with which to compare the district-level data that experienced more differential attrition. Once the data was supplemented, differential attrition was reduced to only about 6 percentage points.
Abstractor: ERIC
Number of References: 7
Entry Date: 2016
Accession Number: ED567205
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED567205
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED567205
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Comparing Performance of Methods to Deal with Differential Attrition in Lottery Based Evaluations
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Zamarro%2C+Gema%22">Zamarro, Gema</searchLink><br /><searchLink fieldCode="AR" term="%22Anderson%2C+Kaitlin%22">Anderson, Kaitlin</searchLink><br /><searchLink fieldCode="AR" term="%22Steele%2C+Jennifer%22">Steele, Jennifer</searchLink><br /><searchLink fieldCode="AR" term="%22Miller%2C+Trey%22">Miller, Trey</searchLink><br /><searchLink fieldCode="AR" term="%22Society+for+Research+on+Educational+Effectiveness+%28SREE%29%22">Society for Research on Educational Effectiveness (SREE)</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Society+for+Research+on+Educational+Effectiveness%22"><i>Society for Research on Educational Effectiveness</i></searchLink>. 2016.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 6
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2016
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Data+Analysis%22">Data Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Attrition%22">Student Attrition</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Research%22">Evaluation Research</searchLink><br /><searchLink fieldCode="DE" term="%22Probability%22">Probability</searchLink><br /><searchLink fieldCode="DE" term="%22Weighted+Scores%22">Weighted Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Randomized+Controlled+Trials%22">Randomized Controlled Trials</searchLink><br /><searchLink fieldCode="DE" term="%22Control+Groups%22">Control Groups</searchLink><br /><searchLink fieldCode="DE" term="%22Experimental+Groups%22">Experimental Groups</searchLink><br /><searchLink fieldCode="DE" term="%22School+Districts%22">School Districts</searchLink><br /><searchLink fieldCode="DE" term="%22School+Statistics%22">School Statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Computation%22">Computation</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring+Rubrics%22">Scoring Rubrics</searchLink><br /><searchLink fieldCode="DE" term="%22State+Standards%22">State Standards</searchLink><br /><searchLink fieldCode="DE" term="%22Benchmarking%22">Benchmarking</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Studies%22">Statistical Studies</searchLink><br /><searchLink fieldCode="DE" term="%22Predictor+Variables%22">Predictor Variables</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Oregon%22">Oregon</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The purpose of this study is to study the performance of different methods (inverse probability weighting and estimation of informative bounds) to control for differential attrition by comparing the results of different methods using two datasets: an original dataset from Portland Public Schools (PPS) subject to high rates of differential attrition, and the expanded PPS and state level dataset that does not suffer as much from differential attrition. The main research questions are: (1) Do various methods (inverse probability weighting or estimation of informative bounds) adequately compensate for differential attrition in a random assignment evaluation; and (2) How do various assumptions within these methods affect the results? This study utilizes two datasets which will be compared using various methods: (1) PPS school district data: this data set experienced high levels of differential attrition. Attrition in the control group was about 24 percentage points higher than attrition in the treatment group; and (2) PPS school district data supplemented with Oregon Department of Education (ODE) State level data. This will serve as the benchmark with which to compare the district-level data that experienced more differential attrition. Once the data was supplemented, differential attrition was reduced to only about 6 percentage points.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: ERIC
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 7
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2016
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED567205
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED567205
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 6
    Subjects:
      – SubjectFull: Data Analysis
        Type: general
      – SubjectFull: Student Attrition
        Type: general
      – SubjectFull: Evaluation Methods
        Type: general
      – SubjectFull: Evaluation Research
        Type: general
      – SubjectFull: Probability
        Type: general
      – SubjectFull: Weighted Scores
        Type: general
      – SubjectFull: Randomized Controlled Trials
        Type: general
      – SubjectFull: Control Groups
        Type: general
      – SubjectFull: Experimental Groups
        Type: general
      – SubjectFull: School Districts
        Type: general
      – SubjectFull: School Statistics
        Type: general
      – SubjectFull: Computation
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Scoring Rubrics
        Type: general
      – SubjectFull: State Standards
        Type: general
      – SubjectFull: Benchmarking
        Type: general
      – SubjectFull: Statistical Studies
        Type: general
      – SubjectFull: Predictor Variables
        Type: general
      – SubjectFull: Oregon
        Type: general
    Titles:
      – TitleFull: Comparing Performance of Methods to Deal with Differential Attrition in Lottery Based Evaluations
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Society for Research on Educational Effectiveness (SREE)
      – PersonEntity:
          Name:
            NameFull: Zamarro, Gema
      – PersonEntity:
          Name:
            NameFull: Anderson, Kaitlin
      – PersonEntity:
          Name:
            NameFull: Steele, Jennifer
      – PersonEntity:
          Name:
            NameFull: Miller, Trey
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2016
          Titles:
            – TitleFull: Society for Research on Educational Effectiveness
              Type: main
ResultId 1