View in EDS

'Phantom' Compositional Effects in English School Value-Added Measures: The Consequences of Random Baseline Measurement Error

Saved in:
Bibliographic Details
Title: 'Phantom' Compositional Effects in English School Value-Added Measures: The Consequences of Random Baseline Measurement Error
Language: English
Authors: Perry, Thomas (ORCID 0000-0002-6124-467X)
Source: Research Papers in Education. 2019 34(2):239-262.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 24
Publication Date: 2019
Document Type: Journal Articles
Reports - Descriptive
Education Level: Secondary Education
Descriptors: Value Added Models, Correlation, Individual Characteristics, Peer Influence, Error of Measurement, Simulation, Databases, Scores, Academic Ability, High Achievement, Selective Admission, Institutional Characteristics, Foreign Countries, Secondary School Students
Geographic Terms: United Kingdom (England)
DOI: 10.1080/02671522.2018.1424926
ISSN: 0267-1522
Abstract: A compositional effect is when pupil attainment is associated with the characteristics of their peers, over and above their own individual characteristics. Pupils at academically selective schools, for example, tend to out-perform similar-ability pupils who are educated with mixed-ability peers. Previous methodological studies however have shown that 'phantom' compositional effects can arise purely from measurement error. Through simulations using English National Pupil Database data, this paper demonstrates that moderate rates of measurement error in baseline scores can produce a systemic bias in value-added ('Progress') scores that favours higher ability pupils. This bias is large enough to explain the ostensibly superior performance of selective 'grammar' schools that is seen in the English data and create an appreciable bias across the school performance distribution. This paper describes how such biases can arise from 'regression attenuation' and discusses the more general methodological implications of the results. The paper concludes that the consequences of baseline measurement error on school value-added scores is greatly mitigated by controls for school-level prior attainment, but that the English 'Progress' value-added measures, as they do not control for prior attainment at school-level, are seriously biased and that the observed 'grammar school effect' is likely to be largely or wholly spurious.
Abstractor: As Provided
Number of References: 49
Entry Date: 2019
Accession Number: EJ1205325
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1205325
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: 'Phantom' Compositional Effects in English School Value-Added Measures: The Consequences of Random Baseline Measurement Error
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Perry%2C+Thomas%22">Perry, Thomas</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-6124-467X">0000-0002-6124-467X</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Research+Papers+in+Education%22"><i>Research Papers in Education</i></searchLink>. 2019 34(2):239-262.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 24
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2019
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Descriptive
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Value+Added+Models%22">Value Added Models</searchLink><br /><searchLink fieldCode="DE" term="%22Correlation%22">Correlation</searchLink><br /><searchLink fieldCode="DE" term="%22Individual+Characteristics%22">Individual Characteristics</searchLink><br /><searchLink fieldCode="DE" term="%22Peer+Influence%22">Peer Influence</searchLink><br /><searchLink fieldCode="DE" term="%22Error+of+Measurement%22">Error of Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22Simulation%22">Simulation</searchLink><br /><searchLink fieldCode="DE" term="%22Databases%22">Databases</searchLink><br /><searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Ability%22">Academic Ability</searchLink><br /><searchLink fieldCode="DE" term="%22High+Achievement%22">High Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Selective+Admission%22">Selective Admission</searchLink><br /><searchLink fieldCode="DE" term="%22Institutional+Characteristics%22">Institutional Characteristics</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Secondary+School+Students%22">Secondary School Students</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22United+Kingdom+%28England%29%22">United Kingdom (England)</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/02671522.2018.1424926
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0267-1522
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: A compositional effect is when pupil attainment is associated with the characteristics of their peers, over and above their own individual characteristics. Pupils at academically selective schools, for example, tend to out-perform similar-ability pupils who are educated with mixed-ability peers. Previous methodological studies however have shown that 'phantom' compositional effects can arise purely from measurement error. Through simulations using English National Pupil Database data, this paper demonstrates that moderate rates of measurement error in baseline scores can produce a systemic bias in value-added ('Progress') scores that favours higher ability pupils. This bias is large enough to explain the ostensibly superior performance of selective 'grammar' schools that is seen in the English data and create an appreciable bias across the school performance distribution. This paper describes how such biases can arise from 'regression attenuation' and discusses the more general methodological implications of the results. The paper concludes that the consequences of baseline measurement error on school value-added scores is greatly mitigated by controls for school-level prior attainment, but that the English 'Progress' value-added measures, as they do not control for prior attainment at school-level, are seriously biased and that the observed 'grammar school effect' is likely to be largely or wholly spurious.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 49
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2019
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1205325
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1205325
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/02671522.2018.1424926
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 24
        StartPage: 239
    Subjects:
      – SubjectFull: Value Added Models
        Type: general
      – SubjectFull: Correlation
        Type: general
      – SubjectFull: Individual Characteristics
        Type: general
      – SubjectFull: Peer Influence
        Type: general
      – SubjectFull: Error of Measurement
        Type: general
      – SubjectFull: Simulation
        Type: general
      – SubjectFull: Databases
        Type: general
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Academic Ability
        Type: general
      – SubjectFull: High Achievement
        Type: general
      – SubjectFull: Selective Admission
        Type: general
      – SubjectFull: Institutional Characteristics
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Secondary School Students
        Type: general
      – SubjectFull: United Kingdom (England)
        Type: general
    Titles:
      – TitleFull: 'Phantom' Compositional Effects in English School Value-Added Measures: The Consequences of Random Baseline Measurement Error
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Perry, Thomas
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2019
          Identifiers:
            – Type: issn-print
              Value: 0267-1522
          Numbering:
            – Type: volume
              Value: 34
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Research Papers in Education
              Type: main
ResultId 1