View in EDS

Is Online a Better Baseline? Comparing the Predictive Validity of Computer- and Paper-Based Tests. Working Paper No. 241-0820

Saved in:
Bibliographic Details
Title: Is Online a Better Baseline? Comparing the Predictive Validity of Computer- and Paper-Based Tests. Working Paper No. 241-0820
Language: English
Authors: Backes, Ben, Cowan, James, National Center for Analysis of Longitudinal Data in Education Research (CALDER) at American Institutes for Research
Source: National Center for Analysis of Longitudinal Data in Education Research (CALDER). 2020.
Availability: National Center for Analysis of Longitudinal Data in Education Research. American Institutes for Research, 1000 Thomas Jefferson Street NW, Washington, DC 20007. Tel: 202-403-5796; Fax: 202-403-6783; e-mail: info@caldercenter.org; Web site: https://caldercenter.org
Peer Reviewed: N
Page Count: 21
Publication Date: 2020
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305A170119
Document Type: Reports - Research
Education Level: Elementary Education
Junior High Schools
Middle Schools
Secondary Education
Descriptors: Predictive Validity, Test Validity, Computer Assisted Testing, Comparative Analysis, Standardized Tests, Achievement Tests, Elementary School Students, Middle School Students
Geographic Terms: Massachusetts
Assessment and Survey Identifiers: Massachusetts Comprehensive Assessment System
Abstract: Prior work has documented a substantial penalty associated with taking the Partnership for Assessment of Readiness for College and Careers (PARCC) online relative to on paper (Backes & Cowan, 2019). However, this penalty does not necessarily make online tests less useful. For example, it could be the case that computer literacy skills are correlated with students' future ability to navigate high school coursework, and thus more predictive of later outcomes. Using a statewide implementation of PARCC in Massachusetts, we test the relative predictive validity of online and paper tests. We are unable to detect a difference between the two and in most cases can rule out even modest differences. Finally, we estimate mode effects for the new Massachusetts statewide assessment. In contrast to the first years of PARCC implementation, we find very small mode effects, showing that it is possible to implement online assessments at scale without large online penalties.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2021
Accession Number: ED609698
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED609698
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED609698
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Is Online a Better Baseline? Comparing the Predictive Validity of Computer- and Paper-Based Tests. Working Paper No. 241-0820
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Backes%2C+Ben%22">Backes, Ben</searchLink><br /><searchLink fieldCode="AR" term="%22Cowan%2C+James%22">Cowan, James</searchLink><br /><searchLink fieldCode="AR" term="%22National+Center+for+Analysis+of+Longitudinal+Data+in+Education+Research+%28CALDER%29+at+American+Institutes+for+Research%22">National Center for Analysis of Longitudinal Data in Education Research (CALDER) at American Institutes for Research</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22National+Center+for+Analysis+of+Longitudinal+Data+in+Education+Research+%28CALDER%29%22"><i>National Center for Analysis of Longitudinal Data in Education Research (CALDER)</i></searchLink>. 2020.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: National Center for Analysis of Longitudinal Data in Education Research. American Institutes for Research, 1000 Thomas Jefferson Street NW, Washington, DC 20007. Tel: 202-403-5796; Fax: 202-403-6783; e-mail: info@caldercenter.org; Web site: https://caldercenter.org
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 21
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2020
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305A170119
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Predictive+Validity%22">Predictive Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Standardized+Tests%22">Standardized Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Achievement+Tests%22">Achievement Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+School+Students%22">Middle School Students</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Massachusetts%22">Massachusetts</searchLink>
– Name: SubjectThesaurus
  Label: Assessment and Survey Identifiers
  Group: Su
  Data: <searchLink fieldCode="SU" term="%22Massachusetts+Comprehensive+Assessment+System%22">Massachusetts Comprehensive Assessment System</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Prior work has documented a substantial penalty associated with taking the Partnership for Assessment of Readiness for College and Careers (PARCC) online relative to on paper (Backes & Cowan, 2019). However, this penalty does not necessarily make online tests less useful. For example, it could be the case that computer literacy skills are correlated with students' future ability to navigate high school coursework, and thus more predictive of later outcomes. Using a statewide implementation of PARCC in Massachusetts, we test the relative predictive validity of online and paper tests. We are unable to detect a difference between the two and in most cases can rule out even modest differences. Finally, we estimate mode effects for the new Massachusetts statewide assessment. In contrast to the first years of PARCC implementation, we find very small mode effects, showing that it is possible to implement online assessments at scale without large online penalties.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2021
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED609698
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED609698
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 21
    Subjects:
      – SubjectFull: Predictive Validity
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Standardized Tests
        Type: general
      – SubjectFull: Achievement Tests
        Type: general
      – SubjectFull: Elementary School Students
        Type: general
      – SubjectFull: Middle School Students
        Type: general
      – SubjectFull: Massachusetts
        Type: general
      – SubjectFull: Massachusetts Comprehensive Assessment System
        Type: general
    Titles:
      – TitleFull: Is Online a Better Baseline? Comparing the Predictive Validity of Computer- and Paper-Based Tests. Working Paper No. 241-0820
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: National Center for Analysis of Longitudinal Data in Education Research (CALDER) at American Institutes for Research
      – PersonEntity:
          Name:
            NameFull: Backes, Ben
      – PersonEntity:
          Name:
            NameFull: Cowan, James
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Type: published
              Y: 2020
          Titles:
            – TitleFull: National Center for Analysis of Longitudinal Data in Education Research (CALDER)
              Type: main
ResultId 1