View in EDS

Is the Pen Mightier than the Keyboard? The Effect of Online Testing on Measured Student Achievement. Working Paper 190

Saved in:
Bibliographic Details
Title: Is the Pen Mightier than the Keyboard? The Effect of Online Testing on Measured Student Achievement. Working Paper 190
Language: English
Authors: Backes, Ben, Cowan, James, National Center for Analysis of Longitudinal Data in Education Research (CALDER) at American Institutes for Research
Source: National Center for Analysis of Longitudinal Data in Education Research (CALDER). 2018.
Availability: National Center for Analysis of Longitudinal Data in Education Research. American Institutes for Research, 1000 Thomas Jefferson Street NW, Washington, DC 20007. Tel: 202-403-5796; Fax: 202-403-6783; e-mail: info@caldercenter.org; Web site: https://caldercenter.org
Peer Reviewed: N
Page Count: 39
Publication Date: 2018
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305A170119
Document Type: Reports - Research
Education Level: Elementary Secondary Education
Descriptors: Computer Assisted Testing, Academic Achievement, Standardized Tests, Achievement Tests, Longitudinal Studies, Test Format, Comparative Analysis, Mathematics, Language Arts, Common Core State Standards, Public Schools, Elementary Secondary Education, Regression (Statistics), Scores, Statistical Analysis
Geographic Terms: Massachusetts
Abstract: Nearly two dozen states now administer online exams. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students' learning. We examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the fact that the state administers the same exam (PARCC) in online and offline formats each year during the transitional period. We find mode effects of about 0.10 standard deviations in math and 0.25 standard deviations in English language arts (ELA). Our preferred estimates of the effects of online testing in the second year of administration suggest that mode effects for second-time test takers were about one third as large as the first year in math and about half as large in ELA. There is little evidence of systematic variation in mode effects by student demographic groups, although on ELA tests they are larger for students scoring at the bottom of the achievement distribution.
Abstractor: As Provided
Number of References: 33
IES Funded: Yes
Entry Date: 2018
Accession Number: ED583636
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED583636
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED583636
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Is the Pen Mightier than the Keyboard? The Effect of Online Testing on Measured Student Achievement. Working Paper 190
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Backes%2C+Ben%22">Backes, Ben</searchLink><br /><searchLink fieldCode="AR" term="%22Cowan%2C+James%22">Cowan, James</searchLink><br /><searchLink fieldCode="AR" term="%22National+Center+for+Analysis+of+Longitudinal+Data+in+Education+Research+%28CALDER%29+at+American+Institutes+for+Research%22">National Center for Analysis of Longitudinal Data in Education Research (CALDER) at American Institutes for Research</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22National+Center+for+Analysis+of+Longitudinal+Data+in+Education+Research+%28CALDER%29%22"><i>National Center for Analysis of Longitudinal Data in Education Research (CALDER)</i></searchLink>. 2018.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: National Center for Analysis of Longitudinal Data in Education Research. American Institutes for Research, 1000 Thomas Jefferson Street NW, Washington, DC 20007. Tel: 202-403-5796; Fax: 202-403-6783; e-mail: info@caldercenter.org; Web site: https://caldercenter.org
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 39
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2018
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305A170119
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Achievement%22">Academic Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Standardized+Tests%22">Standardized Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Achievement+Tests%22">Achievement Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Longitudinal+Studies%22">Longitudinal Studies</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics%22">Mathematics</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Arts%22">Language Arts</searchLink><br /><searchLink fieldCode="DE" term="%22Common+Core+State+Standards%22">Common Core State Standards</searchLink><br /><searchLink fieldCode="DE" term="%22Public+Schools%22">Public Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink><br /><searchLink fieldCode="DE" term="%22Regression+%28Statistics%29%22">Regression (Statistics)</searchLink><br /><searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Analysis%22">Statistical Analysis</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Massachusetts%22">Massachusetts</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Nearly two dozen states now administer online exams. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students' learning. We examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the fact that the state administers the same exam (PARCC) in online and offline formats each year during the transitional period. We find mode effects of about 0.10 standard deviations in math and 0.25 standard deviations in English language arts (ELA). Our preferred estimates of the effects of online testing in the second year of administration suggest that mode effects for second-time test takers were about one third as large as the first year in math and about half as large in ELA. There is little evidence of systematic variation in mode effects by student demographic groups, although on ELA tests they are larger for students scoring at the bottom of the achievement distribution.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 33
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2018
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED583636
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED583636
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 39
    Subjects:
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Academic Achievement
        Type: general
      – SubjectFull: Standardized Tests
        Type: general
      – SubjectFull: Achievement Tests
        Type: general
      – SubjectFull: Longitudinal Studies
        Type: general
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Mathematics
        Type: general
      – SubjectFull: Language Arts
        Type: general
      – SubjectFull: Common Core State Standards
        Type: general
      – SubjectFull: Public Schools
        Type: general
      – SubjectFull: Elementary Secondary Education
        Type: general
      – SubjectFull: Regression (Statistics)
        Type: general
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Statistical Analysis
        Type: general
      – SubjectFull: Massachusetts
        Type: general
    Titles:
      – TitleFull: Is the Pen Mightier than the Keyboard? The Effect of Online Testing on Measured Student Achievement. Working Paper 190
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: National Center for Analysis of Longitudinal Data in Education Research (CALDER) at American Institutes for Research
      – PersonEntity:
          Name:
            NameFull: Backes, Ben
      – PersonEntity:
          Name:
            NameFull: Cowan, James
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 04
              Type: published
              Y: 2018
          Titles:
            – TitleFull: National Center for Analysis of Longitudinal Data in Education Research (CALDER)
              Type: main
ResultId 1