Is Online a Better Baseline? Comparing the Predictive Validity of Computer- and Paper-Based Tests. Working Paper No. 241-0820
Saved in:
| Title: | Is Online a Better Baseline? Comparing the Predictive Validity of Computer- and Paper-Based Tests. Working Paper No. 241-0820 |
|---|---|
| Language: | English |
| Authors: | Backes, Ben, Cowan, James, National Center for Analysis of Longitudinal Data in Education Research (CALDER) at American Institutes for Research |
| Source: | National Center for Analysis of Longitudinal Data in Education Research (CALDER). 2020. |
| Availability: | National Center for Analysis of Longitudinal Data in Education Research. American Institutes for Research, 1000 Thomas Jefferson Street NW, Washington, DC 20007. Tel: 202-403-5796; Fax: 202-403-6783; e-mail: info@caldercenter.org; Web site: https://caldercenter.org |
| Peer Reviewed: | N |
| Page Count: | 21 |
| Publication Date: | 2020 |
| Sponsoring Agency: | Institute of Education Sciences (ED) |
| Contract Number: | R305A170119 |
| Document Type: | Reports - Research |
| Education Level: | Elementary Education Junior High Schools Middle Schools Secondary Education |
| Descriptors: | Predictive Validity, Test Validity, Computer Assisted Testing, Comparative Analysis, Standardized Tests, Achievement Tests, Elementary School Students, Middle School Students |
| Geographic Terms: | Massachusetts |
| Assessment and Survey Identifiers: | Massachusetts Comprehensive Assessment System |
| Abstract: | Prior work has documented a substantial penalty associated with taking the Partnership for Assessment of Readiness for College and Careers (PARCC) online relative to on paper (Backes & Cowan, 2019). However, this penalty does not necessarily make online tests less useful. For example, it could be the case that computer literacy skills are correlated with students' future ability to navigate high school coursework, and thus more predictive of later outcomes. Using a statewide implementation of PARCC in Massachusetts, we test the relative predictive validity of online and paper tests. We are unable to detect a difference between the two and in most cases can rule out even modest differences. Finally, we estimate mode effects for the new Massachusetts statewide assessment. In contrast to the first years of PARCC implementation, we find very small mode effects, showing that it is possible to implement online assessments at scale without large online penalties. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2021 |
| Accession Number: | ED609698 |
| Database: | ERIC |
| Abstract: | Prior work has documented a substantial penalty associated with taking the Partnership for Assessment of Readiness for College and Careers (PARCC) online relative to on paper (Backes & Cowan, 2019). However, this penalty does not necessarily make online tests less useful. For example, it could be the case that computer literacy skills are correlated with students' future ability to navigate high school coursework, and thus more predictive of later outcomes. Using a statewide implementation of PARCC in Massachusetts, we test the relative predictive validity of online and paper tests. We are unable to detect a difference between the two and in most cases can rule out even modest differences. Finally, we estimate mode effects for the new Massachusetts statewide assessment. In contrast to the first years of PARCC implementation, we find very small mode effects, showing that it is possible to implement online assessments at scale without large online penalties. |
|---|
