Are Online and Paper Tests Comparable? Evidence from Statewide K-12 Tests
Saved in:
| Title: | Are Online and Paper Tests Comparable? Evidence from Statewide K-12 Tests |
|---|---|
| Language: | English |
| Authors: | Ben Backes, James Cowan |
| Source: | Grantee Submission. 2024. |
| Peer Reviewed: | Y |
| Page Count: | 30 |
| Publication Date: | 2024 |
| Sponsoring Agency: | Institute of Education Sciences (ED) National Center for Analysis of Longitudinal Data in Education Research (CALDER) at American Institutes for Research (AIR) |
| Contract Number: | R305A170119 |
| Document Type: | Reports - Research |
| Education Level: | Elementary Secondary Education |
| Descriptors: | Computer Assisted Testing, Test Format, Differences, Academic Achievement, State Programs, Standardized Tests, Elementary Secondary Education, Grade Prediction |
| Geographic Terms: | Massachusetts |
| Assessment and Survey Identifiers: | Massachusetts Comprehensive Assessment System |
| DOI: | 10.1080/08957347.2024.2311933 |
| Abstract: | We investigate two research questions using a recent statewide transition from paper to computer-based testing: first, the extent to which test mode effects found in prior studies can be eliminated in large-scale administration; and second, the degree to which online and paper assessments offer different information about underlying student ability. In contrast to the first test transition in Massachusetts, we find very small mode effects for a more recent transition, which may be attributable to an additional step matching on observable characteristics in the equating process. Second, we investigate the predictive evidence of validity for paper and online tests for predictions of future test scores and grades. We generally find minimal differences for the extent to which scores on paper tests can differentially predict future online versus paper test scores. Finally, online and paper test scores are similarly predictive of future grade point average. We conclude that the online test penalty can vary substantially by test and that extreme care should be taken when administering online tests to some students and paper tests to others. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2024 |
| Accession Number: | ED647304 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | We investigate two research questions using a recent statewide transition from paper to computer-based testing: first, the extent to which test mode effects found in prior studies can be eliminated in large-scale administration; and second, the degree to which online and paper assessments offer different information about underlying student ability. In contrast to the first test transition in Massachusetts, we find very small mode effects for a more recent transition, which may be attributable to an additional step matching on observable characteristics in the equating process. Second, we investigate the predictive evidence of validity for paper and online tests for predictions of future test scores and grades. We generally find minimal differences for the extent to which scores on paper tests can differentially predict future online versus paper test scores. Finally, online and paper test scores are similarly predictive of future grade point average. We conclude that the online test penalty can vary substantially by test and that extreme care should be taken when administering online tests to some students and paper tests to others. |
|---|---|
| DOI: | 10.1080/08957347.2024.2311933 |
