What about Tests Make Students Switch Off? Item-Level Predictors of Disengagement on the PISA
Saved in:
| Title: | What about Tests Make Students Switch Off? Item-Level Predictors of Disengagement on the PISA |
|---|---|
| Language: | English |
| Authors: | Juste M. Mehou, Brian C. Leventhal |
| Source: | Journal of Educational Measurement. 2026 63(1). |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 22 |
| Publication Date: | 2026 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Secondary Education |
| Descriptors: | Achievement Tests, Secondary School Students, Foreign Countries, International Assessment, Test Wiseness, Item Response Theory, Guessing (Tests), Predictor Variables, Learner Engagement, Cognitive Processes, Difficulty Level, Science Tests, Test Items, Mathematics Tests, Reading Tests, Adaptive Testing |
| Assessment and Survey Identifiers: | Program for International Student Assessment |
| DOI: | 10.1111/jedm.70034 |
| ISSN: | 0022-0655 1745-3984 |
| Abstract: | International assessments, such as the Programme for International Student Assessment (PISA), play a critical role in monitoring educational progress, evaluating teaching effectiveness, and supporting institutional accountability. The validity of these interpretations rests on the assumption that students exert sufficient effort and perform to the best of their abilities, condition necessary for tests to reflect students' true proficiency. However, researchers have documented concerns about low effort in low-stakes test contexts where results carry no direct consequences for examinees. Despite growing awareness of this issue, research examining low effort correlates in large-scale assessments like PISA remains limited. Moreover, existing approaches often overlook the correlation between low effort and ability, which can bias estimates of both constructs. The current work used an explanatory IRTree model that jointly estimated science ability and rapid-guessing propensity to examine item-level predictors of disengagement in PISA 2022 science, including Item Position, item interactivity, cognitive demand, and assessment sequencing. Interactive items decreased disengagement which rose with Item Position. Cognitive demand was the strongest predictor, with high-demand items sharply increasing disengagement. Also, science items were less likely to be rapidly guessed when students started with adaptive math or reading rather than science, even after controlling for position. As expected, disengagement was strongly negatively related to science ability, underscoring the value of jointly modeling proficiency and effort in low-stakes contexts. We discuss these results and explore avenues for future research. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1501263 |
| Database: | ERIC |
| Abstract: | International assessments, such as the Programme for International Student Assessment (PISA), play a critical role in monitoring educational progress, evaluating teaching effectiveness, and supporting institutional accountability. The validity of these interpretations rests on the assumption that students exert sufficient effort and perform to the best of their abilities, condition necessary for tests to reflect students' true proficiency. However, researchers have documented concerns about low effort in low-stakes test contexts where results carry no direct consequences for examinees. Despite growing awareness of this issue, research examining low effort correlates in large-scale assessments like PISA remains limited. Moreover, existing approaches often overlook the correlation between low effort and ability, which can bias estimates of both constructs. The current work used an explanatory IRTree model that jointly estimated science ability and rapid-guessing propensity to examine item-level predictors of disengagement in PISA 2022 science, including Item Position, item interactivity, cognitive demand, and assessment sequencing. Interactive items decreased disengagement which rose with Item Position. Cognitive demand was the strongest predictor, with high-demand items sharply increasing disengagement. Also, science items were less likely to be rapidly guessed when students started with adaptive math or reading rather than science, even after controlling for position. As expected, disengagement was strongly negatively related to science ability, underscoring the value of jointly modeling proficiency and effort in low-stakes contexts. We discuss these results and explore avenues for future research. |
|---|---|
| ISSN: | 0022-0655 1745-3984 |
| DOI: | 10.1111/jedm.70034 |
