Utilizing Response Time for Item Selection in On-the-Fly Multistage Adaptive Testing for PISA Assessment
Saved in:
| Title: | Utilizing Response Time for Item Selection in On-the-Fly Multistage Adaptive Testing for PISA Assessment |
|---|---|
| Language: | English |
| Authors: | Xiuxiu Tang (ORCID |
| Source: | Journal of Educational Measurement. 2025 62(3):468-495. |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 28 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Secondary Education |
| Descriptors: | Reaction Time, Test Items, Achievement Tests, Foreign Countries, Secondary School Students, International Assessment, Adaptive Testing, Reading Tests, Accuracy, Test Length, Item Banks, Computer Assisted Testing |
| Assessment and Survey Identifiers: | Program for International Student Assessment |
| DOI: | 10.1111/jedm.12403 |
| ISSN: | 0022-0655 1745-3984 |
| Abstract: | Multistage adaptive testing (MST) has been recently adopted for international large-scale assessments such as Programme for International Student Assessment (PISA). MST offers improved measurement efficiency over traditional nonadaptive tests and improved practical convenience over single-item-adaptive computerized adaptive testing (CAT). As a third alternative adaptive test design to MST and CAT, Zheng and Chang proposed the "on-the-fly multistage adaptive testing" (OMST), which combines the benefits of MST and CAT and offsets their limitations. In this study, we adopted the OMST design while also incorporating response time (RT) in item selection. Via simulations emulating the PISA 2018 reading test, including using the real item attributes and replicating PISA 2018 reading test's MST design, we compared the performance of our OMST designs against the simulated MST design in (1) measurement accuracy of test takers' ability, (2) test time efficiency and consistency, and (3) expected gains in precision by design. We also investigated the performance of OMST in item bank usage and constraints management. Results show great potential for the proposed RT-incorporated OMST designs to be used for PISA and potentially other international large-scale assessments. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1487175 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Multistage adaptive testing (MST) has been recently adopted for international large-scale assessments such as Programme for International Student Assessment (PISA). MST offers improved measurement efficiency over traditional nonadaptive tests and improved practical convenience over single-item-adaptive computerized adaptive testing (CAT). As a third alternative adaptive test design to MST and CAT, Zheng and Chang proposed the "on-the-fly multistage adaptive testing" (OMST), which combines the benefits of MST and CAT and offsets their limitations. In this study, we adopted the OMST design while also incorporating response time (RT) in item selection. Via simulations emulating the PISA 2018 reading test, including using the real item attributes and replicating PISA 2018 reading test's MST design, we compared the performance of our OMST designs against the simulated MST design in (1) measurement accuracy of test takers' ability, (2) test time efficiency and consistency, and (3) expected gains in precision by design. We also investigated the performance of OMST in item bank usage and constraints management. Results show great potential for the proposed RT-incorporated OMST designs to be used for PISA and potentially other international large-scale assessments. |
|---|---|
| ISSN: | 0022-0655 1745-3984 |
| DOI: | 10.1111/jedm.12403 |
