Parametric Bootstrap Mantel-Haenszel Statistic for Aggregated Testlet Effects
Saved in:
| Title: | Parametric Bootstrap Mantel-Haenszel Statistic for Aggregated Testlet Effects |
|---|---|
| Language: | English |
| Authors: | Youn Seon Lim (ORCID |
| Source: | Journal of Educational Measurement. 2025 62(4):503-530. |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 28 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Secondary Education |
| Descriptors: | Sampling, Statistical Inference, Tests, Statistical Analysis, Test Items, Achievement Tests, International Assessment, Foreign Countries, Secondary School Students, Cognitive Measurement |
| Assessment and Survey Identifiers: | Program for International Student Assessment |
| DOI: | 10.1111/jedm.12440 |
| ISSN: | 0022-0655 1745-3984 |
| Abstract: | While testlets have proven useful for assessing complex skills, the stem shared by multiple items often induces correlations between responses, leading to violations of local independence (LI), which can result in biased parameter and ability estimates. Diagnostic procedures for detecting testlet effects typically involve model comparisons testing for the inclusion of extra testlet parameters or, at the item level, testing for pairwise LI. Rosenbaum's adaptation of the Mantel-Haenszel (MH) X[superscript 2]-statistic belongs to the latter category. The MH X[superscript 2]-statistic has also been used in cognitive diagnosis for detecting violations of LI and for the identification of testlet effects. However, this approach is not without limitations, as it lacks a rationale for integrating multiple pairwise MH X[superscript 2]-statistics and any notion of the sampling distribution of such an integrated statistic. In this article, a procedure for integrating multiple pairwise MH X[superscript 2]-statistics to evaluate testlet effects in cognitive diagnosis is proposed. The unknown sampling distribution issue is addressed by implementing a parametric bootstrap resampling scheme. Results from simulation studies demonstrate the performance of the proposed parametric bootstrap testlet MH X[superscript 2]-statistic, and its application to the 2015 PISA Collaborative Problem Solving (CPS) data set illustrates the method's practical merits. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1491369 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | While testlets have proven useful for assessing complex skills, the stem shared by multiple items often induces correlations between responses, leading to violations of local independence (LI), which can result in biased parameter and ability estimates. Diagnostic procedures for detecting testlet effects typically involve model comparisons testing for the inclusion of extra testlet parameters or, at the item level, testing for pairwise LI. Rosenbaum's adaptation of the Mantel-Haenszel (MH) X[superscript 2]-statistic belongs to the latter category. The MH X[superscript 2]-statistic has also been used in cognitive diagnosis for detecting violations of LI and for the identification of testlet effects. However, this approach is not without limitations, as it lacks a rationale for integrating multiple pairwise MH X[superscript 2]-statistics and any notion of the sampling distribution of such an integrated statistic. In this article, a procedure for integrating multiple pairwise MH X[superscript 2]-statistics to evaluate testlet effects in cognitive diagnosis is proposed. The unknown sampling distribution issue is addressed by implementing a parametric bootstrap resampling scheme. Results from simulation studies demonstrate the performance of the proposed parametric bootstrap testlet MH X[superscript 2]-statistic, and its application to the 2015 PISA Collaborative Problem Solving (CPS) data set illustrates the method's practical merits. |
|---|---|
| ISSN: | 0022-0655 1745-3984 |
| DOI: | 10.1111/jedm.12440 |
