Two-Phase Content-Balancing CD-CAT Online Item Calibration
Saved in:
| Title: | Two-Phase Content-Balancing CD-CAT Online Item Calibration |
|---|---|
| Language: | English |
| Authors: | Jing Huang, Yuxiao Zhang, Jason W. Morphew, Jayson M. Nissen, Ben Van Dusen, Hua Hua Chang |
| Source: | Journal of Educational Measurement. 2025 62(4):787-808. |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 22 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Adaptive Testing, Computer Assisted Testing, Cognitive Measurement, Test Items, Test Construction, Accuracy, Sample Size, Test Length |
| DOI: | 10.1111/jedm.70012 |
| ISSN: | 0022-0655 1745-3984 |
| Abstract: | Online calibration estimates new item parameters alongside previously calibrated items, supporting efficient item replenishment. However, most existing online calibration procedures for Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) lack mechanisms to ensure content balance during live testing. This limitation can lead to uneven content coverage, potentially undermining the alignment with instructional goals. This research extends the current calibration framework by integrating a two-phase test design with a content-balancing item selection method into the online calibration procedure. Simulation studies evaluated item parameter recovery and attribute profile estimation accuracy under the proposed procedure. Results indicated that the developed procedure yielded more accurate new item parameter estimates. The procedure also maintained content representativeness under both balanced and unbalanced constraints. Attribute profile estimation was sensitive to item parameter values. Accuracy declined when items had larger parameter values. Calibration improved with larger sample sizes and smaller parameter values. Longer test lengths contributed more to profile estimation than to new item calibration. These findings highlight design trade-offs in adaptive item replenishment and suggest new directions for hybrid calibration methods. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1491385 |
| Database: | ERIC |
| Abstract: | Online calibration estimates new item parameters alongside previously calibrated items, supporting efficient item replenishment. However, most existing online calibration procedures for Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) lack mechanisms to ensure content balance during live testing. This limitation can lead to uneven content coverage, potentially undermining the alignment with instructional goals. This research extends the current calibration framework by integrating a two-phase test design with a content-balancing item selection method into the online calibration procedure. Simulation studies evaluated item parameter recovery and attribute profile estimation accuracy under the proposed procedure. Results indicated that the developed procedure yielded more accurate new item parameter estimates. The procedure also maintained content representativeness under both balanced and unbalanced constraints. Attribute profile estimation was sensitive to item parameter values. Accuracy declined when items had larger parameter values. Calibration improved with larger sample sizes and smaller parameter values. Longer test lengths contributed more to profile estimation than to new item calibration. These findings highlight design trade-offs in adaptive item replenishment and suggest new directions for hybrid calibration methods. |
|---|---|
| ISSN: | 0022-0655 1745-3984 |
| DOI: | 10.1111/jedm.70012 |
