A Comparison of IRT Proficiency Estimation Methods Under Adaptive Multistage Testing.

Saved in:

Bibliographic Details
Title:	A Comparison of IRT Proficiency Estimation Methods Under Adaptive Multistage Testing.
Authors:	Kim, Sooyeon¹ skim@ets.org, Moses, Tim² tmoses@collegeboard.org, Yoo, Hanwook (Henry)¹ hyoo@ets.org
Source:	Journal of Educational Measurement. Mar2015, Vol. 52 Issue 1, p70-79. 10p.
Subject Terms:	Item response theory, Educational tests & measurements, Bayesian analysis, Educational quality, *Examination item response theory (Education)
Abstract:	This inquiry is an investigation of item response theory (IRT) proficiency estimators' accuracy under multistage testing (MST). We chose a two-stage MST design that includes four modules (one at Stage 1, three at Stage 2) and three difficulty paths (low, middle, high). We assembled various two-stage MST panels (i.e., forms) by manipulating two assembly conditions in each module, such as difficulty level and module length. For each panel, we investigated the accuracy of examinees' proficiency levels derived from seven IRT proficiency estimators. The choice of Bayesian (prior) versus non-Bayesian (no prior) estimators was of more practical significance than the choice of number-correct versus item-pattern scoring estimators. The Bayesian estimators were slightly more efficient than the non-Bayesian estimators, resulting in smaller overall error. Possible score changes caused by the use of different proficiency estimators would be nonnegligible, particularly for low- and high-performing examinees. [ABSTRACT FROM AUTHOR]
	Copyright of Journal of Educational Measurement is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Education Research Complete

Description
Abstract:	This inquiry is an investigation of item response theory (IRT) proficiency estimators' accuracy under multistage testing (MST). We chose a two-stage MST design that includes four modules (one at Stage 1, three at Stage 2) and three difficulty paths (low, middle, high). We assembled various two-stage MST panels (i.e., forms) by manipulating two assembly conditions in each module, such as difficulty level and module length. For each panel, we investigated the accuracy of examinees' proficiency levels derived from seven IRT proficiency estimators. The choice of Bayesian (prior) versus non-Bayesian (no prior) estimators was of more practical significance than the choice of number-correct versus item-pattern scoring estimators. The Bayesian estimators were slightly more efficient than the non-Bayesian estimators, resulting in smaller overall error. Possible score changes caused by the use of different proficiency estimators would be nonnegligible, particularly for low- and high-performing examinees. [ABSTRACT FROM AUTHOR]
ISSN:	00220655
DOI:	10.1111/jedm.12063