Type I Error Inflation in DIF Identification with Mantel-Haenszel: An Explanation and a Solution
Saved in:
| Title: | Type I Error Inflation in DIF Identification with Mantel-Haenszel: An Explanation and a Solution |
|---|---|
| Language: | English |
| Authors: | Magis, David, De Boeck, Paul |
| Source: | Educational and Psychological Measurement. Aug 2014 74(4):713-728. |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 16 |
| Publication Date: | 2014 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Test Bias, Statistical Analysis, Test Items, Simulation |
| DOI: | 10.1177/0013164413516855 |
| ISSN: | 0013-1644 |
| Abstract: | It is known that sum score-based methods for the identification of differential item functioning (DIF), such as the Mantel-Haenszel (MH) approach, can be affected by Type I error inflation in the absence of any DIF effect. This may happen when the items differ in discrimination and when there is item impact. On the other hand, outlier DIF methods have been developed that are robust against this Type I error inflation, although they are still based on the MH DIF statistic. The present article gives an explanation for why the common MH method is indeed vulnerable to the inflation effect whereas the outlier DIF versions are not. In a simulation study, we were able to produce the Type I error inflation by inducing item impact and item differences in discrimination. At the same time and in parallel with the Type I error inflation, the dispersion of the DIF statistic across items was increased. As expected, the outlier DIF methods did not seem sensitive to impact and differences in item discrimination. |
| Abstractor: | As Provided |
| Number of References: | 44 |
| Entry Date: | 2014 |
| Accession Number: | EJ1039775 |
| Database: | ERIC |
| Abstract: | It is known that sum score-based methods for the identification of differential item functioning (DIF), such as the Mantel-Haenszel (MH) approach, can be affected by Type I error inflation in the absence of any DIF effect. This may happen when the items differ in discrimination and when there is item impact. On the other hand, outlier DIF methods have been developed that are robust against this Type I error inflation, although they are still based on the MH DIF statistic. The present article gives an explanation for why the common MH method is indeed vulnerable to the inflation effect whereas the outlier DIF versions are not. In a simulation study, we were able to produce the Type I error inflation by inducing item impact and item differences in discrimination. At the same time and in parallel with the Type I error inflation, the dispersion of the DIF statistic across items was increased. As expected, the outlier DIF methods did not seem sensitive to impact and differences in item discrimination. |
|---|---|
| ISSN: | 0013-1644 |
| DOI: | 10.1177/0013164413516855 |
