Semisupervised Learning Method to Adjust Biased Item Difficulty Estimates Caused by Nonignorable Missingness in a Virtual Learning Environment
Saved in:
| Title: | Semisupervised Learning Method to Adjust Biased Item Difficulty Estimates Caused by Nonignorable Missingness in a Virtual Learning Environment |
|---|---|
| Language: | English |
| Authors: | Xue, Kang (ORCID |
| Source: | Educational and Psychological Measurement. Jun 2022 82(3):539-567. |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 29 |
| Publication Date: | 2022 |
| Sponsoring Agency: | Institute of Education Sciences (ED) |
| Contract Number: | R305C160004 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Virtual Classrooms, Artificial Intelligence, Item Response Theory, Item Analysis, Testing Programs, Man Machine Systems, Data Analysis, Academic Ability, Response Style (Tests), Test Items, Difficulty Level, Student Behavior, Testing |
| Geographic Terms: | Florida |
| DOI: | 10.1177/00131644211020494 |
| ISSN: | 0013-1644 1552-3888 |
| Abstract: | In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of nonignorable missing data in the VLE log file data, and this is expected to negatively affect IRT item parameter estimation accuracy, which then negatively affects any future ability estimates utilized in the VLE. In the psychometric literature, methods for handling missing data have been studied mostly around conditions in which the data and the amount of missing data are not as large as those that come from VLEs. In this article, we introduce a semisupervised learning method to deal with a large proportion of missingness contained in VLE data from which one needs to obtain unbiased item parameter estimates. First, we explored the factors relating to the missing data. Then we implemented a semisupervised learning method under the two-parameter logistic IRT model to estimate the latent abilities of students. Last, we applied two adjustment methods designed to reduce bias in item parameter estimates. The proposed framework showed its potential for obtaining unbiased item parameter estimates that can then be fixed in the VLE in order to obtain ongoing ability estimates for operational purposes. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2022 |
| Accession Number: | EJ1336691 |
| Database: | ERIC |
| Abstract: | In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of nonignorable missing data in the VLE log file data, and this is expected to negatively affect IRT item parameter estimation accuracy, which then negatively affects any future ability estimates utilized in the VLE. In the psychometric literature, methods for handling missing data have been studied mostly around conditions in which the data and the amount of missing data are not as large as those that come from VLEs. In this article, we introduce a semisupervised learning method to deal with a large proportion of missingness contained in VLE data from which one needs to obtain unbiased item parameter estimates. First, we explored the factors relating to the missing data. Then we implemented a semisupervised learning method under the two-parameter logistic IRT model to estimate the latent abilities of students. Last, we applied two adjustment methods designed to reduce bias in item parameter estimates. The proposed framework showed its potential for obtaining unbiased item parameter estimates that can then be fixed in the VLE in order to obtain ongoing ability estimates for operational purposes. |
|---|---|
| ISSN: | 0013-1644 1552-3888 |
| DOI: | 10.1177/00131644211020494 |
