View in EDS

Internal validation methods to detect patient characteristics associated with differential item functioning in patient-reported outcome measures.

Saved in:
Bibliographic Details
Title: Internal validation methods to detect patient characteristics associated with differential item functioning in patient-reported outcome measures.
Authors: Bodawatte Gedara, Muditha (AUTHOR), Jiang, Depeng (AUTHOR), Kim, Sunmee (AUTHOR), Sanusi, Ridwan (AUTHOR), Sajobi, Tolulope (AUTHOR), Lix, Lisa M. (AUTHOR)
Source: Quality of Life Research. May2026, Vol. 35 Issue 5, p1-14. 14p.
Subjects: Model validation, Machine learning, Simulation methods & models, Patient reported outcome measures, Item response theory, Recursive partitioning
Abstract: Purpose: Differential item functioning (DIF) detection methods are one tool to assess validity of patient-reported outcome measures (PROMs). Given the growing interest in machine-learning models for the analysis of PROMs data, including for DIF detection, internal cross-validation (CV) is recommended to assess model performance. We compared two CV methods for DIF models in simulated and real-world data. Methods: Two common CV methods, k-fold and holdout CV, were compared for a machine-learning recursive partitioning model to detect DIF. Simulation parameters included sample size, number of items and covariates associated with DIF, and DIF effect size. Simulation performance was assessed using true positive rate (TPR; power) and false positive rate (FPR; Type I error) for items, covariates, and all item-covariate combinations. DIF performance was also assessed in real-world Hospital Anxiety and Depression Scale (HADS) item responses using area under the receiver operating characteristic (AUROC) curve. Results: In simulations, TPRs for consistently identifying items and covariates associated with DIF increased as DIF effect size, sample size, and number of items increased for both CV methods. The 10-fold CV method had the highest TPR in many conditions reaching nearly 100% at the item-covariate level while the 50% holdout CV method had the lowest FPR. For HADS items, DIF detection varied between CV methods, with AUROC ranging from 0.74 to 0.80. Conclusion: The choice of a CV method for a machine-learning model to detect DIF affected consistency of DIF item detection. The 10-fold CV method was advantageous for balancing systematic error and data stability. [ABSTRACT FROM AUTHOR]
Copyright of Quality of Life Research is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Psychology and Behavioral Sciences Collection
Description
Abstract:Purpose: Differential item functioning (DIF) detection methods are one tool to assess validity of patient-reported outcome measures (PROMs). Given the growing interest in machine-learning models for the analysis of PROMs data, including for DIF detection, internal cross-validation (CV) is recommended to assess model performance. We compared two CV methods for DIF models in simulated and real-world data. Methods: Two common CV methods, k-fold and holdout CV, were compared for a machine-learning recursive partitioning model to detect DIF. Simulation parameters included sample size, number of items and covariates associated with DIF, and DIF effect size. Simulation performance was assessed using true positive rate (TPR; power) and false positive rate (FPR; Type I error) for items, covariates, and all item-covariate combinations. DIF performance was also assessed in real-world Hospital Anxiety and Depression Scale (HADS) item responses using area under the receiver operating characteristic (AUROC) curve. Results: In simulations, TPRs for consistently identifying items and covariates associated with DIF increased as DIF effect size, sample size, and number of items increased for both CV methods. The 10-fold CV method had the highest TPR in many conditions reaching nearly 100% at the item-covariate level while the 50% holdout CV method had the lowest FPR. For HADS items, DIF detection varied between CV methods, with AUROC ranging from 0.74 to 0.80. Conclusion: The choice of a CV method for a machine-learning model to detect DIF affected consistency of DIF item detection. The 10-fold CV method was advantageous for balancing systematic error and data stability. [ABSTRACT FROM AUTHOR]
ISSN:09629343
DOI:10.1007/s11136-026-04196-x