View in EDS HTML Full Text PDF Full Text

Detecting Differential Item Functioning among Multiple Groups Using IRT Residual DIF Framework

Saved in:
Bibliographic Details
Title: Detecting Differential Item Functioning among Multiple Groups Using IRT Residual DIF Framework
Language: English
Authors: Hwanggyu Lim, Danqi Zhu, Edison M. Choe, Kyung T. Han
Source: Journal of Educational Measurement. 2024 61(4):656-681.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 26
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Descriptors: Item Response Theory, Test Bias, Test Reliability, Test Construction, Group Testing, Error of Measurement
DOI: 10.1111/jedm.12415
ISSN: 0022-0655
1745-3984
Abstract: This study presents a generalized version of the residual differential item functioning (RDIF) detection framework in item response theory, named GRDIF, to analyze differential item functioning (DIF) in multiple groups. The GRDIF framework retains the advantages of the original RDIF framework, such as computational efficiency and ease of implementation. The performance of GRDIF was assessed through a simulation study and compared with existing DIF detection methods, including the generalized Mantel-Haenszel, Lasso-DIF, and alignment methods. Results showed that the GRDIF framework demonstrated well-controlled Type I error rates close to the nominal level of .05 and satisfactory power in detecting uniform, nonuniform, and mixed DIF across different simulated conditions. Each of the three GRDIF statistics, GRDIF[subscript R], GRDIF[subscript S], and GRDIF[subscript RS], effectively detected the specific type of DIF for which it was designed, with GRDIF[subscript RS] exhibiting the most robust performance across all types of DIF. The GRDIF framework outperformed other DIF detection methods under various conditions, suggesting its potential for practical applications, particularly in large-scale assessments involving multiple groups. Additionally, an empirical study demonstrated the efficacy and utility of the GRDIF framework in conducting DIF analysis with a high-stakes assessment data set.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1455024
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGIPCW3cQQkTHS4wmIqMg99AAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDIX511unRs5EgVcF_QIBEICBm4wEgl1cah9rpcYw-IjnXhnNBwLChRGw9um6CI5_HPYo08pLhWdo_g4hkRcwvb7M2eLXwhsYKabpB-IBxOgLtFxS-ZXyap53uHI6dtsT7Jnbe2uCl31w5UNl7XA1YvJzeItNaGHGAi2j0GtMHiagJXQiLQQzpinc86pTAhBo_WmvhxQk6zknC2YwhSNTHs5ljsHfeONRgDL22k32
Text:
  Availability: 1
  Value: <anid>AN0181921599;mea01dec.24;2024Dec31.04:30;v2.2.500</anid> <title id="AN0181921599-1">Detecting Differential Item Functioning among Multiple Groups Using IRT Residual DIF Framework </title> <p>This study presents a generalized version of the residual differential item functioning (RDIF) detection framework in item response theory, named GRDIF, to analyze differential item functioning (DIF) in multiple groups. The GRDIF framework retains the advantages of the original RDIF framework, such as computational efficiency and ease of implementation. The performance of GRDIF was assessed through a simulation study and compared with existing DIF detection methods, including the generalized Mantel‐Haenszel, Lasso‐DIF, and alignment methods. Results showed that the GRDIF framework demonstrated well‐controlled Type I error rates close to the nominal level of.05 and satisfactory power in detecting uniform, nonuniform, and mixed DIF across different simulated conditions. Each of the three GRDIF statistics, GRDIFR$GRDI{{F}_R}$, GRDIFS$GRDI{{F}_S}$, and GRDIFRS$GRDI{{F}_{RS}}$, effectively detected the specific type of DIF for which it was designed, with GRDIFRS$GRDI{{F}_{RS}}$ exhibiting the most robust performance across all types of DIF. The GRDIF framework outperformed other DIF detection methods under various conditions, suggesting its potential for practical applications, particularly in large‐scale assessments involving multiple groups. Additionally, an empirical study demonstrated the efficacy and utility of the GRDIF framework in conducting DIF analysis with a high‐stakes assessment data set.</p> <p>Differential item functioning (DIF) refers to the phenomenon where different groups of test takers, who are matched in terms of the ability being measured by the test, have different expected response to a test item. Accordingly, assessing DIF is important to ensure that tests are fair and unbiased for all test takers, regardless of their demographic or cultural background. Failure to identify and address DIF can result in inaccurate test scores, disadvantages for certain groups, and ultimately, incorrect decisions based on those scores. DIF can occur in two different ways: uniform and nonuniform. An item is said to exhibit uniform DIF when a test item has a consistent DIF effect across the range of ability levels for compared groups, while nonuniform DIF occurs when the effect of the item differs for different levels of ability (Hambleton et al., [<reflink idref="bib14" id="ref1">14</reflink>]; Swaminathan & Rogers, [<reflink idref="bib37" id="ref2">37</reflink>]).</p> <p>Recently, Lim et al. ([<reflink idref="bib20" id="ref3">20</reflink>]) proposed a new method for DIF called the residual DIF (RDIF) detection framework in item response theory (IRT), including three statistics of <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0005" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$RDI{{F}_R}$</annotation></semantics></math> </ephtml> , <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0006" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$RDI{{F}_S}$</annotation></semantics></math> </ephtml> , and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0007" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$RDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> . These three statistics are primarily effective at detecting uniform, nonuniform, and mixed DIF, respectively. Note that while each statistic excels in detecting its primary DIF type, they also retain some power to detect other types of DIF. Based on a linear test, their simulation results demonstrated that the RDIF framework exhibits decent power and effectively controls Type I error rates in comparison to other conventional DIF detection methods (e.g., Mantel‐Haenszel and logistic regression methods) under various conditions, provided that a purification procedure is employed. In addition, Lim and Choe ([<reflink idref="bib21" id="ref4">21</reflink>]) showed that <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0008" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$RDI{{F}_R}$</annotation></semantics></math> </ephtml> is a practical methodology for assessing uniform DIF on pretest items in computerized adaptive testing (CAT). They demonstrated that it performs comparably to the CATSIB procedure (Nandakumar & Roussos [<reflink idref="bib31" id="ref5">31</reflink>]) for detecting DIF, and can be used as a direct and reliable measure of effect size of uniform DIF.</p> <p>The RDIF framework has several advantages that make it more accessible and straightforward for researchers and practitioners to use. First, all RDIF statistics are easy to compute and quick to execute. Unlike other IRT‐based approaches, separate item calibration for each group or multiple model fittings is not required. As a result, there is no need for IRT equating, and relatively smaller sample sizes can be used. Also, it does not require estimating the variance‐covariance matrices of item parameters, which can be computationally intensive and challenging, particularly for large datasets. This is a considerable advantage because it simplifies the implementation of the RDIF framework. Second, the RDIF framework does not require a matching variable such as observed scores or theta bins, unlike traditional non‐IRT‐based approaches. This means that the RDIF framework can be applied directly to CAT without any modifications, which is proven by Lim and Choe ([<reflink idref="bib21" id="ref6">21</reflink>]). This feature makes it highly convenient and suitable for DIF analysis in CAT.</p> <p>While the RDIF approach offers a unique set of merits, its applications have been primarily limited to two‐group comparisons (Lim et al., [<reflink idref="bib20" id="ref7">20</reflink>]; Lim & Choe, [<reflink idref="bib21" id="ref8">21</reflink>]). In practice, however, DIF often occurs across demographic variables that span more than two groups, covering diverse ethnicities, education levels, and socioeconomic statuses[<reflink idref="bib1" id="ref9">1</reflink>]. To analyze DIF for more than two groups, the most straightforward approach is to conduct individual two‐group DIF tests between multiple pairs of groups (e.g., Fidalgo & Scalon, [<reflink idref="bib10" id="ref10">10</reflink>]; Penfield, [<reflink idref="bib32" id="ref11">32</reflink>]; Schmitt & Dorans, [<reflink idref="bib34" id="ref12">34</reflink>]; Zwick & Ercikan, [<reflink idref="bib42" id="ref13">42</reflink>]). However, this approach can lead to the well‐known issue of inflating Type I error (Magis et al., [<reflink idref="bib25" id="ref14">25</reflink>]) unless the familywise error rate is properly controlled. To address this problem, ad hoc methods such as Bonferroni correction can be used to control the familywise error rate, but this may cause lower power in DIF detection (e.g., Fidalgo & Scalon, [<reflink idref="bib10" id="ref15">10</reflink>]; Penfield, [<reflink idref="bib32" id="ref16">32</reflink>]). Accordingly, it may be preferable to use omnibus methods that can accurately identify DIF across all groups simultaneously while appropriately controlling Type I error and maintaining sufficient power. Unfortunately, the number of existing DIF methods readily available for analyzing multiple groups is limited (e.g., Asparouhov & Muthén, [<reflink idref="bib1" id="ref17">1</reflink>]; Kim et al., [<reflink idref="bib18" id="ref18">18</reflink>]; Magis & De Boeck, [<reflink idref="bib24" id="ref19">24</reflink>]; Magis et al., [<reflink idref="bib25" id="ref20">25</reflink>]; Penfield, [<reflink idref="bib32" id="ref21">32</reflink>]; Tutz & Schauberger, [<reflink idref="bib39" id="ref22">39</reflink>]; Wang et al., [<reflink idref="bib40" id="ref23">40</reflink>]), necessitating the need for alternative methodologies.</p> <p>Therefore, this study aims to further generalize the RDIF detection framework, hereafter referred to as the generalized RDIF (GRDIF), to assess DIF across multiple groups while retaining its distinct advantages. Under this generalized framework, the three RDIF statistics originally used for two‐group comparisons can be easily extended into chi‐square test statistics. This extension is based on the asymptotic multivariate normality of the mean raw and mean squared IRT residuals, which are the differences between observed and expected item scores</p> <p>The following sections review previous research related to other existing DIF detection methods for multiple groups and introduce the three GRDIF statistics in more detail. Then, we present the simulation study for evaluating the performance of GRDIF and other benchmark approaches under various conditions, subsequently presenting an empirical study using a data set from a high‐stakes assessment. Finally, we conclude the study with implications of the GRDIF framework and suggestions for future research.</p> <hd id="AN0181921599-2">Current Methods for Detecting DIF for Multiple Groups</hd> <p>A few methods have been developed to assess DIF for multiple groups by extending traditional DIF detection approaches for two groups. Such methods include the generalized Mantel‐Haenszel (GMH; Penfield, [<reflink idref="bib32" id="ref24">32</reflink>]; Somes, [<reflink idref="bib35" id="ref25">35</reflink>]), generalized logistic regression (GLR; Magis et al., [<reflink idref="bib25" id="ref26">25</reflink>]), and generalized Lord's <emph>χ</emph><sups>2</sups> test (GLord‐<emph>χ</emph><sups>2</sups>; Kim et al., [<reflink idref="bib18" id="ref27">18</reflink>]). Apart from the generalized traditional methods, only a limited number of new methodologies have been introduced to evaluate DIF under the multiple‐group context. These include the alignment method (Muthén & Asparouhov, [<reflink idref="bib29" id="ref28">29</reflink>]), regularization DIF detection method (e.g., Belzak, [<reflink idref="bib4" id="ref29">4</reflink>]; Tutz & Schauberger, [<reflink idref="bib39" id="ref30">39</reflink>]; Wang et al., [<reflink idref="bib40" id="ref31">40</reflink>]), and multivariate outlier detection approach (Magis & De Boeck, [<reflink idref="bib24" id="ref32">24</reflink>]).</p> <p>In Finch's ([<reflink idref="bib11" id="ref33">11</reflink>]) simulation study, the GLR method resulted in highly inflated Type I error rates for conditions with more than three groups of unequal sizes, regardless of the sample sizes of the reference group. Additionally, the GLord‐<emph>χ</emph><sups>2</sups> method had lower DIF detection power than other methods, such as GMH, GLR, and alignment methods, in most simulated conditions. In recent years, the regularization method (e.g., Bauer et al., [<reflink idref="bib3" id="ref34">3</reflink>]; Belzak & Bauer, [<reflink idref="bib6" id="ref35">6</reflink>]; Magis et al., [<reflink idref="bib26" id="ref36">26</reflink>]; Tutz & Schauberger, [<reflink idref="bib39" id="ref37">39</reflink>]; Wang et al., [<reflink idref="bib40" id="ref38">40</reflink>]) and the alignment method (e.g., Asparouhov & Muthén, [<reflink idref="bib1" id="ref39">1</reflink>]; DeMars, [<reflink idref="bib9" id="ref40">9</reflink>]; Finch, [<reflink idref="bib11" id="ref41">11</reflink>]; Flake & McCoach, [<reflink idref="bib13" id="ref42">13</reflink>]; Muthén & Asparouhov, [<reflink idref="bib29" id="ref43">29</reflink>]) have become increasingly popular as new approaches to detecting DIF. These methods have also shown promise when analyzing DIF for multiple groups (Finch, [<reflink idref="bib11" id="ref44">11</reflink>]; Flake & McCoach, [<reflink idref="bib13" id="ref45">13</reflink>]; Muthén & Asparouhov, [<reflink idref="bib29" id="ref46">29</reflink>]; Tutz & Schauberger, [<reflink idref="bib39" id="ref47">39</reflink>]; Wang et al., [<reflink idref="bib40" id="ref48">40</reflink>]). While the outlier detection approach (Magis & De Boeck, [<reflink idref="bib24" id="ref49">24</reflink>]) does not necessitate purification of the matching variable and is relatively simple to implement, the authors have acknowledged that more research is needed to assess its effectiveness in accurately identifying DIF. Therefore, we chose the GMH, alignment, and regularization DIF detection methods from the aforementioned techniques as benchmarks to evaluate the performance of GRDIF approaches in detecting DIF for multiple groups.</p> <hd id="AN0181921599-3">Previous Research</hd> <p></p> <hd id="AN0181921599-4">Generalized Mantel‐Haenszel method</hd> <p>The GMH is an extension of the Mantel‐Haenszel test statistic (Holland & Thayer, [<reflink idref="bib15" id="ref50">15</reflink>]), which is used for detecting DIF in two groups. The GMH can test DIF across multiple groups simultaneously by comparing their item responses while controlling for multiple levels of a matching variable (Finch, [<reflink idref="bib11" id="ref51">11</reflink>]). In the simulations of Penfield ([<reflink idref="bib32" id="ref52">32</reflink>]) and Finch ([<reflink idref="bib11" id="ref53">11</reflink>]), the performance of GMH was examined by varying important factors, such as the number of groups, size of groups, the number of focal groups experiencing DIF, and impact between the groups. Specifically, Penfield used three versions of the MH test consisting of GMH, MH for multiple pairwise comparison, and MH for all pairwise comparison with a Bonferroni correction, while Finch compared four different methods including GMH, GLR, GLord‐<emph>χ</emph><sups>2</sups>, and alignment approaches. Their studies commonly found that GMH generally provides good balance between controlling the Type I error and maintaining reasonable power. However, it is important to note that their simulation studies only manipulated uniform DIF, which is well‐suited for identification by the MH test (Lim et al., [<reflink idref="bib20" id="ref54">20</reflink>]; Magis et al., [<reflink idref="bib25" id="ref55">25</reflink>]). In addition, Magis et al. ([<reflink idref="bib25" id="ref56">25</reflink>]) found that, on a real language skill assessment, the GMH method showed comparable ability to the LR method in flagging the same items with potential DIF.</p> <hd id="AN0181921599-5">Lasso regularization method</hd> <p>Least absolute shrinkage and selection operator (Lasso; Tibshirani, [<reflink idref="bib38" id="ref57">38</reflink>]) is a particular form of regularization that utilizes a penalized estimation approach to shrink certain model parameters towards zero, thereby improving stability of the model estimation. Consequently, it can effectively keep a subset of possible predictors significantly impacting model fit (McNeish, [<reflink idref="bib27" id="ref58">27</reflink>]). Due to this property, lasso regularization has been accepted as a useful tool for assessing DIF on test items (e.g., Belzak & Bauer, [<reflink idref="bib6" id="ref59">6</reflink>]; Magis et al., [<reflink idref="bib26" id="ref60">26</reflink>]; Tutz & Schauberger, [<reflink idref="bib39" id="ref61">39</reflink>]; Wang et al., [<reflink idref="bib40" id="ref62">40</reflink>]) in the IRT context. It enables free estimation of item parameters while imposing a lasso penalty on the parameters of person‐specific covariates (i.e., DIF predictors; Wang et al., [<reflink idref="bib40" id="ref63">40</reflink>]). The remaining nonzero DIF parameters imply which of the variables is responsible for DIF. This approach is henceforth referred to as Lasso‐DIF in this study.</p> <p>Because Lasso‐DIF allows including multiple covariates in the evaluated model, it facilitates DIF analysis for multiple groups by using dummy variables to represent the groups. Tutz and Schauberger ([<reflink idref="bib39" id="ref64">39</reflink>]) and Wang et al. ([<reflink idref="bib40" id="ref65">40</reflink>]) demonstrated that Lasso‐DIF can be applied to find the potential source of DIF by allowing the item parameters to differ by multiple groups. Tutz and Schauberger ([<reflink idref="bib39" id="ref66">39</reflink>]) applied the Lasso‐DIF method based on the dichotomous Rasch model to simulated conditions featuring five and six multiple groups to identify uniform DIF on a 20‐item test. Their simulation results showed that the Lasso‐DIF method is promising for detecting medium and large magnitudes of DIF. When the magnitude of DIF is small, however, power of the Lasso‐DIF method in detecting DIF was notably lower compared to other traditional competitors.</p> <p>On the other hand, Wang et al. ([<reflink idref="bib40" id="ref67">40</reflink>]) investigated Lasso‐DIF's ability to detect DIF in a three‐group comparison scenario using the two‐dimensional IRT two‐parameter logistic model (2PLM). They proposed two variations of the lasso method with expectation‐maximization: lasso with expectation‐maximization‐maximization and adaptive Lasso. Their simulations demonstrated that both proposed methods have great potential for large sample sizes, outperforming the IRT likelihood ratio test, especially with a large percentage of DIF items in a test.</p> <p>Note that Lasso‐DIF can become computationally demanding and time‐consuming with increased test length or sample size due to the need for fitting a larger number of models with varied tuning parameters.</p> <hd id="AN0181921599-6">Alignment method</hd> <p>The alignment method, proposed by Asparouhov and Muthén ([<reflink idref="bib1" id="ref68">1</reflink>]), is a technique used in the framework of multiple‐group confirmatory factor analysis to compare factor means and variances (and covariances) across groups without requiring exact measurement invariance. The absence of measurement invariance for item parameters is denoted as DIF in IRT. Muthén and Asparouhov ([<reflink idref="bib29" id="ref69">29</reflink>]) and Finch ([<reflink idref="bib11" id="ref70">11</reflink>]) have demonstrated that the alignment method can be applied to assess DIF on test items for multiple groups in the context of IRT. This method is practical for comparing many groups and extra useful because it can estimate separate model parameters for each group (Finch, [<reflink idref="bib11" id="ref71">11</reflink>]). Additionally, since the alignment model estimation can be implemented using Bayesian approach, it is possible to relax a specific distributional assumption, such as the normality of the item parameter distributions across groups (Muthén & Asparouhov, [<reflink idref="bib29" id="ref72">29</reflink>]).</p> <p>Muthén and Asparouhov ([<reflink idref="bib29" id="ref73">29</reflink>]) applied the alignment method to empirical international assessment data and showed that it could produce well‐recovered parameter values for 28 country groups and identify DIF for these groups. Munck et al. ([<reflink idref="bib28" id="ref74">28</reflink>]) confirmed that the alignment method is feasible to assess DIF even for 92 groups. Meanwhile, Flake and McCoach ([<reflink idref="bib13" id="ref75">13</reflink>]) analyzed the performance of the alignment method in detecting DIF for polytomous items across multiple groups under various simulation scenarios. While it effectively retrieved item parameters under conditions with small to moderate percentages of items exhibiting DIF, its statistical invariance tests were notably conservative, leading to reduced power, particularly for detecting smaller or medium magnitudes of DIF.</p> <p>In Finch's ([<reflink idref="bib11" id="ref76">11</reflink>]) simulation study, while the alignment method demonstrated comparable power to the GMH method for detecting DIF and acceptable control of Type I error, it resulted in biased estimates of item parameters when there was impact between groups. Additionally, this method may have a large chance of producing inaccurate identification of DIF when the sample size is small or when DIF exists for a large proportion of items in a test (Finch, [<reflink idref="bib11" id="ref77">11</reflink>]; Muthén & Asparouhov, [<reflink idref="bib29" id="ref78">29</reflink>]).</p> <p>Table A1 in Appendix A provides an overview of the three benchmark methods. For a detailed description of the procedures for implementing these benchmarks, please refer to Appendix B.</p> <hd id="AN0181921599-7">Generalized Residual DIF Framework for Multiple Groups</hd> <p>We describe the computational processes of three GRDIF statistics for multiple group comparisons, namely, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0009" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> , <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0010" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_S}$</annotation></semantics></math> </ephtml> , and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0011" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> . These statistics are designed to mainly effective at detecting uniform, nonuniform, and mixed DIF, respectively, which are similar to the original RDIF statistics used for two‐group comparison. While each GRDIF statistic is specialized in identifying a specific type of DIF, they may collectively flag an item exhibiting different types of DIF across various group pairings. For instance, an item exhibiting uniform DIF between groups 1 and 2, and nonuniform DIF between groups 2 and 3, might be detected by all three GRDIF statistics. Unlike the two‐group comparison, the null ( <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0012" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>H</mi><mn>0</mn></msub><mrow><mo>)</mo></mrow></mrow><annotation encoding="application/x-tex">${{H}_0})$</annotation></semantics></math> </ephtml> and alternative ( <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0013" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>H</mi><mn>1</mn></msub><annotation encoding="application/x-tex">${{H}_1}$</annotation></semantics></math> </ephtml> ) hypotheses for testing the existence of DIF among multiple groups can be stated:</p> <p></p> <ulist> <item> H <subs>0</subs> : <emph>DIF is not present in any pair among all groups</emph>.</item> <p></p> <item> H <subs>1</subs> : <emph>DIF is present in at least one pair among all groups</emph>.</item> </ulist> <p>To compute the GRDIF statistics for a dichotomously scored item <emph>j</emph>, the followings are required: (a) item parameter estimates based on the aggregate data (i.e., responses across all groups), (b) individual responses to the item, and (c) examines' IRT‐based ability estimates ( <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0014" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>θ</mi><mo>̂</mo></mover><mi mathvariant="normal">s</mi></mrow><annotation encoding="application/x-tex">$\hat{\theta }{\mathrm{s}}$</annotation></semantics></math> </ephtml> ). Note that <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0015" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>θ</mi><mo>̂</mo></mover><mi mathvariant="normal">s</mi></mrow><annotation encoding="application/x-tex">$\hat{\theta }{\mathrm{s}}$</annotation></semantics></math> </ephtml> are computed using the item parameter estimates based on aggregate data. Lim et al. ([<reflink idref="bib20" id="ref79">20</reflink>]) recommended using the maximum likelihood estimation (MLE) for <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0016" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>θ</mi><mo>̂</mo></mover><mi mathvariant="normal">s</mi></mrow><annotation encoding="application/x-tex">$\hat{\theta }{\mathrm{s}}$</annotation></semantics></math> </ephtml> .</p> <p>For a dichotomously scored item, let <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0017" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>x</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub><annotation encoding="application/x-tex">${{x}_{ig}}$</annotation></semantics></math> </ephtml> be the observed responses (i.e., 0 for incorrect or 1 for correct) and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0018" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub><annotation encoding="application/x-tex">${{P}_{ig}}$</annotation></semantics></math> </ephtml> be the IRT model‐based probability of correct answer for individual <emph>i</emph> ( <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0019" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mspace width="0.33em" /><mn>2</mn><mo>,</mo><mspace width="0.33em" /><mtext>...</mtext><mo>,</mo><mspace width="0.33em" /><msub><mi>N</mi><mi>g</mi></msub></mrow><annotation encoding="application/x-tex">$i = 1,\ 2,\ \ldots ,\ {{N}_g}$</annotation></semantics></math> </ephtml> ) in group <emph>g</emph> ( <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0020" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>g</mi><mo>=</mo><mn>1</mn><mo>,</mo><mspace width="0.33em" /><mn>2</mn><mo>,</mo><mspace width="0.33em" /><mtext>...</mtext><mo>,</mo><mspace width="0.33em" /><mi>G</mi></mrow><annotation encoding="application/x-tex">$g = 1,\ 2,\ \ldots ,\ G$</annotation></semantics></math> </ephtml> ). When the IRT 2PM is fit to a certain test data set, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0021" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub><annotation encoding="application/x-tex">${{P}_{ig}}$</annotation></semantics></math> </ephtml> can be computed using the item discrimination parameter estimates <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0022" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mover accent="true"><mi>a</mi><mo>̂</mo></mover><annotation encoding="application/x-tex">$\hat{a}$</annotation></semantics></math> </ephtml> , difficulty parameter estimates <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0023" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mover accent="true"><mi>b</mi><mo>̂</mo></mover><annotation encoding="application/x-tex">$\hat{b}$</annotation></semantics></math> </ephtml> (i.e., <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0024" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo>−</mo><mover accent="true"><mi>c</mi><mo>̂</mo></mover><mo>/</mo><mover accent="true"><mi>a</mi><mo>̂</mo></mover><mrow><mo>)</mo></mrow></mrow><annotation encoding="application/x-tex">$ - \hat{c}/\hat{a})$</annotation></semantics></math> </ephtml> , and ability estimate <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0025" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mover accent="true"><mi>θ</mi><mo>̂</mo></mover><annotation encoding="application/x-tex">$\hat{\theta }$</annotation></semantics></math> </ephtml> based on Equation B4 in Appendix B, assuming that the model parameters are known. Using the individual raw residuals for the item, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0026" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>r</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub><mo>=</mo><msub><mi>x</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub><mspace width="0.33em" /><mo>−</mo><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub></mrow><annotation encoding="application/x-tex">${{r}_{ig}} = {{x}_{ig}}\ - {{P}_{ig}}$</annotation></semantics></math> </ephtml> , the <emph>mean raw residuals</emph> (MRR) and <emph>mean squared residuals</emph> (MSR) for group <emph>g</emph> are computed as, respectively, 1 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0027" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>M</mi><mi>R</mi><msub><mi>R</mi><mi>g</mi></msub><mo linebreak="badbreak">=</mo><mfrac><mrow><msubsup><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>N</mi><mi>g</mi></msub></msubsup><msub><mi>r</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub></mrow><msub><mi>N</mi><mi>g</mi></msub></mfrac><mo>,</mo><mspace width="1em" /><mi>M</mi><mi>S</mi><msub><mi>R</mi><mi>g</mi></msub><mo linebreak="goodbreak">=</mo><mfrac><mrow><msubsup><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>N</mi><mi>g</mi></msub></msubsup><msubsup><mi>r</mi><mrow><mi>i</mi><mi>g</mi></mrow><mn>2</mn></msubsup></mrow><msub><mi>N</mi><mi>g</mi></msub></mfrac><mo>.</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation}MR{{R}_g} = \frac{{\sum_{i = 1}^{{{N}_g}} {{r}_{ig}}}}{{{{N}_g}}},\quad MS{{R}_g} = \frac{{\sum_{i = 1}^{{{N}_g}} r_{ig}^2}}{{{{N}_g}}}.\end{equation}$$</annotation></semantics></math> </ephtml></p> <p>Given that observed responses to an item are independently but not identically distributed in the IRT models, MRR and MSR both approach normality in distribution under the null hypothesis of no DIF by Lyapunov's Central Limit Theorem (Billingsley, [<reflink idref="bib7" id="ref80">7</reflink>]) as follows: 2 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0028" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>M</mi><mi>R</mi><msub><mi>R</mi><mi>g</mi></msub><mover><mo>→</mo><mi>d</mi></mover><mi>N</mi><mfenced separators="" open="(" close=")"><mrow><msub><mi>μ</mi><mrow><mi>M</mi><mi>R</mi><msub><mi>R</mi><mi>g</mi></msub></mrow></msub><mo>,</mo><msubsup><mi>σ</mi><mrow><mi>M</mi><mi>R</mi><msub><mi>R</mi><mi>g</mi></msub></mrow><mn>2</mn></msubsup></mrow></mfenced><mo>,</mo><mspace width="1em" /><mi>M</mi><mi>S</mi><msub><mi>R</mi><mi>g</mi></msub><mover><mo>→</mo><mi>d</mi></mover><mi>N</mi><mfenced separators="" open="(" close=")"><mrow><msub><mi>μ</mi><mrow><mi>M</mi><mi>S</mi><msub><mi>R</mi><mi>g</mi></msub></mrow></msub><mo>,</mo><msubsup><mi>σ</mi><mrow><mi>M</mi><mi>S</mi><msub><mi>R</mi><mi>g</mi></msub></mrow><mn>2</mn></msubsup></mrow></mfenced><mo>.</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation}MR{{R}_g}\mathop \to \limits^d N\left({{{\mu }_{MR{{R}_g}}},\sigma _{MR{{R}_g}}^2} \right),\quad MS{{R}_g}\mathop \to \limits^d N\left({{{\mu }_{MS{{R}_g}}},\sigma _{MS{{R}_g}}^2} \right).\end{equation}$$</annotation></semantics></math> </ephtml></p> <p>According to Lim et al. ([<reflink idref="bib20" id="ref81">20</reflink>], appendix A), the means and variances in Equation 2 can be derived as 3 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0029" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>μ</mi><mrow><mi>M</mi><mi>R</mi><msub><mi>R</mi><mi>g</mi></msub></mrow></msub><mo linebreak="badbreak">=</mo><mn>0</mn><mo>,</mo><mspace width="1em" /><msub><mi>μ</mi><mrow><mi>M</mi><mi>S</mi><msub><mi>R</mi><mi>g</mi></msub></mrow></msub><mo linebreak="goodbreak">=</mo><mfrac><mrow><msubsup><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>N</mi><mi>g</mi></msub></msubsup><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub><mfenced separators="" open="(" close=")"><mrow><mn>1</mn><mo>−</mo><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub></mrow></mfenced></mrow><msub><mi>N</mi><mi>g</mi></msub></mfrac><mo>,</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation}{{\mu }_{MR{{R}_g}}} = 0,\quad {{\mu }_{MS{{R}_g}}} = \frac{{\sum_{i = 1}^{{{N}_g}} {{P}_{ig}}\left({1 - {{P}_{ig}}} \right)}}{{{{N}_g}}},\end{equation}$$</annotation></semantics></math> </ephtml> 4 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0030" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msubsup><mi>σ</mi><mrow><mi>M</mi><mi>R</mi><msub><mi>R</mi><mi>g</mi></msub></mrow><mn>2</mn></msubsup><mo linebreak="badbreak">=</mo><mfrac><mrow><msubsup><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>N</mi><mi>g</mi></msub></msubsup><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub><mfenced separators="" open="(" close=")"><mrow><mn>1</mn><mspace width="0.33em" /><mo>−</mo><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub></mrow></mfenced></mrow><msubsup><mi>N</mi><mi>g</mi><mn>2</mn></msubsup></mfrac><mo>,</mo><mspace width="1em" /><msubsup><mi>σ</mi><mrow><mi>M</mi><mi>S</mi><msub><mi>R</mi><mi>g</mi></msub></mrow><mn>2</mn></msubsup><mo linebreak="goodbreak">=</mo><mfrac><mrow><msubsup><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>N</mi><mi>g</mi></msub></msubsup><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub><mfenced separators="" open="(" close=")"><mrow><mn>1</mn><mo>−</mo><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub></mrow></mfenced><msup><mfenced separators="" open="(" close=")"><mrow><mn>1</mn><mo>−</mo><mn>2</mn><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub></mrow></mfenced><mn>2</mn></msup></mrow><msubsup><mi>N</mi><mi>g</mi><mn>2</mn></msubsup></mfrac><mo>.</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation}\sigma _{MR{{R}_g}}^2 = \frac{{\sum_{i = 1}^{{{N}_g}} {{P}_{ig}}\left({1\ - {{P}_{ig}}} \right)}}{{N_g^2}},\quad \sigma _{MS{{R}_g}}^2 = \frac{{\sum_{i = 1}^{{{N}_g}} {{P}_{ig}}\left({1 - {{P}_{ig}}} \right){{{\left({1 - 2{{P}_{ig}}} \right)}}^2}}}{{N_g^2}}.\end{equation}$$</annotation></semantics></math> </ephtml></p> <p>In addition, the covariance between MRR and MSR for group <emph>g</emph> is computed, as described in Lim et al. ([<reflink idref="bib20" id="ref82">20</reflink>], Appendix A),5 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0031" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>σ</mi><mfrac denomalign="left" numalign="left" linethickness="0pt"><mstyle scriptlevel="1" displaystyle="false"><mrow><mrow><mi>M</mi><mi>R</mi><msub><mi>R</mi><mi>g</mi></msub><mo>,</mo></mrow></mrow></mstyle><mstyle scriptlevel="1" displaystyle="false"><mrow><mrow><mi>M</mi><mi>S</mi><msub><mi>R</mi><mi>g</mi></msub></mrow></mrow></mstyle></mfrac></msub><mo linebreak="badbreak">=</mo><mfrac><mrow><msubsup><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>N</mi><mi>g</mi></msub></msubsup><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub><mfenced separators="" open="(" close=")"><mrow><mn>1</mn><mspace width="0.33em" /><mo>−</mo><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub></mrow></mfenced><mfenced separators="" open="(" close=")"><mrow><mn>1</mn><mspace width="0.33em" /><mo>−</mo><mn>2</mn><msub><mi>P</mi><mrow><mi>i</mi><mi>g</mi></mrow></msub></mrow></mfenced></mrow><msubsup><mi>N</mi><mi>g</mi><mn>2</mn></msubsup></mfrac><mspace width="0.33em" /><mo>,</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation}{{\sigma }_{\scriptstyle MR{{R}_g},\hfill\atop \scriptstyle MS{{R}_g}\hfill}} = \frac{{\sum_{i = 1}^{{{N}_g}} {{P}_{ig}}\left({1\ - {{P}_{ig}}} \right)\left({1\ - 2{{P}_{ig}}} \right)}}{{N_g^2}}\ ,\end{equation}$$</annotation></semantics></math> </ephtml> while the covariances for any combinations of MRR and MSR for two different groups (i.e., <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0032" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mi>g</mi><annotation encoding="application/x-tex">$g$</annotation></semantics></math> </ephtml> and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0033" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msup><mi>g</mi><mo>′</mo></msup><annotation encoding="application/x-tex">$g^{\prime}$</annotation></semantics></math> </ephtml> ) are all zero such as: 6 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0034" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>σ</mi><mrow><mi>M</mi><mi>R</mi><msub><mi>R</mi><mi>g</mi></msub><mo>,</mo><mi>M</mi><mi>R</mi><msub><mi>R</mi><msup><mi>g</mi><mo>′</mo></msup></msub></mrow></msub><mo linebreak="badbreak">=</mo><mn>0</mn><mo>,</mo><mspace width="1em" /><msub><mi>σ</mi><mrow><mi>M</mi><mi>S</mi><msub><mi>R</mi><mi>g</mi></msub><mo>,</mo><mi>M</mi><mi>S</mi><msub><mi>R</mi><msup><mi>g</mi><mo>′</mo></msup></msub></mrow></msub><mo linebreak="goodbreak">=</mo><mn>0</mn><mo>,</mo><mspace width="1em" /><msub><mi>σ</mi><mrow><mi>M</mi><mi>R</mi><msub><mi>R</mi><mi>g</mi></msub><mo>,</mo><mi>M</mi><mi>S</mi><msub><mi>R</mi><msup><mi>g</mi><mo>′</mo></msup></msub></mrow></msub><mo linebreak="goodbreak">=</mo><mn>0</mn><mo>.</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation}{{\sigma }_{MR{{R}_g},MR{{R}_{g^{\prime}}}}} = 0,\quad {{\sigma }_{MS{{R}_g},MS{{R}_{g^{\prime}}}}} = 0,\quad {{\sigma }_{MR{{R}_g},MS{{R}_{g^{\prime}}}}} = 0.\end{equation}$$</annotation></semantics></math> </ephtml></p> <p>See Appendix A in Lim et al. ([<reflink idref="bib20" id="ref83">20</reflink>]) for details of the derivations.</p> <p>Next, given <emph>G</emph> groups consisting of a reference group (R) and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0035" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mo>−</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">$G - 1$</annotation></semantics></math> </ephtml> focal groups (Fs), let <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0036" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi mathvariant="bold-italic">v</mi><mrow><mi>M</mi><mi>R</mi><mi>R</mi></mrow></msub><annotation encoding="application/x-tex">${{{\bm{v}}}_{MRR}}$</annotation></semantics></math> </ephtml> , <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0037" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi mathvariant="bold-italic">v</mi><mrow><mi>M</mi><mi>S</mi><mi>R</mi></mrow></msub><annotation encoding="application/x-tex">${{{\bm{v}}}_{MSR}}$</annotation></semantics></math> </ephtml> , and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0038" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi mathvariant="bold-italic">v</mi><mrow><mi>M</mi><mi>R</mi><mi>S</mi><mi>R</mi></mrow></msub><annotation encoding="application/x-tex">${{{\bm{v}}}_{MRSR}}$</annotation></semantics></math> </ephtml> be the vectors of MRRs, MSRs, and pairs of MRR and MSR, respectively, as 7 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0039" xmlns="http://www.w3.org/1998/Math/MathML"><semantics>vMRR=MRRR,MRRF1,...,MRRFG−1T,vMSR=MSRR,MSRF1,...,MSRFG−1T,vMRSR=MRRR,MSRR,...,MRRFG−1,MSRFG−1T.<annotation encoding="application/x-tex">$$\begin{equation} \def\eqcellsep{&}\begin{array}{l} {{{\bm{v}}}_{{\bm{MRR}}}} = {{\left({MR{{R}_R},MR{{R}_{F1}}, \ldots ,MR{{R}_{F\left({G - 1} \right)}}} \right)}^T},\\ {{{\bm{v}}}_{{\bm{MSR}}}} = {{\left({MS{{R}_R},MS{{R}_{F1}}, \ldots ,MS{{R}_{F\left({G - 1} \right)}}} \right)}^T},\\ {{{\bm{v}}}_{{\bm{MRSR}}}} = {{\left({MR{{R}_R},MS{{R}_R}, \ldots ,MR{{R}_{F\left({G - 1} \right)}},MS{{R}_{F\left({G - 1} \right)}}} \right)}^T}. \end{array} \end{equation}$$</annotation></semantics></math> </ephtml></p> <p>Note that the allocation of group labels is arbitrary, as the computation of GRDIF statistics remains invariant, irrespective of the group designated as the reference. For the three vectors in Equation 7, the corresponding mean vectors are defined as 8 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0040" xmlns="http://www.w3.org/1998/Math/MathML"><semantics>μMRR=μMRRR,μMRRF1,...,μMRRFG−1T,μMSR=μMSRR,μMSRF1,...,μMSRFG−1T,μMRSR=μMRRR,μMSRR,...,μMRRFG−1,μMSRFG−1T,<annotation encoding="application/x-tex">$$\begin{equation} \def\eqcellsep{&}\begin{array}{l} {{{\bm{\mu }}}_{{\bm{MRR}}}} = {{\left({{{\mu }_{MR{{R}_{\mathrm{R}}}}},\,{{\mu }_{MR{{R}_{F1}}}}, \ldots ,{{\mu }_{MR{{R}_{F\left({G - 1} \right)}}}}} \right)}^T},\\ {{{\bm{\mu }}}_{{\mathrm{MSR}}}} = {{\left({{{\mu }_{MS{{R}_R}}},\,{{\mu }_{MS{{R}_{F1}}}}, \ldots ,{{\mu }_{MS{{R}_{F\left({G - 1} \right)}}}}} \right)}^T},\\ {{{\bm{\mu }}}_{{\bm{MRSR}}}} = {{\left({{{\mu }_{MR{{R}_R}}},\,{{\mu }_{MS{{R}_R}}}, \ldots ,{{\mu }_{MR{{R}_{F\left({G - 1} \right)}}}},\,{{\mu }_{MS{{R}_{F\left({G - 1} \right)}}}}} \right)}^T}, \end{array} \end{equation}$$</annotation></semantics></math> </ephtml> and the block‐diagonal dispersion matrices are defined as <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0041" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><munder><mo>∑</mo><mi>MRR</mi></munder><mo linebreak="badbreak">=</mo><mfenced separators="" open="(" close=")">σMRRR200σMRRF12⋯0000⋮⋱⋮0000⋯σMRRFG−2200σMRRFG−12</mfenced><mo>,</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation*}\sum_{{\mathrm{MRR}}} = \left({ \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} { \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {\sigma _{MR{{R}_R}}^2}&0\\ 0&{\sigma _{MR{{R}_{F1}}}^2} \end{array} }& \cdots &{ \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {\ 0\ }&{\ 0}\\ {0\ }&{\ 0} \end{array} }\\ \vdots & \ddots & \vdots \\ { \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {0\ }&0\\ {0\ }&0 \end{array} }& \cdots &{ \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {\sigma _{MR{{R}_{F\left({G - 2} \right)}}}^2}&0\\ 0&{\sigma _{MR{{R}_{F\left({G - 1} \right)}}}^2} \end{array} } \end{array} } \right),\end{equation*}$$</annotation></semantics></math> </ephtml><ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0042" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><munder><mo>∑</mo><mi mathvariant="bold-italic">MSR</mi></munder><mo linebreak="badbreak">=</mo><mfenced separators="" open="(" close=")">σMSRR200σMSRF12⋯0000⋮⋱⋮0000⋯σMSRFG−2200σMSRFG−12</mfenced><mspace width="0.33em" /><mo>,</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation*}\sum_{{\bm{MSR}}} = \left({ \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} { \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {\sigma _{MS{{R}_R}}^2}&0\\ 0&{\sigma _{MS{{R}_{F1}}}^2} \end{array} }& \cdots &{ \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {\ 0\ }&{\ 0}\\ {0\ }&{\ 0} \end{array} }\\ \vdots & \ddots & \vdots \\ { \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {0\ }&{0\ }\\ {0\ }&{0\ } \end{array} }& \cdots &{ \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {\sigma _{MS{{R}_{F\left({G - 2} \right)}}}^2}&0\\ 0&{\sigma _{MS{{R}_{F\left({G - 1} \right)}}}^2} \end{array} } \end{array} } \right)\ ,\end{equation*}$$</annotation></semantics></math> </ephtml> 9 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0043" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><munder><mo>∑</mo><mi mathvariant="bold-italic">MRSR</mi></munder><mo linebreak="badbreak">=</mo><mfenced separators="" open="(" close=")">σMRRR2σMRRR,MSRRσMRRR,MSRRσMSRR2⋯0000⋮⋱⋮0000⋯σMRRFG−12σMRRFG−1,MSRFG−1σMRRFG−1,MSRFG−1σMSRFG−12</mfenced><mspace width="0.33em" /><mo>.</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation}\sum_{{\bm{MRSR}}} = \left({ \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} { \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {\sigma _{MR{{R}_R}}^2}&{{{\sigma }_{\scriptstyle MR{{R}_R},\hfill\atop \scriptstyle MS{{R}_R}\hfill}}}\\ {{{\sigma }_{\scriptstyle MR{{R}_R},\hfill\atop \scriptstyle MS{{R}_R}\hfill}}}&{\sigma _{MS{{R}_R}}^2} \end{array} }& \cdots &{ \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {0\ }&0\\ {0\ }&0 \end{array} }\\ \vdots & \ddots & \vdots \\ { \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {0\ }&0\\ {0\ }&0 \end{array} }& \cdots &{ \def\eqcellsep{&}\begin{array}{@{}*{2}{c}@{}} {\sigma _{MR{{R}_{F\left({G - 1} \right)}}}^2}&{{{\sigma }_{\scriptstyle MR{{R}_{F\left({G - 1} \right)}},\hfill\atop \scriptstyle MS{{R}_{F\left({G - 1} \right)}}\hfill}}}\\ {{{\sigma }_{\scriptstyle MR{{R}_{F\left({G - 1} \right)}},\hfill\atop \scriptstyle MS{{R}_{F\left({G - 1} \right)}}\hfill}}}&{\sigma _{MS{{R}_{F\left({G - 1} \right)}}}^2} \end{array} } \end{array} } \right)\.\end{equation}$$</annotation></semantics></math> </ephtml></p> <p>Now, the GRDIF statistics for multiple groups can be computed using the appropriate contrast matrices (<bold><emph>C</emph></bold>s) defining the groups to be compared. Assuming that all <emph>G</emph> groups are compared simultaneously, <bold><emph>C</emph></bold> has (<emph>G</emph>–1)‐by‐<emph>G</emph> dimensions for the <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0044" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0045" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_S}$</annotation></semantics></math> </ephtml> statistics and 2(<emph>G</emph>–1)‐by‐2<emph>G</emph> dimensions for the <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0046" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> statistic. Due to the asymptotic multivariate normality of <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0047" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi mathvariant="bold-italic">v</mi><mrow><mi>M</mi><mi>R</mi><mi>R</mi></mrow></msub><annotation encoding="application/x-tex">${{{\bm{v}}}_{MRR}}$</annotation></semantics></math> </ephtml> , <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0048" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi mathvariant="bold-italic">v</mi><mrow><mi>M</mi><mi>S</mi><mi>R</mi></mrow></msub><annotation encoding="application/x-tex">${{{\bm{v}}}_{MSR}}$</annotation></semantics></math> </ephtml> , and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0049" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi mathvariant="bold-italic">v</mi><mrow><mi>M</mi><mi>R</mi><mi>S</mi><mi>R</mi></mrow></msub><annotation encoding="application/x-tex">${{{\bm{v}}}_{MRSR}}$</annotation></semantics></math> </ephtml> , the (<emph>G</emph>–1)‐dimensional vectors, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0050" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MRR</mi></msub></mrow><annotation encoding="application/x-tex">${\bm{C}}{{{\bm{\nu }}}_{{\bm{MRR}}}}$</annotation></semantics></math> </ephtml> and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0051" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MSR</mi></msub></mrow><annotation encoding="application/x-tex">${\bm{C}}{{{\bm{\nu }}}_{{\bm{MSR}}}}$</annotation></semantics></math> </ephtml> , and 2(<emph>G</emph>–1)‐dimensional vector, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0052" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MRSR</mi></msub></mrow><annotation encoding="application/x-tex">${\bm{C}}{{{\bm{\nu }}}_{{\bm{MRSR}}}}$</annotation></semantics></math> </ephtml> , have asymptotically multivariate normal distributions. Their corresponding mean vectors are <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0053" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">μ</mi><mi mathvariant="bold-italic">MRR</mi></msub></mrow><annotation encoding="application/x-tex">${\bm{C}}{{{\bm{\mu }}}_{{\bm{MRR}}}}$</annotation></semantics></math> </ephtml> , <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0054" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">μ</mi><mi mathvariant="bold-italic">MRR</mi></msub></mrow><annotation encoding="application/x-tex">${\bm{C}}{{{\bm{\mu }}}_{{\bm{MRR}}}}$</annotation></semantics></math> </ephtml> , and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0055" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">μ</mi><mi mathvariant="bold-italic">MRSR</mi></msub></mrow><annotation encoding="application/x-tex">${\bm{C}}{{{\bm{\mu }}}_{{\bm{MRSR}}}}$</annotation></semantics></math> </ephtml> , respectively, and dispersion matrices are <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0056" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mo>∑</mo><mi mathvariant="bold-italic">MRR</mi></msub><msup><mi mathvariant="bold-italic">C</mi><mi>T</mi></msup></mrow><annotation encoding="application/x-tex">${\bm{C}}\sum_{{\bm{MRR}}} {{{\bm{C}}}^T}$</annotation></semantics></math> </ephtml> , <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0057" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mo>∑</mo><mi mathvariant="bold-italic">MSR</mi></msub><msup><mi mathvariant="bold-italic">C</mi><mi>T</mi></msup></mrow><annotation encoding="application/x-tex">${\bm{C}}\sum_{{\bm{MSR}}} {{{\bm{C}}}^T}$</annotation></semantics></math> </ephtml> , and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0058" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mo>∑</mo><mi mathvariant="bold-italic">MRSR</mi></msub><msup><mi mathvariant="bold-italic">C</mi><mi>T</mi></msup></mrow><annotation encoding="application/x-tex">${\bm{C}}\sum_{{\bm{MRSR}}} {{{\bm{C}}}^T}$</annotation></semantics></math> </ephtml> , respectively (Johnson & Wichern, [<reflink idref="bib17" id="ref84">17</reflink>], p. 165). Then, the three <emph>GRDIF</emph> statistics are computed as <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0059" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub><mo linebreak="badbreak">=</mo><msup><mfenced separators="" open="(" close=")"><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MRR</mi></msub><mo>−</mo><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">μ</mi><mi mathvariant="bold-italic">MRR</mi></msub></mrow></mfenced><mi>T</mi></msup><msup><mfenced separators="" open="(" close=")"><mrow><mi mathvariant="bold-italic">C</mi><munder><mo>∑</mo><mi mathvariant="bold-italic">MRR</mi></munder><msup><mi mathvariant="bold-italic">C</mi><mi>T</mi></msup></mrow></mfenced><mrow><mo>−</mo><mn>1</mn></mrow></msup><mfenced separators="" open="(" close=")"><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MRR</mi></msub><mo>−</mo><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">μ</mi><mi mathvariant="bold-italic">MRR</mi></msub></mrow></mfenced><mover><mo>→</mo><mi>d</mi></mover><msubsup><mi>χ</mi><mrow><mi>d</mi><mi>f</mi><mo>=</mo><mi>G</mi><mo>−</mo><mn>1</mn></mrow><mn>2</mn></msubsup><mo>,</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation*}GRDI{{F}_R} = {{\left({{\bm{C}}{{{\bm{\nu }}}_{{\bm{MRR}}}} - {\bm{C}}{{{\bm{\mu }}}_{{\bm{MRR}}}}} \right)}^T}{{\left({{\bm{C}}\sum_{{\bm{MRR}}} {{{\bm{C}}}^T}} \right)}^{ - 1}}\left({{\bm{C}}{{{\bm{\nu }}}_{{\bm{MRR}}}} - {\bm{C}}{{{\bm{\mu }}}_{{\bm{MRR}}}}} \right)\mathop \to \limits^d \chi _{df = G - 1}^2,\end{equation*}$$</annotation></semantics></math> </ephtml><ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0060" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub><mo linebreak="badbreak">=</mo><msup><mfenced separators="" open="(" close=")"><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MSR</mi></msub><mo>−</mo><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">μ</mi><mi mathvariant="bold-italic">MSR</mi></msub></mrow></mfenced><mi mathvariant="bold-italic">T</mi></msup><msup><mfenced separators="" open="(" close=")"><mrow><mi mathvariant="bold-italic">C</mi><munder><mo>∑</mo><mi mathvariant="bold-italic">MSR</mi></munder><msup><mi mathvariant="bold-italic">C</mi><mi>T</mi></msup></mrow></mfenced><mrow><mo>−</mo><mn>1</mn></mrow></msup><mfenced separators="" open="(" close=")"><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MSR</mi></msub><mo>−</mo><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">μ</mi><mi mathvariant="bold-italic">MSR</mi></msub></mrow></mfenced><mover><mo>→</mo><mi>d</mi></mover><msubsup><mi>χ</mi><mrow><mi>d</mi><mi>f</mi><mo>=</mo><mi>G</mi><mo>−</mo><mn>1</mn></mrow><mn>2</mn></msubsup><mo>,</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation*}GRDI{{F}_S} = {{\left({{\bm{C}}{{{\bm{\nu }}}_{{\bm{MSR}}}} - {\bm{C}}{{{\bm{\mu }}}_{{\bm{MSR}}}}} \right)}^{\bm{T}}}{{\left({{\bm{C}}\sum_{{\bm{MSR}}} {{{\bm{C}}}^T}} \right)}^{ - 1}}\left({{\bm{C}}{{{\bm{\nu }}}_{{\bm{MSR}}}} - {\bm{C}}{{{\bm{\mu }}}_{{\bm{MSR}}}}} \right)\mathop \to \limits^d \chi _{df = G - 1}^2,\end{equation*}$$</annotation></semantics></math> </ephtml> 10 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0061" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub><mo linebreak="badbreak">=</mo><msup><mfenced separators="" open="(" close=")"><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MRSR</mi></msub><mo>−</mo><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">μ</mi><mi mathvariant="bold-italic">MRSR</mi></msub></mrow></mfenced><mi mathvariant="bold-italic">T</mi></msup><msup><mfenced separators="" open="(" close=")"><mrow><mi mathvariant="bold-italic">C</mi><munder><mo>∑</mo><mi mathvariant="bold-italic">MRSR</mi></munder><msup><mi mathvariant="bold-italic">C</mi><mi>T</mi></msup></mrow></mfenced><mrow><mo>−</mo><mn>1</mn></mrow></msup><mfenced separators="" open="(" close=")"><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MRSR</mi></msub><mo>−</mo><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">μ</mi><mi mathvariant="bold-italic">MRSR</mi></msub></mrow></mfenced><mover><mo>→</mo><mi>d</mi></mover><msubsup><mi>χ</mi><mrow><mi>d</mi><mi>f</mi><mo>=</mo><mn>2</mn><mfenced separators="" open="(" close=")"><mrow><mi>G</mi><mo>−</mo><mn>1</mn></mrow></mfenced><mo>,</mo></mrow><mn>2</mn></msubsup></mrow><annotation encoding="application/x-tex">$$\begin{equation}GRDI{{F}_{RS}} = {{\left({{\bm{C}}{{{\bm{\nu }}}_{{\bm{MRSR}}}} - {\bm{C}}{{{\bm{\mu }}}_{{\bm{MRSR}}}}} \right)}^{\bm{T}}}{{\left({{\bm{C}}\sum_{{\bm{MRSR}}} {{{\bm{C}}}^T}} \right)}^{ - 1}}\left({{\bm{C}}{{{\bm{\nu }}}_{{\bm{MRSR}}}} - {\bm{C}}{{{\bm{\mu }}}_{{\bm{MRSR}}}}} \right)\mathop \to \limits^d \chi _{df = 2\left({G - 1} \right),}^2\end{equation}$$</annotation></semantics></math> </ephtml> which all follow asymptotic chi‐squared distributions with degrees of freedom (<emph>df</emph>) as shown in Equation 10. If the statistics exceed a critical percentile value at significance level <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0062" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mi>α</mi><annotation encoding="application/x-tex">$\alpha $</annotation></semantics></math> </ephtml> , the <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0063" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>H</mi><mn>0</mn></msub><annotation encoding="application/x-tex">${{H}_0}$</annotation></semantics></math> </ephtml> of no DIF is rejected.</p> <p>In Equation 10, <bold><emph>C</emph></bold> consists of rows of linearly independent contrasts that are defined based on the type of GRDIF statistic being calculated and the groups being compared. It allows for the testing of the <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0064" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>H</mi><mn>0</mn></msub><annotation encoding="application/x-tex">${{H}_0}$</annotation></semantics></math> </ephtml> that no DIF exists between any pair within all groups. For example, in the case of three groups (e.g., one reference and two focal groups), one way to define <bold><emph>C</emph></bold> for the <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0065" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0066" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_S}$</annotation></semantics></math> </ephtml> is[<reflink idref="bib2" id="ref85">2</reflink>]11 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0067" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><mo linebreak="badbreak">=</mo><mfenced separators="" open="(" close=")">−110−101</mfenced><mo>,</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation}{\bm{C}} = \left({ \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} { - 1}&1&0\\ { - 1}&0&1 \end{array} } \right),\end{equation}$$</annotation></semantics></math> </ephtml> and <bold><emph>C</emph></bold> for the <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0068" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> is expressed as 12 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0069" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><mo linebreak="badbreak">=</mo><mfenced separators="" open="(" close=")">−1010−10000100−1000−10010001</mfenced><mo>.</mo></mrow><annotation encoding="application/x-tex">$$\begin{equation}{\bm{C}} = \left({ \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} { \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} { \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} { - 1}&0&1\\ 0&{ - 1}&0 \end{array} }&{ \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} 0&0&0\\ 1&0&0 \end{array} } \end{array} }\\ { \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} { \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} { - 1}&0&0\\ 0&{ - 1}&0 \end{array} }&{ \def\eqcellsep{&}\begin{array}{@{}*{3}{c}@{}} 0&1&0\\ 0&0&1 \end{array} } \end{array} } \end{array} } \right).\end{equation}$$</annotation></semantics></math> </ephtml></p> <p>Also, for the three groups 13 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0070" xmlns="http://www.w3.org/1998/Math/MathML"><semantics>vMRR=MRRR,MRRF1,MRRF2T,vMSR=MSRR,MSRF1,MSRF2T,vMRSR=MRRR,MSRR,MRRF1,MSRF1,MRRF2,MSRF2T.<annotation encoding="application/x-tex">$$\begin{equation} \def\eqcellsep{&}\begin{array}{l} {{{\bm{v}}}_{{\bm{MRR}}}} = {{\left({MR{{R}_R},MR{{R}_{F1}},MR{{R}_{F2}}} \right)}^T},\\ {{{\bm{v}}}_{{\bm{MSR}}}} = {{\left({MS{{R}_R},MS{{R}_{F1}},MS{{R}_{F2}}} \right)}^T},\\ {{{\bm{v}}}_{{\bm{MRSR}}}} = {{\left({MR{{R}_R},MS{{R}_R},MR{{R}_{F1}},MS{{R}_{F1}},MR{{R}_{F2}},MS{{R}_{F2}}} \right)}^T}. \end{array} \end{equation}$$</annotation></semantics></math> </ephtml></p> <p>Then, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0071" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MRR</mi></msub></mrow><annotation encoding="application/x-tex">${\bm{C}}{{{\bm{\nu }}}_{{\bm{MRR}}}}$</annotation></semantics></math> </ephtml> , <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0072" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MSR</mi></msub></mrow><annotation encoding="application/x-tex">${\bm{C}}{{{\bm{\nu }}}_{{\bm{MSR}}}}$</annotation></semantics></math> </ephtml> , and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0073" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MRSR</mi></msub></mrow><annotation encoding="application/x-tex">${\bm{C}}{{{\bm{\nu }}}_{{\bm{MRSR}}}}$</annotation></semantics></math> </ephtml> yield the vectors of original <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0074" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$RDI{{F}_R}$</annotation></semantics></math> </ephtml> , <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0075" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$RDI{{F}_S}$</annotation></semantics></math> </ephtml> , and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0076" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$RDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> statistics, respectively, as follows: 14 <ephtml> <math display="block" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0077" xmlns="http://www.w3.org/1998/Math/MathML"><semantics>CνMRR=−MRRR+MRRF1−MRRR+MRRF2=RDIFRR,F1RDIFRR,F2,CνMSR=−MSRR+MSRF1−MSRR+MSRF2=RDIFSR,F1RDIFSR,F2,CνMRSR=−MRRR+MRRF1−MSRR+MSRF1−MRRR+MRRF2−MSRR+MSRF2=RDIFRR,F1RDIFSR,F1RDIFRR,F2RDIFSR,F2,<annotation encoding="application/x-tex">$$\begin{equation} \def\eqcellsep{&}\begin{array}{l} {\bm{C}}{{{\bm{\nu }}}_{{\bm{MRR}}}} = \left({ \def\eqcellsep{&}\begin{array}{@{}*{1}{c}@{}} { - MR{{R}_R} + MR{{R}_{F1}}}\\ { - MR{{R}_R} + MR{{R}_{F2}}} \end{array} } \right) = \left({ \def\eqcellsep{&}\begin{array}{@{}*{1}{c}@{}} {RDI{{F}_{R\left({R,F1} \right)}}}\\ {RDI{{F}_{R\left({R,\ F2} \right)}}\ } \end{array} } \right),\\ {\bm{C}}{{{\bm{\nu }}}_{{\bm{MSR}}}} = \left({ \def\eqcellsep{&}\begin{array}{@{}*{1}{c}@{}} { - MS{{R}_R} + MS{{R}_{F1}}}\\ { - MS{{R}_R} + MS{{R}_{F2}}} \end{array} } \right) = \left({ \def\eqcellsep{&}\begin{array}{@{}*{1}{c}@{}} {RDI{{F}_{S\left({R,F1} \right)\ }}}\\ {RDI{{F}_{S\left({R,\ F2} \right)}}\ } \end{array} } \right),\\ {\bm{C}}{{{\bm{\nu }}}_{{\bm{MRSR}}}} = \left({ \def\eqcellsep{&}\begin{array}{@{}*{1}{c}@{}} { - MR{{R}_R} + MR{{R}_{F1}}}\\ { - MS{{R}_R} + MS{{R}_{F1}}}\\ { - MR{{R}_R} + MR{{R}_{F2}}}\\ { - MS{{R}_R} + MS{{R}_{F2}}} \end{array} } \right) = \left({ \def\eqcellsep{&}\begin{array}{@{}*{1}{c}@{}} {RDI{{F}_{R\left({R,\ F1} \right)\ }}}\\ {RDI{{F}_{S\left({R,\ F1} \right)\ }}}\\ {RDI{{F}_{R\left({R,\ F2} \right)\ }}}\\ {RDI{{F}_{S\left({R,\ F2} \right)}}\ } \end{array} } \right), \end{array} \end{equation}$$</annotation></semantics></math> </ephtml> where the letters in parentheses indicate the two groups being compared by each row of <bold><emph>C</emph></bold>. Therefore, the GRDIF approach can be regarded as a multivariate analysis that utilizes the two‐group RDIF statistics.</p> <p>The omnibus test statistics in Equation 10 only indicate the presence of DIF between at least one pair within all groups. Therefore, post‐hoc tests can be additionally conducted after the omnibus significant tests to identify the specific pairs of groups that exhibit DIF while controlling the experiment‐wise error rate (e.g., using a Bonferroni correction). For the post‐hoc analysis, <bold><emph>C</emph></bold> can be used to compare any two groups in a pairwise fashion. In the previous example, <bold><emph>C</emph></bold> for the pairwise comparison between the reference and the first focal groups in <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0078" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> is defined as <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0079" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="bold-italic">C</mi><mo>=</mo><msup><mrow><mo>(</mo>−110<mo>)</mo></mrow><mi>T</mi></msup></mrow><annotation encoding="application/x-tex">${\bm{C}} = {{({ \def\eqcellsep{&}\begin{array}{*{20}{c}} { - 1}&1&0 \end{array} })}^T}$</annotation></semantics></math> </ephtml> , with which <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0080" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msup><mi mathvariant="bold-italic">C</mi><mi mathvariant="bold-italic">T</mi></msup><msub><mi mathvariant="bold-italic">ν</mi><mi mathvariant="bold-italic">MRR</mi></msub></mrow><annotation encoding="application/x-tex">${{{\bm{C}}}^{\bm{T}}}{{{\bm{\nu }}}_{{\bm{MRR}}}}$</annotation></semantics></math> </ephtml> returns the <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0081" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$RDI{{F}_R}$</annotation></semantics></math> </ephtml> statistic for those two groups. This indicates that the significant test results of the GRDIF statistics for two‐group DIF analysis will be equivalent to the significant results of the original RDIF statistics.</p> <p>As with the RDIF statistics for two‐group comparison, items with DIF may increase bias in <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0082" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mover accent="true"><mi>θ</mi><mo>̂</mo></mover><annotation encoding="application/x-tex">$\hat{\theta }$</annotation></semantics></math> </ephtml> , leading to deterioration in detection of DIF. To mitigate this issue, the same process of purification suggested by Lim et al. ([<reflink idref="bib20" id="ref86">20</reflink>]) can be applied.</p> <hd id="AN0181921599-8">Simulation Study</hd> <p>We conducted a simulation study to assess the performance of GRDIF methods in detecting DIF across multiple groups, compared to the existing methods of GMH, Lasso‐DIF, and alignment, under various conditions. The present simulation study aims to answer two main research questions as below:</p> <p></p> <ulist> <item> How accurately can the GRDIF framework identify items with DIF in comparison to the three existing benchmark approaches?</item> <p></p> <item> Which type of DIF can each of the three GRDIF statistics best detect?</item> </ulist> <hd id="AN0181921599-9">Design</hd> <p>In this simulation study, we utilized a 40‐item linear test based on the IRT 2PLM. The test data set was borrowed from Lim et al.'s ([<reflink idref="bib20" id="ref87">20</reflink>]) simulation study, where the true item parameters were drawn from the item bank distribution of a real high‐stakes assessment. See Table A2 in Appendix A for the true item parameters of the test pertaining to the reference group. To manipulate DIF on the items, we adopted the same strategy used in Lim et al.'s ([<reflink idref="bib20" id="ref88">20</reflink>]) simulation study as well (see below for more details on the procedure). With this arrangement, their simulation results can serve as a baseline for the performance of the GRDIF framework in two‐group cases.</p> <p>Regarding the simulated conditions, we manipulated six factors to represent various real testing contexts, including (a) number of groups in total, (b) number of focal groups experiencing DIF, (c) sample size of combined groups, (d) type of DIF, (e) percentage of items with DIF, and (f) impact of ability distributions. The detailed conditions are described below.</p> <hd id="AN0181921599-10">Number of groups in total</hd> <p>Three conditions of groups were simulated: 3, 5, and 8. In each condition, the first group was considered the reference group, and the others were treated as the focal groups. The three conditions were created to represent the multiple groups encountered in real‐world contexts, such as socioeconomic status (e.g., low, medium, and high‐income households), ethnicity groups (e.g., White, Black, African American, Hispanic or Latino, Asian, and others), and language groups (e.g., English, Spanish, Mandarin, Hindi, Arabic, French, German, and Russian). Similar numbers of groups were used in previous studies, including Finch ([<reflink idref="bib11" id="ref89">11</reflink>]) and Penfield ([<reflink idref="bib32" id="ref90">32</reflink>]).</p> <hd id="AN0181921599-11">Number of focal groups experiencing DIF</hd> <p>When examining DIF for multiple groups, it is possible that DIF may exist between different pairs of groups (e.g., Penfield, [<reflink idref="bib32" id="ref91">32</reflink>]). In this study, we simulated two scenarios where the focal groups experienced DIF compared to the reference group. In the first condition, only one of the focal groups exhibited DIF, regardless of the total number of groups. In the second condition, all focal groups were disadvantaged due to DIF. For example, in the five group conditions, all four focal groups exhibited DIF.</p> <hd id="AN0181921599-12">Sample size of combined groups</hd> <p>To determine realistic sample sizes for different groups when calibrating items using the IRT 2PLM, we employed total sample sizes ranging from 400 (minimum) to 2,400 (maximum). These sizes reflect the range of small to large sample conditions typically encountered in high‐stakes assessments. Proportions of equal (i.e., 150R/150Fs and 300R/300Fs) and unequal (i.e., 200R/100Fs and 400R/200Fs) group sizes were adopted to reflect real‐world scenarios, resulting in 12 combinations of total group sizes. To provide a systematically organized representation of the combined sample size conditions in the simulation results, the 12 conditions are abbreviated as in Table 2.</p> <hd id="AN0181921599-13">Type of DIF</hd> <p>Three types of DIF were studied: uniform, nonuniform, and mixed. All the items displaying DIF were adjusted so that they better discriminated the examinees in the reference group (i.e., lower <emph>a</emph>‐parameters for the focal groups) and/or they were more difficult to the examinees in the focal groups (i.e., higher <emph>b</emph>‐parameters for the focal groups). The same technique was also employed by Lim et al. ([<reflink idref="bib20" id="ref92">20</reflink>]).</p> <hd id="AN0181921599-14">Percentage of items with DIF</hd> <p>Three different percentages of items with DIF were simulated: 0%, 10%, and 20%. In cases of no items exhibiting DIF, all items in the test had identical parameters across all groups. To create the items with DIF, we employed the same technique used by Lim et al. ([<reflink idref="bib20" id="ref93">20</reflink>]) as mentioned earlier. We systematically altered the degree of DIF for certain items by adding or subtracting a constant vector of (.3, .5, .7, .9) to either the <emph>a‐</emph> or <emph>b</emph>‐parameters for the focal groups experiencing DIF. For instance, to simulate the 10% mixed DIF conditions, the constant vector was added to the <emph>b</emph>‐parameters and subtracted from the <emph>a</emph>‐parameters of the first four items (Items 1 through 4) for the focal groups. For the 20% mixed DIF conditions, the same vector was reused to modify the parameters of subsequent four items (Items 5 through 8). Uniform and nonuniform DIF were generated by applying the same technique to either the <emph>a</emph>‐parameters or <emph>b</emph>‐parameters, respectively. The parameter variations of .3 to .5 are deemed as slight to moderate DIF, while .7 to .9 are regarded as large DIF (e.g., Finch, [<reflink idref="bib11" id="ref94">11</reflink>]; Woods et al., [<reflink idref="bib41" id="ref95">41</reflink>]). Table 2 provides an illustration of the true parameters of the first 8 items for both the reference group and focal groups experiencing DIF under mixed DIF conditions.</p> <hd id="AN0181921599-15">Impact of ability distributions</hd> <p>Two impact conditions were simulated: mean latent ability differences of 0 and 1. For the no impact conditions, the true latent abilities of all groups were randomly sampled from <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0083" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>N</mi><mo>(</mo><mrow><mn>0</mn><mo>,</mo><mspace width="0.33em" /><mn>1</mn></mrow><mo>)</mo></mrow><annotation encoding="application/x-tex">$N({0,\ 1})$</annotation></semantics></math> </ephtml> . For the impact conditions of 1, true ability of the reference group <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0084" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>θ</mi><mi>R</mi></msub><annotation encoding="application/x-tex">${{\theta }_R}$</annotation></semantics></math> </ephtml> was randomly drawn from <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0085" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>N</mi><mo>(</mo><mrow><mo>.</mo><mn>5</mn><mo>,</mo><mspace width="0.33em" /><mn>1</mn></mrow><mo>)</mo></mrow><annotation encoding="application/x-tex">$N({.5,\ 1})$</annotation></semantics></math> </ephtml> , while the true abilities of all focal groups <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0086" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>θ</mi><mrow><mi>F</mi><mi>s</mi></mrow></msub><annotation encoding="application/x-tex">${{\theta }_{Fs}}$</annotation></semantics></math> </ephtml> were sampled from <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0087" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>N</mi><mo>(</mo><mrow><mo>−</mo><mo>.</mo><mn>5</mn><mo>,</mo><mspace width="0.33em" /><mn>1</mn></mrow><mo>)</mo></mrow><annotation encoding="application/x-tex">$N({ -.5,\ 1})$</annotation></semantics></math> </ephtml> . Although the mean ability levels may vary across focal groups, we chose to use equal means across all groups to manage the scope of the simulated conditions. The impact size of 1 represents realistic testing situations where the reference group has a substantially higher mean ability than the focal groups (e.g., Finch & French, [<reflink idref="bib12" id="ref96">12</reflink>]; Lei et al., [<reflink idref="bib19" id="ref97">19</reflink>]; Nandakumar & Roussos, [<reflink idref="bib31" id="ref98">31</reflink>]; Penfield, [<reflink idref="bib32" id="ref99">32</reflink>]).</p> <hd id="AN0181921599-16">Analysis</hd> <p>To assess the effectiveness of six different approaches for detecting DIF across multiple groups, two primary criteria were used: (a) Type I error rate, measured as the percentage of items without DIF incorrectly identified as having DIF; and (b) power, measured as the percentage of items with DIF correctly identified as having DIF. In all cases, a nominal alpha ( <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0088" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mi>α</mi><annotation encoding="application/x-tex">$\alpha $</annotation></semantics></math> </ephtml> ) level of .05 was employed for a hypothesis testing.</p> <p>Before conducting the GRDIF analysis for each condition, we calibrated the 40 test items using the <emph>est_irt</emph> function in the R (R Core Team, [<reflink idref="bib33" id="ref100">33</reflink>]) package irtQ (Lim & Wells, [<reflink idref="bib22" id="ref101">22</reflink>]) based on the aggregated data of all groups. To estimate the item parameters, we utilized the marginal maximum likelihood estimation with the expectation‐maximization (MMLE‐EM) algorithm (Bock & Aitkin, [<reflink idref="bib8" id="ref102">8</reflink>]). We set the convergence criterion and maximum cycle of the E‐step to .001 and 1,000, respectively. To calculate the MLE scores ( <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0089" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>θ</mi><mo>̂</mo></mover><mi mathvariant="normal">s</mi></mrow><annotation encoding="application/x-tex">$\hat{\theta }{\mathrm{s}}$</annotation></semantics></math> </ephtml> ) restricted within [–5.0, 5.0], we used the <emph>est_score</emph> function, followed by the GRDIF analysis using the <emph>grdif</emph> function, both from the <bold>irtQ</bold> R package.</p> <p>The GMH method was executed using the <emph>difGMH</emph> function in the difR R package (Magis et al., [<reflink idref="bib23" id="ref103">23</reflink>]), while the Lasso‐DIF method was carried out using the <emph>regDIF</emph> function in the regDIF R package (Belzak, [<reflink idref="bib5" id="ref104">5</reflink>]). For the Lasso‐DIF method, the item and DIF parameters were estimated using the MMLE‐EM with the same convergence criterion and maximum cycle of the E‐step as those set in the <emph>est_irt</emph> function. To determine the optimal tuning parameter <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0090" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mi>τ</mi><annotation encoding="application/x-tex">$\tau $</annotation></semantics></math> </ephtml> for the best‐fitting regularization model with a lasso penalty, the total number of <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0091" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mi>τ</mi><annotation encoding="application/x-tex">$\tau $</annotation></semantics></math> </ephtml> values was set to 50 in the <emph>regDIF</emph> function, which were then automatically generated by default. The <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0092" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mi>τ</mi><annotation encoding="application/x-tex">$\tau $</annotation></semantics></math> </ephtml> value with the smallest Bayesian Information Criterion (BIC) was selected as the best‐fitting model. For more detailed model parameterization of Lasso‐DIF, see Appendix B.</p> <p>The alignment method was implemented using Mplus 8.1 (Muthén & Muthén, [<reflink idref="bib30" id="ref105">30</reflink>]) with MLE and free alignment optimization. During the estimation process, the default value of <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0093" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>ε</mi><mo>=</mo><mo>.</mo><mn>01</mn></mrow><annotation encoding="application/x-tex">$\epsilon =.01$</annotation></semantics></math> </ephtml> was set for the component loss function (CLF). Also, the "TYPE = Mixture" and "Parameterization = LOGIT" settings were selected to implement the IRT multiple‐group analysis in the alignment method. For more detailed procedure of alignment method, see Appendix B. It is important to note that our use of Mplus 8.1 imposes some software limitations. Specifically, while the recent study by Asparouhov and Muthén ([<reflink idref="bib2" id="ref106">2</reflink>]) introduced the availability of theta or delta parameterization and more advanced estimators for categorical data (e.g., WLS, WLSM, WLSMV, and ULS) for the alignment method, these options are only supported in Mplus starting from version 8.6. As we used Mplus 8.1 for this study, these newer options were not available.</p> <p>In addition, instead of using the postestimation procedure of the alignment method suggested by Asparouhov and Muthén ([<reflink idref="bib1" id="ref107">1</reflink>]), we used a simpler postestimation procedure to test the <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0094" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi>H</mi><mn>0</mn></msub><annotation encoding="application/x-tex">${{H}_0}$</annotation></semantics></math> </ephtml> , which posits that DIF is absent across all group pairs. For each parameter under consideration, we conducted multiple pairwise testes between the reference group and each of the focal groups. Given <emph>G</emph> total groups, this results in <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0095" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mo>−</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">$G - 1$</annotation></semantics></math> </ephtml> comparisons. Considering both <emph>a</emph>‐ and <emph>b</emph>‐parameters were examined for DIF, the procedure necessitated <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0096" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>2</mn><mo>(</mo><mrow><mi>G</mi><mo>−</mo><mn>1</mn></mrow><mo>)</mo></mrow><annotation encoding="application/x-tex">$2({G - 1})$</annotation></semantics></math> </ephtml> pairwise tests. To control the familywise Type I error rate, we applied a Bonferroni‐corrected significance level, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0097" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msup><mi>α</mi><mo>∗</mo></msup><mo>=</mo><mfrac><mi>α</mi><mrow><mn>2</mn><mo>(</mo><mrow><mi>G</mi><mo>−</mo><mn>1</mn></mrow><mo>)</mo></mrow></mfrac></mrow><annotation encoding="application/x-tex">${{\alpha }^*} = \frac{\alpha }{{2({G - 1})}}$</annotation></semantics></math> </ephtml> (e.g., <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0098" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msup><mi>α</mi><mo>∗</mo></msup><mo>=</mo><mo>.</mo><mn>00625</mn></mrow><annotation encoding="application/x-tex">${{\alpha }^*} =.00625$</annotation></semantics></math> </ephtml> with <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0099" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>α</mi><mo>=</mo><mo>.</mo><mn>05</mn></mrow><annotation encoding="application/x-tex">$\alpha =.05$</annotation></semantics></math> </ephtml> and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0100" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mo>=</mo><mn>5</mn></mrow><annotation encoding="application/x-tex">$G = 5$</annotation></semantics></math> </ephtml> ). A significant difference in any pair indicates the presence of DIF in at least one group pairing. Although our procedure does not specify all potential group pairs exhibiting DIF, unlike Asparouhov and Muthén's ([<reflink idref="bib1" id="ref108">1</reflink>]) ad hoc procedure, it aligns more closely with the omnibus DIF detection approach used in this simulation study.</p> <p>To enhance the accuracy of DIF detection, we utilized purification procedures for the GRDIF and GMH approaches, setting the maximum iterations to 20 for all simulated conditions. Further details on the purification procedures of the GRDIF framework and the GMH method can be found in Lim et al. ([<reflink idref="bib20" id="ref109">20</reflink>]) and Magis et al. ([<reflink idref="bib23" id="ref110">23</reflink>]), respectively. Note that Lasso‐DIF and alignment methods do not require purification. The study on the six DIF detection methods was replicated 100 times for each simulated condition.</p> <hd id="AN0181921599-17">Results</hd> <p>The simulation results for the six DIF detection approaches, including three GRDIF approaches, GMH, Lasso‐DIF, and alignment methods, are presented in the following figures. In the figures, the <emph>x</emph>‐axis of each panel indicates the combined sample size (see Table 1 for descriptions of the abbreviations), and the <emph>y</emph>‐axis represents the Type I error rate or power. Additionally, the outcomes are summarized as the average Type I error rate and power across studied items in each condition.</p> <p>1 Table Abbreviations and Descriptions of 12 Combined Sample Size Conditions</p> <p> <ephtml> <table><thead><tr><th align="left">Abbreviation</th><th align="center">Description</th></tr></thead><tbody><tr><td>UG3CN400</td><td>Unbalanced 3 groups (200R/100Fs) with a combined group size of 400</td></tr><tr><td>BG3CN450</td><td>Balanced 3 groups (150R/150Fs) with a combined group size of 450</td></tr><tr><td>UG5CN600</td><td>Unbalanced 5 groups (200R/100Fs) with a combined group size of 600</td></tr><tr><td>BG5CN750</td><td>Balanced 5 groups (150R/150Fs) with a combined group size of 750</td></tr><tr><td>UG8CN900</td><td>Unbalanced 8 groups (200R/100Fs) with a combined group size of 900</td></tr><tr><td>BG8CN1200</td><td>Balanced 8 groups (150R/150Fs) with a combined group size of 1,200</td></tr><tr><td>UG3CN800</td><td>Unbalanced 3 groups (400R/200Fs) with a combined group size of 800</td></tr><tr><td>BG3CN900</td><td>Balanced 3 groups (300R/300Fs) with a combined group size of 900</td></tr><tr><td>UG5CN1200</td><td>Unbalanced 5 groups (400R/200Fs) with a combined group size of 1,200</td></tr><tr><td>BG5CN1500</td><td>Balanced 5 groups (300R/300Fs) with a combined group size of 1,500</td></tr><tr><td>UG8CN1800</td><td>Unbalanced 8 groups (400R/200Fs) with a combined group size of 1,800</td></tr><tr><td>BG8CN2400</td><td>Balanced 8 groups (300R/300Fs) with a combined group size of 2,400</td></tr></tbody></table> </ephtml> </p> <p>2 Table True Item Parameters of the First Eight Items for the Reference and Focal Groups Experiencing DIF under Mixed DIF Conditions</p> <p> <ephtml> <table><thead><tr><th>Item</th><th align="center">Reference</th><th align="center">Focal</th><th align="center">Amount of DIF</th></tr><tr><th align="left" /><th align="left" /><th align="left" /><th align="center">10% DIF</th><th align="center">20% DIF</th><th align="left" /></tr></thead><tbody><tr><td /><td><p><math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0101" xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mi>a</mi><mi>R</mi><annotation encoding="application/x-tex">${{a}_R}$</annotation></semantics></math></p></td><td><p><math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0102" xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mi>b</mi><mi>R</mi><annotation encoding="application/x-tex">${{b}_R}$</annotation></semantics></math></p></td><td><p><math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0103" xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mi>a</mi><mi>F</mi><mi>s</mi><annotation encoding="application/x-tex">${{a}_{Fs}}$</annotation></semantics></math></p></td><td><p><math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0104" xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mi>b</mi><mi>F</mi><mi>s</mi><annotation encoding="application/x-tex">${{b}_{Fs}}$</annotation></semantics></math></p></td><td><p><math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0105" xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mi>a</mi><mi>F</mi><mi>s</mi><annotation encoding="application/x-tex">${{a}_{Fs}}$</annotation></semantics></math></p></td><td><p><math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0106" xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mi>b</mi><mi>F</mi><mi>s</mi><annotation encoding="application/x-tex">${{b}_{Fs}}$</annotation></semantics></math></p></td><td /></tr><tr><td>1</td><td>1.67</td><td>−1.10</td><td>1.37</td><td>–.80</td><td>1.37</td><td>–.80</td><td>.3</td></tr><tr><td>2</td><td>2.18</td><td>.79</td><td>1.68</td><td>1.29</td><td>1.68</td><td>1.29</td><td>.5</td></tr><tr><td>3</td><td>1.95</td><td>−1.33</td><td>1.25</td><td>–.63</td><td>1.25</td><td>–.63</td><td>.7</td></tr><tr><td>4</td><td>1.50</td><td>.30</td><td>.60</td><td>1.20</td><td>.60</td><td>1.20</td><td>.9</td></tr><tr><td>5</td><td>2.77</td><td>–.69</td><td>‐</td><td>‐</td><td>2.47</td><td>–.39</td><td>.3</td></tr><tr><td>6</td><td>1.09</td><td>.28</td><td>‐</td><td>‐</td><td>.59</td><td>.78</td><td>.5</td></tr><tr><td>7</td><td>1.57</td><td>–.11</td><td>‐</td><td>‐</td><td>.87</td><td>.59</td><td>.7</td></tr><tr><td>8</td><td>1.27</td><td>1.32</td><td>‐</td><td>‐</td><td>.37</td><td>2.22</td><td>.9</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note</emph>. The true item parameters in this table were obtained from Lim et al. ([<reflink idref="bib20" id="ref111">20</reflink>], p. 11). Also, the blank cells indicate that the item parameters of the focal groups are identical with those of the reference group.</p> <hd id="AN0181921599-18">Type I error</hd> <p>We first assessed Type I errors of all six methods in the absence of DIF in a test (see Figure 1). All methods displayed well‐controlled Type I errors, predominantly around or below .05, regardless of the simulated conditions. Notably, Type I errors of the three GRDIF and GMH methods were consistently close to the nominal level of .05, oscillating within .03 and .06. The alignment method, meanwhile, produced slightly lower Type I errors, fluctuating between .01 and .03. The Lasso‐DIF approach exhibited the lowest Type I errors among all methods, consistently approaching zero, thus suggesting that the Lasso‐DIF approach may be overly conservative in detecting DIF in our simulation study.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/MEA/01dec24/jedm12415-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="jedm12415-fig-0001.jpg" title="1 Type I error rates with no items exhibiting DIF.Note. GRDIF(R) = GRDIFR; GRDIF(S) = GRDIFS; GRDIF(RS) = GRDIFRS; LASSO = Lasso‐DIF; ALIGN = Alignment." /> </p> <p></p> <p>In conditions where some items exhibit DIF in a test, the patterns of Type I error and power for all six approaches were similar between scenarios with and without impact. To streamline the presentation of results, the simulation outcomes with impact are presented in the main figures (Figures 2–5), while those for zero impact are relegated to Appendix C (Figures C1–C4).</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/MEA/01dec24/jedm12415-fig-0002.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="jedm12415-fig-0002.jpg" title="2 Type I error rates with items exhibiting DIF under impact = 1 and balanced groups.Note. GRDIF(R) = GRDIFR; GRDIF(S) = GRDIFS; GRDIF(RS) = GRDIFRS; LASSO = Lasso‐DIF; ALIGN = Alignment; DIF.Pct = percentage of items with DIF; No.DIF = number of focal groups experiencing DIF." /> </p> <p></p> <p>Figures 2 and 3 show the Type I error rates of the six approaches for balanced and unbalanced group conditions, respectively, in tests where items exhibit DIF with impact size of 1 (see Figures C1 and C2 for conditions with zero impact). The results in both figures demonstrated that all DIF detection approaches generally maintained good control over Type I errors across various conditions when DIF items were present in a test. Echoing the findings in Figure 1, the three GRDIF and GMH methods consistently showed Type I errors around the .05 level. In contrast, the alignment method yielded slightly lower Type I errors compared to the three GRDIF and GMH methods. The Lasso‐DIF method still displayed relatively much lower Type I errors than all other methods, with a few exceptions (e.g., conditions with uniform DIF and larger combined sample sizes, such as BG3CN900, BG5CN1500, and BG8CN2400, particularly when only one focal group contained DIF). Note that the Type I error results for the GRDIF methods are consistent with the simulation findings of Lim et al. ([<reflink idref="bib20" id="ref112">20</reflink>]). Their study showed that none of the three RDIF methods for two‐groups exhibited significantly inflated Type I error rate, even up to 20% of items with DIF in a test when the purification procedure was applied, regardless of other simulated conditions.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/MEA/01dec24/jedm12415-fig-0003.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="jedm12415-fig-0003.jpg" title="3 Type I error rates with items exhibiting DIF under impact = 1 and unbalanced groups.Note. GRDIF(R) = GRDIFR; GRDIF(S) = GRDIFS; GRDIF(RS) = GRDIFRS; LASSO = Lasso‐DIF; ALIGN = Alignment; DIF.Pct = percentage of items with DIF; No.DIF = number of focal groups experiencing DIF." /> </p> <p></p> <hd id="AN0181921599-22">Power</hd> <p>Figures 4 and 5 provide insights into the statistical power for balanced and unbalanced group conditions, respectively, with impact size of 1 (see Figures C3 and C4 for conditions with zero impact). Consistent patterns emerge from both figures, indicating that larger group sizes correlate with increased detection rates for all methods, holding all other conditions constant. Additionally, the pattern of power for the six approaches varies depending on whether one or all focal groups exhibit DIF, as well as whether group sizes are balanced or unbalanced with other conditions controlled. Specifically, when a single focal group displays DIF, all methods tend to show slightly higher power within balanced groups (e.g., BG3CN900 in Figure 4) compared to unbalanced ones (e.g., UG3CN800 in Figure 5) under otherwise equal conditions. Conversely, when all focal groups manifest DIF, this trend is often reversed under various conditions. Figure 6 succinctly illustrates these observations, revealing the power across all methods when the number of DIF‐afflicted focal groups intersects with the type of DIF, aggregating over all other factors.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/MEA/01dec24/jedm12415-fig-0004.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="jedm12415-fig-0004.jpg" title="4 Power rates of detecting DIF under impact = 1 and balanced groups.Note. GRDIF(R) = GRDIFR; GRDIF(S) = GRDIFS; GRDIF(RS) = GRDIFRS; LASSO = Lasso‐DIF; ALIGN = Alignment; DIF.Pct = percentage of items with DIF; No.DIF = number of focal groups experiencing DIF." /> </p> <p></p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/MEA/01dec24/jedm12415-fig-0005.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="jedm12415-fig-0005.jpg" title="5 Power rates of detecting DIF under impact = 1 and unbalanced groups.Note. GRDIF(R) = GRDIFR; GRDIF(S) = GRDIFS; GRDIF(RS) = GRDIFRS; LASSO = Lasso‐DIF; ALIGN = Alignment; DIF.Pct = percentage of items with DIF; No.DIF = number of focal groups experiencing DIF." /> </p> <p></p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/MEA/01dec24/jedm12415-fig-0006.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="jedm12415-fig-0006.jpg" title="6 Aggregated power comparison by type of DIF and the number of focal groups exhibiting DIF.Note. GRDIF(R) = GRDIFR; GRDIF(S) = GRDIFS; GRDIF(RS) = GRDIFRS; LASSO = Lasso‐DIF; ALIGN = Alignment; No.DIF = number of focal groups experiencing DIF." /> </p> <p></p> <p>We also identified interesting trends in the effectiveness of the six methods by averaging their power across certain conditions. To better illustrate these findings, we included additional plots in Figures C5‐C7 of Appendix C, as they are not easily discernible from Figures 4 and 5 as well as Figures C3 and C4. On average, uniform and mixed DIF were more readily detected by most methods in the absence of impact, as shown in Figure C5. When all focal groups display DIF, similar trends emerged, except for the Lasso‐DIF method, which was less effective in such scenarios (refer to Figure C6). The power to detect nonuniform DIF, however, did not follow a consistent pattern across conditions. Furthermore, we found that all six methods generally had higher power in detecting uniform and mixed DIF when 10% of items had DIF compared to 20%, but the opposite was observed for nonuniform DIF (refer to Figure C7).</p> <p>Regarding the overall performance of three GRDIF methods compared to the benchmarks, each of them exceled in identifying the specific type of DIF for which it was designed. See Figure C8 in Appendix C, which presents the marginalized powers of six methods for each type of DIF. For uniform DIF, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0107" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> and GMH led in detection power, with <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0108" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> following closely. The alignment, Lasso‐DIF, and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0109" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>s</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_s}$</annotation></semantics></math> </ephtml> , however, fell behind. For nonuniform DIF, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0110" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_S}$</annotation></semantics></math> </ephtml> stood out, with <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0111" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> and alignment methods trailing just behind. As anticipated, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0112" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> and GMH were less potent here, and Lasso‐DIF showed significantly reduced power. For mixed DIF, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0113" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> was the most powerful, followed in order by <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0114" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> , GMH, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0115" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_S}$</annotation></semantics></math> </ephtml> and alignment methods. Lasso‐DIF exhibited the least power in detecting mixed DIF. Among the three GRDIF methods, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0116" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> showcased the best overall performance in identifying all types of DIF, maintaining high power and controlled Type I errors. This robustness is consistent with Lim et al.'s ([<reflink idref="bib20" id="ref113">20</reflink>]) findings in the simulation study with two groups, where <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0117" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$RDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> proved to be a versatile and reliable detector of various DIF types, balancing power and Type I error effectively.</p> <hd id="AN0181921599-26">Empirical Study</hd> <p>To demonstrate the utility of the GRDIF framework, we conducted a DIF analysis using empirical data sourced from a retired linear test form of a high‐stakes assessment. The test consists of 31 quantitative reasoning items, calibrated using the IRT 2PLM. For the group variable, we chose ethnicity, subsequently categorizing the 877 test takers into four groups. The largest group had 515 (57.8%) test takers, followed by groups of 168 (19.2%), 152 (15.2%), and 42 (4.8%) test takers. Their mean <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0118" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>θ</mi><mo>̂</mo></mover><mi mathvariant="normal">s</mi></mrow><annotation encoding="application/x-tex">$\hat{\theta }{\mathrm{s}}$</annotation></semantics></math> </ephtml> on the <emph>N</emph>(0, 1) scale were .06, .80, –.37, and .40, respectively. We employed six detection approaches— <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0119" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> , <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0120" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_S}$</annotation></semantics></math> </ephtml> , <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0121" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> , GMH, Lasso‐DIF, and alignment—all executed using settings and procedures consistent with those in the simulation study.</p> <p>Of the 31 items, eight were marked as potentially exhibiting DIF by at least one approach given <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0122" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>α</mi><mo>=</mo><mo>.</mo><mn>05</mn></mrow><annotation encoding="application/x-tex">$\alpha =.05$</annotation></semantics></math> </ephtml> (see Table 3). The Lasso‐DIF approach, in contrast to the other methods, did not flag any items, potentially due to its lower detection rates in scenarios with more than three groups, as observed in the simulation study. Items 15, 21, and 28, flagged by three or four methods, show a strong likelihood of DIF. Notably, both <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0123" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> and GMH consistently identified the same four items (Items 7, 14, 21, and 28), suggesting the presence of uniform DIF in at least one group pairing. Additionally, <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0124" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_S}$</annotation></semantics></math> </ephtml> and the alignment method concurred on Items 9, 15, and 21, indicating potential nonuniform DIF at least one group pairing among the groups. <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0125" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> highlighted three items, uniquely identifying Item 31 and concurring with <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0126" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_S}$</annotation></semantics></math> </ephtml> and the alignment method on Item 15, and with <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0127" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> and GMH on Item 28. Item 30, flagged solely by the alignment method as potentially exhibiting DIF, was not identified by any of the GRDIF approaches.</p> <p>3 Table Eight Suspicious Items with Potential DIF among 31 Quantitative Reasoning Items</p> <p> <ephtml> <table><thead><tr><th>Item</th><th align="center"><p><math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0128" xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math></p></th><th align="center"><p><math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0129" xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_S}$</annotation></semantics></math></p></th><th align="center"><p><math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0130" xmlns="http://www.w3.org/1998/Math/MathML"><semantics xmlns=""><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math></p></th><th align="center">GMH</th><th align="center">Lasso‐DIF</th><th align="center">Alignment</th></tr></thead><tbody><tr><td>7</td><td>✓</td><td /><td /><td>✓</td><td /><td /></tr><tr><td>9</td><td /><td>✓</td><td /><td /><td /><td>✓</td></tr><tr><td>14</td><td>✓</td><td /><td /><td>✓</td><td /><td /></tr><tr><td>15</td><td /><td>✓</td><td>✓</td><td /><td /><td>✓</td></tr><tr><td>21</td><td>✓</td><td>✓</td><td /><td>✓</td><td /><td>✓</td></tr><tr><td>28</td><td>✓</td><td /><td>✓</td><td>✓</td><td /><td /></tr><tr><td>30</td><td /><td /><td /><td /><td /><td>✓</td></tr><tr><td>31</td><td /><td /><td>✓</td><td /><td /><td /></tr></tbody></table> </ephtml> </p> <p>2 <emph>Note</emph>. The check mark (✓) indicates significant hypothesis testing results at <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0131" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>α</mi><mo>=</mo><mo>.</mo><mn>05</mn></mrow><annotation encoding="application/x-tex">$\alpha =.05$</annotation></semantics></math> </ephtml> .</p> <p>In total, seven of the eight flagged items were identified by at least one of the three GRDIF methods. Each method identified a different subset of items, indicating that they may have distinct strengths in detecting various types of DIF across different group pairings. This suggests that the GRDIF framework is useful for evaluating DIF in multi‐group contexts. As highlighted in the simulation study, however, the statistical testing results of the GRDIF framework, like those of other DIF detection methods, may be subject to Type I errors. Therefore, it is important to interpret these results with caution. Content experts should carefully review the flagged items before taking any further actions based on these findings.</p> <hd id="AN0181921599-27">Discussion</hd> <p>This study aimed to extend the RDIF detection framework for analyzing DIF in multiple groups. Through a simulation study, we investigated the performance of the GRDIF framework in comparison to existing DIF detection methods, including GMH, Lasso‐DIF, and alignment. The results revealed that the GRDIF framework mirrored the qualities of the original RDIF framework by effectively detecting uniform, nonuniform, and mixed DIF in multiple groups with a conceptually simple, statistically robust, and computationally efficient methodology.</p> <p>More specifically, the GRDIF framework demonstrated well‐controlled Type I error rates and satisfactory power in detecting various types of DIF across different simulated conditions. Each of the three GRDIF statistics was particularly effective in detecting the specific type of DIF it was designed for, with <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0132" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> exhibiting the most robust performance across all types of DIF. While <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0133" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mrow><mi>R</mi><mi>S</mi></mrow></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_{RS}}$</annotation></semantics></math> </ephtml> offers the most desirable and flexible capability for detecting DIF and can be used as the primary DIF detection criterion, as recommended by Lim et al. ([<reflink idref="bib20" id="ref114">20</reflink>]), <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0134" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_R}$</annotation></semantics></math> </ephtml> and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0135" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>S</mi></msub></mrow><annotation encoding="application/x-tex">$GRDI{{F}_S}$</annotation></semantics></math> </ephtml> also demonstrated excellent performance in detecting uniform and nonuniform DIF, respectively. Therefore, depending on the type of DIF that is of greater concern in a testing program, they can also be used as the primary tool for assessing DIF of test items, as suggested by Lim and Choe ([<reflink idref="bib21" id="ref115">21</reflink>]). Furthermore, the GRDIF framework outperformed other DIF detection methods in the majority of conditions. These findings support the effectiveness of the GRDIF framework for screening items with potential DIF and suggest its practical utility in real‐world applications.</p> <p>Because the GRDIF statistics are employed for omnibus hypothesis tests, it is important to note that statistically significant results do not explicitly indicate the specific groups for which DIF might be present in flagged items. In such instances, conducting a post‐hoc pairwise analysis using the original RDIF framework can help identify the particular groups affected by DIF. Alternatively, as mentioned earlier, the contrast design matrix <bold><emph>C</emph></bold> can be easily modified to enable a specific pairwise comparison of interest to compute the GRDIF statistics. Both approaches yield identical results because they are theoretically equivalent.</p> <p>In terms of the three benchmark methods—GMH, Lasso‐DIF, and alignment—employed alongside the GRDIF framework, each offer unique insights into the detection of DIF with multiple groups. The GMH method could be praised for its computational ease and effectiveness, particularly in identifying uniform DIF, though it showed less capability in detecting nonuniform DIF. The alignment method, with its streamlined postestimation process involving multiple pairwise comparisons, adeptly managed Type I error rates. However, its power fell short of the GRDIF framework, notably in identifying uniform and mixed DIF. This diminished power could be attributed to the conservative nature of the Bonferroni correction applied to manage the family‐wise Type I error—an issue commonly encountered with multiple pairwise tests. The Bonferroni correction, while reducing the likelihood of false positives, may be excessively stringent, potentially underpowering the test for items with small to moderate amount of DIF, as noted by Stark et al. ([<reflink idref="bib36" id="ref116">36</reflink>]). Therefore, the choice of critical <emph>p</emph>‐value should be a pivotal consideration when implementing the alignment method with the simplified postestimation procedure.</p> <p>While the Lasso‐DIF method has shown promise in DIF detection, our simulation studies revealed that its power was significantly lower in most scenarios, particularly where all focal groups displayed DIF. It struggled notably with nonuniform DIF, and its power decreased substantially with smaller group sizes. Considering previous findings that the BIC can be more conservative than Akaike's Information Criterion (AIC) for small samples (e.g., Huang, [<reflink idref="bib16" id="ref117">16</reflink>]; Magis et al., [<reflink idref="bib26" id="ref118">26</reflink>]), we explored using AIC for optimal tuning parameter selection in Lasso‐DIF under the identical simulation conditions used in this study. However, this approach resulted in substantially inflated Type I errors. This underscores the necessity for additional research to determine the most suitable conditions for the Lasso‐DIF method. Moreover, larger group sizes could potentially enhance its detection capability within the IRT 2PLM context, aligning with Belzak and Bauer ([<reflink idref="bib6" id="ref119">6</reflink>]), who observed that Lasso‐DIF is most effective with large sample sizes.</p> <p>Despite the valuable insights provided by this study, it is important to acknowledge limitations that require further research to address. The present study employed a limited number of simulated conditions, which may not fully represent the complexities encountered in real‐world situations. For instance, we only considered three scenarios with different numbers of groups (three, five, and eight groups). In contrast, large‐scale international assessments, such as PISA, typically involve a substantially larger number of country groups. Consequently, future research should evaluate how well the GRDIF framework can detect DIF with a larger number of groups. Additionally, the simulation assumed that all focal groups possessed the same mean abilities, while in reality each focal group may exhibit different mean ability, potentially influencing the performance of the GRDIF framework. As highlighted by Lim et al. ([<reflink idref="bib20" id="ref120">20</reflink>]) and Lim and Choe ([<reflink idref="bib21" id="ref121">21</reflink>]), the efficiency of the GRDIF framework could be compromised with shorter tests because of potential inaccuracies in recovering item and ability parameters. Therefore, future studies should investigate the performance of GRDIF across different test lengths. In addition, we recognize the importance of both DIF magnitude and direction, as highlighted by reviewers. These are vital factors that warrant further exploration in subsequent studies.</p> <p>Also, the simulation exclusively focused on dichotomously scored items with unidimensional IRT models. Furthermore, the study assumed that the IRT model fit the testing data well without examining potential model misspecification. As a result, future research should investigate the performance of the GRDIF framework under more complex IRT models (e.g., multidimensional or polytomous) and assess its robustness in the face of model misspecification. The practicality of the GRDIF framework should be further examined through application to real‐world assessment data, taking into consideration the complexities and challenges that may arise in such settings. For instance, while Lim and Choe ([<reflink idref="bib21" id="ref122">21</reflink>]) demonstrated that <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0136" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>R</mi><mi>D</mi><mi>I</mi><msub><mi>F</mi><mi>R</mi></msub></mrow><annotation encoding="application/x-tex">$RDI{{F}_R}$</annotation></semantics></math> </ephtml> performs efficiently in assessing DIF for pretest items in CAT, no research has yet explored the RDIF framework for analyzing DIF on operational items in CAT. Given that a majority of current DIF detection methods have difficulties handling sparse response data, proposing a suitable procedure for applying GRDIF on both operational and pretest items in real CAT assessments can enhance its versatility.</p> <p>As a closing remark, the GRDIF framework offers a valuable and practical alternative for DIF detection in multiple groups. By addressing the limitations and further exploring the framework's potential, researchers and practitioners will be better equipped to ensure fairness and validity in educational and psychological assessments in diverse contexts.</p> <p>GRAPH</p> <p>GRAPH</p> <p>GRAPH</p> <p>GRAPH</p> <ref id="AN0181921599-28"> <title> Footnotes </title> <blist> <bibl id="bib1" idref="ref9" type="bt">1</bibl> <bibtext> Although some large‐scale assessments, such as the Program for International Student Assessment (PISA), include a multitude of groups, our study deliberately narrows its scope to focus on a limited number of groups, typically those with single‐digit groups numbers, for example, distinct ethnic groups (e.g., White, Black, Hispanic, Asian, European, and others).</bibtext> </blist> <blist> <bibl id="bib2" idref="ref85" type="bt">2</bibl> <bibtext> As explained earlier, the GRDIF statistics remains invariant, irrespective of the specific choice of a contrast matrix as long as any pair of contrast matrices <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0137" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi mathvariant="bold-italic">C</mi><mn>1</mn></msub><annotation encoding="application/x-tex">${{{\bm{C}}}_1}$</annotation></semantics></math> </ephtml> and <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0138" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><msub><mi mathvariant="bold-italic">C</mi><mn>2</mn></msub><annotation encoding="application/x-tex">${{{\bm{C}}}_2}$</annotation></semantics></math> </ephtml> are related by <ephtml> <math display="inline" altimg="urn:x-wiley:00220655:media:jedm12415:jedm12415-math-0139" xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi mathvariant="bold-italic">C</mi><mn>1</mn></msub><mo>=</mo><mi mathvariant="bold-italic">B</mi><msub><mi mathvariant="bold-italic">C</mi><mn>2</mn></msub></mrow><annotation encoding="application/x-tex">${{{\bm{C}}}_1} = {\bm{B}}{{{\bm{C}}}_2}$</annotation></semantics></math> </ephtml> , where <emph>B</emph> is a nonsingular matrix. See Kim et al. (1995, appendix) for more discussion on the contrast matrix.</bibtext> </blist> </ref> <ref id="AN0181921599-29"> <title> References </title> <blist> <bibtext> Asparouhov, T., & Muthén, B. (2014). Multiple‐group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21 (4), 495 – 508.</bibtext> </blist> <blist> <bibtext> Asparouhov, T., & Muthén, B. (2023). Multiple group alignment for exploratory and structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 30 (2), 169 – 191.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref34" type="bt">3</bibl> <bibtext> Bauer, D. J., Belzak, W. C., & Cole, V. T. (2020). Simplifying the assessment of measurement invariance over multiple background variables: Using regularized moderated nonlinear factor analysis to detect differential item functioning. Structural Equation Modeling: a Multidisciplinary Journal, 27 (1), 43 – 55.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref29" type="bt">4</bibl> <bibtext> Belzak, W. C. (2022). The multidimensionality of measurement bias in high‐stakes testing: Using machine learning to evaluate complex sources of differential item functioning. Educational Measurement: Issues and Practice, 42, 24 – 33. https://doi.org/10.1111/emip.12486.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref104" type="bt">5</bibl> <bibtext> Belzak, W. C. (2021). regDIF: Regularized Differential Item Functioning, version 1.0.0 [Computer software]. Retrieved from https://github.com/wbelzak/regDIF.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref35" type="bt">6</bibl> <bibtext> Belzak, W., & Bauer, D. J. (2020). Improving the assessment of measurement invariance: Using regularization to select anchor items and identify differential item functioning. Psychological Methods, 25 (6), 673.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref80" type="bt">7</bibl> <bibtext> Billingsley, P. (1995). Probability and measure (3rd ed.). New York : Wiley.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref102" type="bt">8</bibl> <bibtext> Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443 – 459.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref40" type="bt">9</bibl> <bibtext> DeMars, C. E. (2020). Alignment as an alternative to anchor purification in DIF analyses. Structural Equation Modeling: A Multidisciplinary Journal, 27 (1), 56 – 72.</bibtext> </blist> <blist> <bibtext> Fidalgo, Á. M., & Scalon, J. D. (2010). Using generalized Mantel‐Haenszel statistics to assess DIF among multiple groups. Journal of Psychoeducational Assessment, 28 (1), 60 – 69.</bibtext> </blist> <blist> <bibtext> Finch, W. H. (2016). Detection of differential item functioning for more than two groups: A Monte Carlo comparison of methods. Applied measurement in Education, 29 (1), 30 – 45.</bibtext> </blist> <blist> <bibtext> Finch, W. H., & French, B. F. (2007). Detection of crossing differential item functioning: A comparison of four methods. Educational and Psychological Measurement, 67 (4), 565 – 582.</bibtext> </blist> <blist> <bibtext> Flake, J. K., & McCoach, D. B. (2018). An investigation of the alignment method with polytomous indicators under conditions of partial measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 25 (1), 56 – 70.</bibtext> </blist> <blist> <bibtext> Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA : Sage Publishing.</bibtext> </blist> <blist> <bibtext> Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel–Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129 – 145). Hillsdale, NJ : Erlbaum.</bibtext> </blist> <blist> <bibtext> Huang, P. H. (2018). A penalized likelihood method for multi‐group structural equation modelling. British Journal of Mathematical and Statistical Psychology, 71 (3), 499 – 522.</bibtext> </blist> <blist> <bibtext> Johnson, R. A., & Wichern, D. W. (1998). Applied multivariate statistical analysis (4th ed.). Upper Saddle River, NJ : Prentice‐Hall.</bibtext> </blist> <blist> <bibtext> Kim, S. H., Cohen, A. S., & Park, T. H. (1995). Detection of differential item functioning in multiple groups. Journal of Educational Measurement, 32 (3), 261 – 276.</bibtext> </blist> <blist> <bibtext> Lei, P.‐W., Chen, S.‐Y., & Yu, L. (2006). Comparing methods of assessing differential item functioning in a computerized adaptive testing environment. Journal of Educational Measurement, 43 (3), 245 – 264.</bibtext> </blist> <blist> <bibtext> Lim, H., Choe, E. M., & Han, K. (2022). A residual‐based differential item functioning detection framework in item response theory. Journal of Educational Measurement, 59 (1), 80 – 104.</bibtext> </blist> <blist> <bibtext> Lim, H., & Choe, E. M. (2023). Detecting differential item functioning in CAT using IRT residual DIF approach. Journal of Educational Measurement, 60 (4), 626 – 650</bibtext> </blist> <blist> <bibtext> Lim, H., & Wells, C. S. (2024). irtQ: Unidimensional item response theory modeling, version 0.2.1 [Computer software]. Retrieved from https://CRAN.R‐project.org/package=irtQ.</bibtext> </blist> <blist> <bibtext> Magis, D., Béland, S., Tuerlinckx, F., & De Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42 (3), 847 – 862.</bibtext> </blist> <blist> <bibtext> Magis, D., & De Boeck, P. (2011). Identification of differential item functioning in multiple‐group settings: A multivariate outlier detection approach. Multivariate Behavioral Research, 46 (5), 733 – 755.</bibtext> </blist> <blist> <bibtext> Magis, D., Raîche, G., Béland, S., & Gérard, P. (2011). A generalized logistic regression procedure to detect differential item functioning among multiple groups. International Journal of Testing, 11 (4), 365 – 386.</bibtext> </blist> <blist> <bibtext> Magis, D., Tuerlinckx, F., & De Boeck, P. (2015). Detection of differential item functioning using the lasso approach. Journal of Educational and Behavioral Statistics, 40 (2), 111 – 135.</bibtext> </blist> <blist> <bibtext> McNeish, D. M. (2015). Using lasso for predictor selection and to assuage overfitting: A method long overlooked in behavioral sciences. Multivariate Behavioral Research, 50 (5), 471 – 484.</bibtext> </blist> <blist> <bibtext> Munck, I., Barber, C., & Torney‐Purta, J. (2018). Measurement invariance in comparing attitudes toward immigrants among youth across Europe in 1999 and 2009: The alignment method applied to IEA CIVED and ICCS. Sociological Methods & Research, 47 (4), 687 – 728.</bibtext> </blist> <blist> <bibtext> Muthén, B., & Asparouhov, T. (2014). IRT studies of many groups: The alignment method. Frontiers in Psychology, 5, 978.</bibtext> </blist> <blist> <bibtext> Muthén, L. K., & Muthén, B. O. (2017). Mplus: Statistical analysis with latent variables: User's Guide (Version 8). Los Angeles, CA : Muthén & Muthén.</bibtext> </blist> <blist> <bibtext> Nandakumar, R., & Roussos, L. (2004). Evaluation of the CATSIB DIF procedure in a pretest setting. Journal of Educational and Behavioral Statistics, 29 (2), 177 – 199.</bibtext> </blist> <blist> <bibtext> Penfield, R. D. (2001). Assessing differential item functioning among multiple groups: A comparison of three Mantel‐Haenszel procedures. Applied Measurement in Education, 14 (3), 235 – 259.</bibtext> </blist> <blist> <bibtext> R Core Team. (2024). R: A language and environment for statistical computing. Vienna, Austria : R Foundation for Statistical Computing.</bibtext> </blist> <blist> <bibtext> Schmitt, A. P., & Dorans, N. J. (1990). Differential item functioning for minority examinees on the SAT. Journal of Educational Measurement, 27, 67 – 81.</bibtext> </blist> <blist> <bibtext> Somes, G. W. (1986). The generalized Mantel–Haenszel statistic. The American Statistician, 40 (2), 106 – 108.</bibtext> </blist> <blist> <bibtext> Stark, S., Chernyshenko, O. S., & Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: toward a unified strategy. Journal of Applied Psychology, 91 (6), 1292.</bibtext> </blist> <blist> <bibtext> Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27 (4), 361 – 370.</bibtext> </blist> <blist> <bibtext> Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58 (1), 267 – 288.</bibtext> </blist> <blist> <bibtext> Tutz, G., & Schauberger, G. (2015). A penalty approach to differential item functioning in Rasch models. Psychometrika, 80, 21 – 43.</bibtext> </blist> <blist> <bibtext> Wang, C., Zhu, R., & Xu, G. (2021). Using lasso and adaptive lasso to identify DIF in multidimensional 2PL models. Multivariate Behavioral Research, 1 – 21.</bibtext> </blist> <blist> <bibtext> Woods, C. M., Cai, L., & Wang, M. (2013). The Langer‐improved Wald test for DIF testing with multiple groups: Evaluation and comparison to two‐group IRT. Educational and Psychological Measurement, 73 (3), 532 – 547.</bibtext> </blist> <blist> <bibtext> Zwick, R., & Ercikan, K. (1989). Analysis of differential item functioning in the NAEP history assessment. Journal of Educational Measurement, 26, 55 – 66.</bibtext> </blist> </ref> <aug> <p>By Hwanggyu Lim; Danqi Zhu; Edison M. Choe and KyungT. Han</p> <p>Reported by Author; Author; Author; Author</p> <p></p> <p>Hwanggyu Lim is an associate professor in the College of Education at Inha University, Unit 327, West Lake Building, 100 Inha‐ro, Michuhol‐gu, Incheon 22212, Republic of Korea (South Korea); hglim83@gmail.com. His primary research interests include computerized adaptive testing, differential item functioning, automated test assembly, IRT model fit, IRT calibration, test equating, and the development of psychometric R software packages.</p> <p>Danqi Zhu is a PhD candidate at Fordham University, 441 East Fordham Road Bronx, NY 10458; dzhu17@fordham.edu. Her primary research interests include factor analysis and measurement invariance.</p> <p>Edison M. Choe is a senior psychometrician at Renaissance, 2911 Peach Street, Wisconsin Rapids, WI 54494; edison.choe@renaissance.com. His primary research interests include computerized adaptive testing, psychometric forensics, score integrity and fairness, and classification accuracy and consistency.</p> <p>Kyung (Chris) T. Han is the head of Test Development and Psychometrics (TD&P) at the Graduate Management Admission Council (GMAC), 11921 Freedom Dr. Suite 300, Reston, VA 20190; truetheta@gmail.com. His primary research interests include adaptive testing, DIF, test construction, score estimation, and software development.</p> </aug> <nolink nlid="nl1" bibid="bib14" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib37" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib20" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib21" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib31" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib10" firstref="ref10"></nolink> <nolink nlid="nl7" bibid="bib32" firstref="ref11"></nolink> <nolink nlid="nl8" bibid="bib34" firstref="ref12"></nolink> <nolink nlid="nl9" bibid="bib42" firstref="ref13"></nolink> <nolink nlid="nl10" bibid="bib25" firstref="ref14"></nolink> <nolink nlid="nl11" bibid="bib18" firstref="ref18"></nolink> <nolink nlid="nl12" bibid="bib24" firstref="ref19"></nolink> <nolink nlid="nl13" bibid="bib39" firstref="ref22"></nolink> <nolink nlid="nl14" bibid="bib40" firstref="ref23"></nolink> <nolink nlid="nl15" bibid="bib35" firstref="ref25"></nolink> <nolink nlid="nl16" bibid="bib29" firstref="ref28"></nolink> <nolink nlid="nl17" bibid="bib11" firstref="ref33"></nolink> <nolink nlid="nl18" bibid="bib26" firstref="ref36"></nolink> <nolink nlid="nl19" bibid="bib13" firstref="ref42"></nolink> <nolink nlid="nl20" bibid="bib15" firstref="ref50"></nolink> <nolink nlid="nl21" bibid="bib38" firstref="ref57"></nolink> <nolink nlid="nl22" bibid="bib27" firstref="ref58"></nolink> <nolink nlid="nl23" bibid="bib28" firstref="ref74"></nolink> <nolink nlid="nl24" bibid="bib17" firstref="ref84"></nolink> <nolink nlid="nl25" bibid="bib41" firstref="ref95"></nolink> <nolink nlid="nl26" bibid="bib12" firstref="ref96"></nolink> <nolink nlid="nl27" bibid="bib19" firstref="ref97"></nolink> <nolink nlid="nl28" bibid="bib33" firstref="ref100"></nolink> <nolink nlid="nl29" bibid="bib22" firstref="ref101"></nolink> <nolink nlid="nl30" bibid="bib23" firstref="ref103"></nolink> <nolink nlid="nl31" bibid="bib30" firstref="ref105"></nolink> <nolink nlid="nl32" bibid="bib36" firstref="ref116"></nolink> <nolink nlid="nl33" bibid="bib16" firstref="ref117"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1455024
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Detecting Differential Item Functioning among Multiple Groups Using IRT Residual DIF Framework
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Hwanggyu+Lim%22">Hwanggyu Lim</searchLink><br /><searchLink fieldCode="AR" term="%22Danqi+Zhu%22">Danqi Zhu</searchLink><br /><searchLink fieldCode="AR" term="%22Edison+M%2E+Choe%22">Edison M. Choe</searchLink><br /><searchLink fieldCode="AR" term="%22Kyung+T%2E+Han%22">Kyung T. Han</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Educational+Measurement%22"><i>Journal of Educational Measurement</i></searchLink>. 2024 61(4):656-681.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 26
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Bias%22">Test Bias</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Group+Testing%22">Group Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Error+of+Measurement%22">Error of Measurement</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/jedm.12415
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0022-0655<br />1745-3984
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study presents a generalized version of the residual differential item functioning (RDIF) detection framework in item response theory, named GRDIF, to analyze differential item functioning (DIF) in multiple groups. The GRDIF framework retains the advantages of the original RDIF framework, such as computational efficiency and ease of implementation. The performance of GRDIF was assessed through a simulation study and compared with existing DIF detection methods, including the generalized Mantel-Haenszel, Lasso-DIF, and alignment methods. Results showed that the GRDIF framework demonstrated well-controlled Type I error rates close to the nominal level of .05 and satisfactory power in detecting uniform, nonuniform, and mixed DIF across different simulated conditions. Each of the three GRDIF statistics, GRDIF[subscript R], GRDIF[subscript S], and GRDIF[subscript RS], effectively detected the specific type of DIF for which it was designed, with GRDIF[subscript RS] exhibiting the most robust performance across all types of DIF. The GRDIF framework outperformed other DIF detection methods under various conditions, suggesting its potential for practical applications, particularly in large-scale assessments involving multiple groups. Additionally, an empirical study demonstrated the efficacy and utility of the GRDIF framework in conducting DIF analysis with a high-stakes assessment data set.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1455024
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1455024
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/jedm.12415
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 26
        StartPage: 656
    Subjects:
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Test Bias
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Test Construction
        Type: general
      – SubjectFull: Group Testing
        Type: general
      – SubjectFull: Error of Measurement
        Type: general
    Titles:
      – TitleFull: Detecting Differential Item Functioning among Multiple Groups Using IRT Residual DIF Framework
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Hwanggyu Lim
      – PersonEntity:
          Name:
            NameFull: Danqi Zhu
      – PersonEntity:
          Name:
            NameFull: Edison M. Choe
      – PersonEntity:
          Name:
            NameFull: Kyung T. Han
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0022-0655
            – Type: issn-electronic
              Value: 1745-3984
          Numbering:
            – Type: volume
              Value: 61
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Journal of Educational Measurement
              Type: main
ResultId 1