View in EDS HTML Full Text PDF Full Text

FIPC Linking across Multidimensional Test Forms: Effects of Confounding Difficulty within Dimensions

Saved in:
Bibliographic Details
Title: FIPC Linking across Multidimensional Test Forms: Effects of Confounding Difficulty within Dimensions
Language: English
Authors: Kim, Sohee, Cole, Ki Lynn, Mwavita, Mwarumba
Source: International Journal of Testing. 2018 18(4):323-345.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 23
Publication Date: 2018
Document Type: Journal Articles
Reports - Research
Descriptors: Test Items, Item Response Theory, Test Format, Difficulty Level, Test Construction, Error of Measurement, Testing Problems
DOI: 10.1080/15305058.2018.1428980
ISSN: 1530-5058
Abstract: This study investigated the effects of linking potentially multidimensional test forms using the fixed item parameter calibration. Forms had equal or unequal total test difficulty with and without confounding difficulty. The mean square errors and bias of estimated item and ability parameters were compared across the various confounding tests. The estimated discrimination parameters were influenced by the levels of correlation between dimensions. The mean square errors (MSEs) of the average of the true discrimination parameters with the estimated value were smallest when the correlation equaled 0; however, the MSEs of the multidimensional discrimination parameter were smallest when the correlation was larger than 0. The estimated difficulty parameters were highly affected by different amount of confounding difficulty within dimensions. Furthermore, the MSEs of the average of the true ability parameters on the first and second dimensions with the estimated ability were smaller than those from the ability parameter on each dimension for all conditions. The pattern varied according to the number of common items, and the measures of MSE and squared bias were relatively consistent across forms at the same level of correlation, except for the condition where the correlation was 0 and the number of common items was 8.
Abstractor: As Provided
Number of References: 44
Entry Date: 2018
Accession Number: EJ1198319
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHOZbfFAMDL-8btPwpwC9hyAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDBXXkF7bEAx_o4vjOwIBEICBm55_-QNQq1m2v8y0FNEudA8FoJ2mPfIyNokPEaIxqQGIi07DUsQXHMTAk8l7xixY-3CS-Jn5xEh3y3uLkBZpsZ01FPEfAKKca_53i-R9ByUWX_4an2SyEMdk4oVE6DjNVWvfAhHCaNO8wp5zNA0H0O4GlafDLWrcmswR_CDeVV8p1IyOwDmnIM_iVsLMPviFV7vCGHe2ZGq73znd
Text:
  Availability: 1
  Value: <anid>AN0133291119;k0301oct.18;2018Dec01.14:03;v2.2.500</anid> <title id="AN0133291119-1">FIPC Linking Across Multidimensional Test Forms: Effects of Confounding Difficulty within Dimensions </title> <p>This study investigated the effects of linking potentially multidimensional test forms using the fixed item parameter calibration. Forms had equal or unequal total test difficulty with and without confounding difficulty. The mean square errors and bias of estimated item and ability parameters were compared across the various confounding tests. The estimated discrimination parameters were influenced by the levels of correlation between dimensions. The mean square errors (MSEs) of the average of the true discrimination parameters with the estimated value were smallest when the correlation equaled 0; however, the MSEs of the multidimensional discrimination parameter were smallest when the correlation was larger than 0. The estimated difficulty parameters were highly affected by different amount of confounding difficulty within dimensions. Furthermore, the MSEs of the average of the true ability parameters on the first and second dimensions with the estimated ability were smaller than those from the ability parameter on each dimension for all conditions. The pattern varied according to the number of common items, and the measures of MSE and squared bias were relatively consistent across forms at the same level of correlation, except for the condition where the correlation was 0 and the number of common items was 8.</p> <p>Keywords: item response theory; multidimensional item response theory; linking; test design; fixed item parameter calibrations</p> <hd id="AN0133291119-2">INTRODUCTION</hd> <p>The fixed item parameter calibration (FIPC) method is one item response theory (IRT) linking technique in which item parameters on one form are calibrated, and item parameters of a new form are estimated after fixing the parameters of the subset of items that are common to each form equal to those of the first form. The FIPC method has many favorable characteristics: accurate linking results, invariant item parameter estimates, and comparable item parameters across different administrations and forms (Kim, 2008). Various IRT methods of linking are used on many large-scale tests worldwide, e.g., the Test of English as a Foreign Language (TOEFL; Hickman, Fu, & Hill, 2012) and the Programme for International Student Assessment (PISA; Organization for Economic Co-organization and Development [OECD], 2012). More specifically, the FIPC method is used to link forms of the National Assessment of Educational Achievement (NAEA) in Korea (Kim et al., 2011).</p> <p>One assumption of the FIPC method is that data are unidimensional, and most studies investigate FIPC linking in this context (Hu, Rogers, & Vukmirovic, 2008; Kang & Petersen, 2009; Keller, Keller, & Parker, 2010; Kim & Kang, 2012; Pang, Madera, Radwan, & Zhang, 2010). In practice, however, tests may measure multiple sub-content areas, and examinees may have differing true abilities on each sub-content area. As a result, the datasets may be multidimensional. Under certain conditions, however, unidimensional analyses may be applied to multidimensional data (Ackerman, 1994; Reckase, 1985). Other IRT-based linking methods have been investigated with multidimensional data (see, for example, Andrews, 2011; Béguin & Hanson, 2001; Béguin, Hanson, & Glas, 2000; Bolt, 1999; Camille, Wang, & Fesq, 1995; Chaplain, 1996; Harris & Kolen, 1986; Lee, Lee, & Brennan, 2014; Spence, 1996). The purpose of this study is to extend the studies of FIPC unidimensional linking to potentially multidimensional datasets, which is very likely on an educational test.</p> <p>Tests are often composed of two or more sub-content areas, for example, a mathematics test composed of algebra and of geometry items. When multiple forms are constructed, they may or may not have equal total test difficulty; linking is used to accommodate this difference. The total test difficulty is the primary focus of linking. However, forms may also vary in average item difficulty of the sub-content areas, for instance, if the algebra items are more difficult and geometry items are easier on one form, and, on another form, algebra items are easier and geometry items are more difficult. This confounding of difficulty within dimensions may affect the performance of unidimensional test linking methods such as FIPC, especially when the examinees' true abilities in the two dimensions varies (Ackerman, 1994; Bolt, 1999).</p> <p>When multiple forms of a standardized test are used to assess the ability of individual students, the item and ability parameter estimates across multiple test forms are required to be on the same scale when linking. There are several IRT linking procedures commonly used for this purpose such as separate calibration with linking, concurrent calibration, and FIPC. Previous studies related to FIPC were only in the context of unidimensional data. Earlier studies reported biased estimates of parameters and results were less accurate due to software issues (Baldwin, Baldwin, & Nering, 2007; Keller, Keller, & Baldwin, 2007; Paek & Young, 2005; Skorupski, Jodoin, Keller, & Swaminathan, 2003). The detailed explanation about the problems is discussed in the FIPC section. Kang and Peterson (2009) compared the corrected FIPC procedure using the software PARSCALE with various methods, and reported that the FIPC method could be as good of an item-scale linking tool as the others.</p> <p>This study extends prior investigations of unidimensional linking of potentially multidimensional datasets by investigating the effects of multidimensional data with confounding difficulty within dimensions on unidimensional FIPC linking across nonequivalent groups. Multidimensional item specifications for the ACT Mathematics Usage Test form 24B (Reckase & McKinley, 1991) were used as the initial form. Additional forms had a subset of common items and varied in average item difficulty overall and/or average item difficulty within dimensions. Situations were presented where forms had an equal total test difficulty or varied total test difficulty, and with one form having harder items loading on the first dimension and easier items loading on the second dimension, similar to the base form, and another form having harder items loading on the second dimension and easier items loading on the first dimension, opposite of the base form. The mean square error (MSE), squared bias (SB), and variance (VAR) between the true and estimated item and ability parameters from FIPC linking across forms were measured and compared across the multiple forms at four levels of correlation and three levels of number of common items.</p> <hd id="AN0133291119-3">UNIDIMENSIONAL ITEM RESPONSE THEORY</hd> <p>IRT is used to estimate item difficulty and discrimination and to estimate examinees' underlying latent trait scores based on their item response patterns. One of the assumptions of many IRT models is unidimensionality, which implies that a single ability or the same composite of abilities is needed in order to correctly respond to a set of items (Baker & Kim, 2004). The two parameter logistic (2PL) unidimensional IRT model (Birnbaum, 1968) is given bywhere is the probability of a correct response to item from examinee with ability . Item parameters are the item discrimination, , and the item difficulty, . The marginal maximum likelihood estimation (MMLE) method is one method used for estimating the item parameters, and the expected a posterior (EAP) is a method of estimating the ability parameter. (Other methods exist, but these are the ones used in this study.) The calibration of item and ability parameters occurs in separate stages: item parameters are estimated by assuming the normal distribution of the examinees' abilities, and then ability parameters are obtained by fixing the previously estimated item parameters and applying the EAP procedure.</p> <hd id="AN0133291119-4">MULTIDIMENSIONAL ITEM RESPONSE THEORY</hd> <p>Multidimensional item response theory (MIRT) models are classified into two types: noncompensatory, suggested by Sympson (1978), and compensatory, suggested by Reckase (1985). The noncompensatory model assumes a high ability on one dimension does not compensate for a lower ability on the other dimension. The probability of a correct response is a function of the product of probabilities for each ability dimension. The compensatory model, on the other hand, enables the low level on one dimension to be compensated by high levels on another dimension, and is modeled by summing the probabilities of a correct response across dimensions.</p> <p>The choice between these two MIRT models may depend on the assumed interaction of the characteristics of the test and the abilities across dimensions. The compensatory model has been primarily used in previous studies (Miller, 1991; Zhang & Stout, 1999), due to the difficulty of estimating parameters under the noncompensatory model (Bolt & Lall, 2003) and the similarities in estimated parameters between the two MIRT models with real data (Ackerman, 1989; Bolt & Lall, 2003; Spray, 1990). The compensatory MIRT model used in this study is given as follows (Reckase, 2009):where is the score (0 or 1) for person on item and is the probability of a correct response to item by examinee in a dimensional ability space. The is an vector of person 's ability on the dimensions. The item discrimination parameter, , is an vector of the discrimination parameters for item across dimensions, and is a measure of the multidimensional difficulty of item across dimensions.</p> <p>The multidimensional discrimination () of an item across dimensions can be represented by the following equation:</p> <p>This statistic is an overall measure of capability of an item to distinguish between individuals in -space (Reckase & McKinley, 1991) and is analogous to the unidimensional -parameter. The is a measure of the degree of the item's discrimination on the dimension. The is directly related to the angle between each coordinate axis and the point of the steepest slope. These angles are calculated by following equation:</p> <p>If the angle of the is bigger than the angle of the , the item measures the second dimension more than the first dimension. On the contrary, if the angle of is bigger than the angle of the , the first dimension is measured more by the item than the second dimension.</p> <p>The location of an item is no longer equal to the difficulty of that item, as was the case in the unidimensional model. Under the multidimensional model, a measure of the multidimensional location of an item for the most discrimination combination of dimensions is , and the equation is:</p> <p>This term is the location on the ability scale where the item response surface (IRS) has the maximum slope and is analogous to the unidimensional parameter, with the same directional interpretation, where a positive value indicates a more difficult item, and a negative value indicates an easier item. The gives the difficulty of the item related to the corresponding coordinate dimension (Reckase, 2009).</p> <hd id="AN0133291119-5">FIXED ITEM PARAMETER CALIBRATION</hd> <p>In IRT linking, dealing with the scale indeterminacy property is critical and there are different calibrations in order to solve this problem: calibration with linking coefficients, and calibration without linking coefficients (Kang & Peterson, 2009; Kim, 2006; Kim & Kang, 2012; Kim & Kolen, 2006). For calibration with linking coefficients, item and ability parameters are calculated for each test form, and then a linear transformation is conducted by using the linking coefficients in order to place item and ability parameter estimates on a common scale. Some methods include mean-sigma, mean-mean, Haebara, and Stocking-Lord methods. In contrast, calibration without linking coefficients is a way to use the common item or examinees, allowing newly calibrated item and ability parameters to be estimated on the common item scale. FIPC and concurrent calibration are some linking methods without linking coefficients.</p> <p>The goal of test linking is to put all corresponding IRT parameter estimates on a common scale in order to obtain comparable and interchangeable results when different test forms are administered and data are generated using the common-item nonequivalent group design. Various test linking procedures are applied to large-scale tests that utilize multiple forms, for example, TOEFL (Wainer & Wang, 2001), SAT (Samuel, 2004), ACT (ACT, 2014) and National Assessment of Education Progress (NAEP; Allen, Donoghue, & Schoeps, 2001). The goal of FIPC is to put the newly estimated item parameters on the scale of the common items that were "fixed" during calibration. FIPC is a two-step, calibration and linking method (Pang et al., 2010). First, the item parameters of the reference test are calibrated. Next, the item parameters of common items across the reference and secondary forms are fixed to match those of the reference test. Then, the non-common item parameters of the secondary test form are calibrated. This method is one of the IRT-based linking without linking coefficients. Instead, the item parameters of new forms are estimated on a common item scale by fixing the common item parameters. Some previous studies reported that FIPC tends to yield less accurate parameter estimates than other methods including mean-sigma, mean-mean, Haebara, and Stocking-Lord methods including those with calibrations with linking coefficients (e.g., mean-sigma, mean-mean, Haebara, and Stocking-Lord methods). Unfortunately, these studies were based on software calibration problems (Baldwin et al., 2007; Keller, Keller & Baldwin, 2007; Skorupski et al., 2003). However, the accuracy of FIPC results was favorably supported by other studies (Kang & Petersen, 2009; Kim, 2006; Kim & Kang, 2012; Paek & Young, 2005) with the corrected FIPC procedures. Kang and Peterson (2009) provided the table for the correct options in PARSCALE in order to use the FIPC procedure. These corrected studies used options in PARSCALE by preventing any rescaling and using multiple EM cycles for estimating parameters, whereas the problematic studies did not use the proper options. When the FIPC procedure uses the marginal maximum likelihood estimation (MMLE) method with the expectation maximization (EM) algorithm approach, there are two essential elements: (<reflink idref="bib1" id="ref1">1</reflink>) no rescaling of the posterior ability distribution and (<reflink idref="bib2" id="ref2">2</reflink>) updating the prior ability distribution. The statistical software PARSCALE can be used to implement FIPC in the proper manner using the NOADJUST command for preventing rescaling and the POSTERIOR command for updating of the prior ability distribution are used (Kang & Peterson, 2009; Kim & Kang, 2012).</p> <hd id="AN0133291119-6">METHOD</hd> <p>A simulation study was used in order to compare the effect of different test forms, levels of correlations, and numbers of common items on FIPC linking under the nonequivalent, anchor test (NEAT) design. The FIPC linking method was used to link three test forms with varied item difficulties to the reference form. With this method, all the item parameters across all forms were on a common item scale, so that the estimated trait scores were also on the same scale and could be compared across the varied conditions.</p> <hd id="AN0133291119-7">Sample</hd> <p>A NEAT design was chosen to replicate the situation where forms are administered to different groups which may differ in true ability distributions (Kim et al., 2011). Group 1 was assumed to have a two-dimensional, standard normal distribution of abilities. The abilities of group 2 followed a two-dimensional normal distribution with a . The ability distributions were simulated for 1,000 examinees in each group at different levels of correlation ( and the number of common items (<reflink idref="bib8" id="ref3">8</reflink>, 16, and 32) on 40-item tests (see Figure 1).</p> <hd id="AN0133291119-8">Item Specifications</hd> <p>Following the NEAT design, four test forms of 40 items were constructed. The two-dimensional item specifications of the first test form were taken from the ACT Mathematics Usage Test form 24B (Reckase & McKinley, 1991). This test consisted of 40 items and had a total test difficulty of , calculated as the average of individual item difficulties. This negative value indicated a somewhat difficult test. Items were grouped into subsets of items that primarily measured one of the two dimensions or that measured each dimension somewhat equally based on the parameter. Twenty items were grouped in set 1; these primarily measured the first dimension and were easy (. Eleven items were in set 2, measured both dimensions somewhat equally, and were difficult (. The remaining nine items were in set 3, primarily measured the second dimension, and were very difficult (. Table 1 displays the average true item parameters within subsets of items measuring the same combination of dimensions for each condition of the number of common items.</p> <p>Item Specifications (Mean and Standard Deviation) for Generated Dataset for Each Form and Subset of Items</p> <p> <ephtml> <table border="1" cellpadding="9"><tbody><tr><td align="center" colspan="2"><italic>n</italic></td><td align="center" colspan="2"><italic>a</italic><sub>1</sub></td><td align="center" colspan="2"><italic>a</italic><sub>2</sub></td><td align="center" colspan="2"><italic>d</italic></td></tr><tr><td align="left">Form 1: </td><td align="center">40</td><td align="char">1.041</td><td align="char">(0.536)</td><td align="char">0.710</td><td align="char">(0.560)</td><td align="char">−0.324</td><td align="char">(0.770)</td></tr></tbody><tbody><tr><td align="left"> Set 1</td><td align="center">20</td><td align="char">1.352</td><td align="char">(0.443)</td><td align="char">0.281</td><td align="char">(0.267)</td><td align="char">0.164</td><td align="char">(0.567)</td></tr><tr><td align="left"> Set 2</td><td align="center">11</td><td align="char">1.007</td><td align="char">(0.376)</td><td align="char">1.050</td><td align="char">(0.469)</td><td align="char">−0.430</td><td align="char">(0.526)</td></tr><tr><td align="left"> Set 3</td><td align="center">9</td><td align="char">0.390</td><td align="char">(0.192)</td><td align="char">1.247</td><td align="char">(0.374)</td><td align="char">−1.277</td><td align="char">(0.389)</td></tr><tr><td align="left">Form 2: </td><td align="center">40</td><td align="char">1.041</td><td align="char">(0.536)</td><td align="char">0.710</td><td align="char">(0.560)</td><td align="char">−0.224</td><td align="char">(0.791)</td></tr><tr><td align="left"> Set 1</td><td align="center">20</td><td align="char">1.352</td><td align="char">(0.443)</td><td align="char">0.281</td><td align="char">(0.267)</td><td align="char">0.264</td><td align="char">(0.725)</td></tr><tr><td align="left"> Set 2</td><td align="center">11</td><td align="char">1.007</td><td align="char">(0.376)</td><td align="char">1.050</td><td align="char">(0.469)</td><td align="char">−0.430</td><td align="char">(0.526)</td></tr><tr><td align="left"> Set 3</td><td align="center">9</td><td align="char">0.390</td><td align="char">(0.192)</td><td align="char">1.247</td><td align="char">(0.374)</td><td align="char">−1.053</td><td align="char">(0.161)</td></tr><tr><td align="left">Form 3: </td><td align="center">40</td><td align="char">1.041</td><td align="char">(0.536)</td><td align="char">0.710</td><td align="char">(0.560)</td><td align="char">−0.324</td><td align="char">(0.770)</td></tr><tr><td align="left"> Set 1</td><td align="center">18</td><td align="char">1.352</td><td align="char">(0.443)</td><td align="char">0.281</td><td align="char">(0.267)</td><td align="char">−0.356</td><td align="char">(0.775)</td></tr><tr><td align="left"> Set 2</td><td align="center">11</td><td align="char">1.007</td><td align="char">(0.376)</td><td align="char">1.050</td><td align="char">(0.469)</td><td align="char">−0.430</td><td align="char">(0.526)</td></tr><tr><td align="left"> Set 3</td><td align="center">11</td><td align="char">0.390</td><td align="char">(0.192)</td><td align="char">1.247</td><td align="char">(0.374)</td><td align="char">−0.122</td><td align="char">(1.028)</td></tr><tr><td align="left">Form 4: </td><td align="center">40</td><td align="char">1.041</td><td align="char">(0.536)</td><td align="char">0.710</td><td align="char">(0.560)</td><td align="char">−0.224</td><td align="char">(0.791)</td></tr><tr><td align="left"> Set 1</td><td align="center">18</td><td align="char">1.352</td><td align="char">(0.443)</td><td align="char">0.281</td><td align="char">(0.267)</td><td align="char">−0.256</td><td align="char">(0.604)</td></tr><tr><td align="left"> Set 2</td><td align="center">11</td><td align="char">1.007</td><td align="char">(0.376)</td><td align="char">1.050</td><td align="char">(0.469)</td><td align="char">−0.430</td><td align="char">(0.526)</td></tr><tr><td align="left"> Set 3</td><td align="center">11</td><td align="char">0.390</td><td align="char">(0.192)</td><td align="char">1.247</td><td align="char">(0.374)</td><td align="char">0.100</td><td align="char">(1.286)</td></tr></tbody></table> </ephtml> </p> <p>Across the four forms, the item difficulty of a select number of non-common items was manipulated for forms 2, 3, and 4. This design was replicated across three conditions of common item proportions: 8 (20%), 16 (40%), or 32 (80%); this followed the suggestion of Kang and Peterson (2009). A pre-specified number of items within the common set of the original specifications were chosen randomly to be kept as common, in order to maintain the same proportion of items within sets. On form 2, the difficulty of the non-common items was harder than those of the form 1 by increasing by 0.5. Thus, the total test difficulty of form 2 was slightly harder than form 1, but the pattern of confounding difficulty within dimensions was the same. The first set of items was easier than the third set on both forms.</p> <p>The total test difficulty of form 3 was the same as form 1, but the average difficulty of the first and third sets of items was changed by switching the values of the selected non-common items in the first set and third set. Thus, the average difficulty of the first set of items was harder than those of the third set of items, which was the opposite of average difficulty within sets of items on the first form. The total difficulty of form 4 was increased (similar to form 2), and the difficulty values of the non-common items within each set were switched. In summary, form 2 did not have confounding difficulty, but the total test difficulty differed; form 3 had the same overall difficulty as the reference form but difficulty was confounded within dimensions. Form 4 had a different total test difficulty than the reference form and had confounding difficulty within dimensions.</p> <hd id="AN0133291119-9">Analysis and Instruments</hd> <p>Under the NEAT design, item responses were generated for group 1 taking form 1 and item responses were generated for group 2 taking forms 2, 3, and 4 for all conditions of numbers of common items using the "mirt" package (Chalmers, 2012) in R version 3.3.1 (R Core Team, 2016) according to the 2PL MIRT model. Item parameters were estimated using the FIPC method to link forms 2, 3, and 4 to form 1 using PARSCALE (Muraki & Bock, 2003) with the 2PL unidimensional IRT model. Conditions were replicated 500 times.</p> <hd id="AN0133291119-10">Evaluation</hd> <p>The estimated linked item and ability parameters and their standard errors were averaged across replications under each condition of form, number of common items, and correlation. The average mean square error (MSE), squared bias (SB), and variance (VAR) of the estimated item and ability parameters were compared across various conditions, similar to Kang and Petersen (2009).</p> <p>The MSE was the criteria to assess the accuracy of estimations, measuring the difference between the true and estimated parameters. Let be the true parameter of an item on form 1—true item discrimination on each dimension (true multidimensional discrimination (, equation 3average of the true item discriminations on each dimension (true multidimensional difficulty (, negative due to the opposite meaning of the multidimensional and unidimensional difficulty parameters)and true multidimensional location , equation 5), and be the corresponding estimated parameter of item at replication on form 2, 3, or 4 (). The MSE criterion in Equation 6 was commonly used as a criterion in many simulation studies (Béguin & Hanson, 2001; Béguin, Hanson, & Glass, 2000; Hanson & Béguin, 1999; Hu et al., 2008; Kang & Petersen, 2009; Lee et al., 2014).</p> <p>The MSE can be partitioned into the sum of SB and VAR. The SB and VAR are measures of the systematic and random error, respectively. The relationship among these criteria is present in Equation 7:</p> <p>The MSE value for the ability parameters can be calculated in a similar way as described above, and the equations are as follows:</p> <hd id="AN0133291119-11">RESULTS</hd> <p>Item and ability parameters on form 2, 3, and 4 were linked to those on form 1 using the FIPC methods. Datasets used in this study had one of four levels of correlation (0, 0.3, 0.6, or 0.9) and one of three levels of common items proportions (20%, 40%, or 80%). The results were evaluated by measures of MSE.</p> <hd id="AN0133291119-12">Item Parameter</hd> <p>Tables 2, 3, and 4 reported the measures of MSE, SB, and VAR of the item parameters under all conditions of forms, levels of correlation, and the number of common items. These were reported as the average of the evaluation measures across all items or subsets of items within each condition.</p> <p>Average MSE, SB, and VAR of the Estimated Unidimensional Item Parameters with the True Multidimensional Parameter for All 40 Items (The number of common items: 8 [20%])</p> <p> <ephtml> <table border="1" cellpadding="9"><tbody><tr><td align="left" /><td align="center" /><td align="center" /><td align="center" colspan="4">Discrimination</td><td align="center" colspan="2">Difficulty</td></tr><tr><td align="left" /><td align="left" /><td align="left" /><td align="center" colspan="4">Estimated <inline-graphic href="" /> with true multidimensional values:</td><td align="center" colspan="2">Estimated <inline-graphic href="" /> with true multidimensional values:</td></tr><tr><td align="center">Form</td><td align="center"><inline-graphic href="" /></td><td align="center">Criteria</td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td></tr></tbody><tbody><tr><td align="left">2</td><td align="center">0</td><td align="left">MSE</td><td align="char">0.129</td><td align="char">0.542</td><td align="char">0.194</td><td align="char">0.112</td><td align="char">0.249</td><td align="char">0.109</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.120</td><td align="char">0.534</td><td align="char">0.185</td><td align="char">0.103</td><td align="char">0.247</td><td align="char">0.106</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.009</td><td align="char">0.009</td><td align="char">0.009</td><td align="char">0.009</td><td align="char">0.003</td><td align="char">0.003</td></tr><tr><td align="left" /><td align="char">.3</td><td align="left">MSE</td><td align="char">0.270</td><td align="char">0.672</td><td align="char">0.112</td><td align="char">0.247</td><td align="char">0.299</td><td align="char">0.150</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.259</td><td align="char">0.661</td><td align="char">0.101</td><td align="char">0.236</td><td align="char">0.297</td><td align="char">0.148</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.011</td><td align="char">0.011</td><td align="char">0.011</td><td align="char">0.011</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.6</td><td align="left">MSE</td><td align="char">0.356</td><td align="char">0.780</td><td align="char">0.086</td><td align="char">0.345</td><td align="char">0.314</td><td align="char">0.167</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.342</td><td align="char">0.767</td><td align="char">0.073</td><td align="char">0.331</td><td align="char">0.312</td><td align="char">0.165</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.014</td><td align="char">0.014</td><td align="char">0.014</td><td align="char">0.014</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.9</td><td align="left">MSE</td><td align="char">0.594</td><td align="char">1.090</td><td align="char">0.118</td><td align="char">0.619</td><td align="char">0.319</td><td align="char">0.169</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.577</td><td align="char">1.073</td><td align="char">0.101</td><td align="char">0.601</td><td align="char">0.317</td><td align="char">0.167</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.017</td><td align="char">0.017</td><td align="char">0.017</td><td align="char">0.017</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left">3</td><td align="center">0</td><td align="left">MSE</td><td align="char">0.110</td><td align="char">0.548</td><td align="char">0.203</td><td align="char">0.105</td><td align="char">0.894</td><td align="char">0.577</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.102</td><td align="char">0.540</td><td align="char">0.195</td><td align="char">0.097</td><td align="char">0.891</td><td align="char">0.574</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.003</td><td align="char">0.003</td></tr><tr><td align="left" /><td align="char">.3</td><td align="left">MSE</td><td align="char">0.249</td><td align="char">0.690</td><td align="char">0.113</td><td align="char">0.246</td><td align="char">0.790</td><td align="char">0.502</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.239</td><td align="char">0.680</td><td align="char">0.102</td><td align="char">0.236</td><td align="char">0.787</td><td align="char">0.500</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.6</td><td align="left">MSE</td><td align="char">0.341</td><td align="char">0.764</td><td align="char">0.081</td><td align="char">0.329</td><td align="char">0.779</td><td align="char">0.498</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.328</td><td align="char">0.751</td><td align="char">0.068</td><td align="char">0.316</td><td align="char">0.777</td><td align="char">0.495</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.013</td><td align="char">0.013</td><td align="char">0.013</td><td align="char">0.013</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.9</td><td align="left">MSE</td><td align="char">0.576</td><td align="char">1.080</td><td align="char">0.114</td><td align="char">0.604</td><td align="char">0.742</td><td align="char">0.469</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.559</td><td align="char">1.064</td><td align="char">0.095</td><td align="char">0.588</td><td align="char">0.740</td><td align="char">0.467</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left">4</td><td align="center">0</td><td align="left">MSE</td><td align="char">0.107</td><td align="char">0.561</td><td align="char">0.203</td><td align="char">0.111</td><td align="char">1.007</td><td align="char">0.683</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.098</td><td align="char">0.553</td><td align="char">0.195</td><td align="char">0.102</td><td align="char">1.003</td><td align="char">0.679</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.004</td><td align="char">0.004</td></tr><tr><td align="left" /><td align="char">.3</td><td align="left">MSE</td><td align="char">0.245</td><td align="char">0.695</td><td align="char">0.112</td><td align="char">0.246</td><td align="char">0.856</td><td align="char">0.566</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.235</td><td align="char">0.685</td><td align="char">0.103</td><td align="char">0.236</td><td align="char">0.853</td><td align="char">0.564</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.6</td><td align="left">MSE</td><td align="char">0.341</td><td align="char">0.771</td><td align="char">0.080</td><td align="char">0.333</td><td align="char">0.836</td><td align="char">0.555</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.329</td><td align="char">0.759</td><td align="char">0.069</td><td align="char">0.320</td><td align="char">0.833</td><td align="char">0.552</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.9</td><td align="left">MSE</td><td align="char">0.575</td><td align="char">1.081</td><td align="char">0.112</td><td align="char">0.604</td><td align="char">0.792</td><td align="char">0.519</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.558</td><td align="char">1.064</td><td align="char">0.097</td><td align="char">0.588</td><td align="char">0.790</td><td align="char">0.517</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.002</td><td align="char">0.002</td></tr></tbody></table> </ephtml> MSE of with is reported since and have the opposite directional meanings. <emph>Note.</emph> MSE = Mean Square Error, SB = Squared Bias, VAR = Variance.</p> <p>Average MSE, SB, and VAR of the Estimated Unidimensional Item Parameters with the True Multidimensional Parameter for All 40 Items (The number of common items: 16 [40%])</p> <p> <ephtml> <table border="1" cellpadding="9"><tbody><tr><td align="left" /><td align="center" /><td align="center" /><td align="center" colspan="4">Discrimination</td><td align="center" colspan="2">Difficulty</td></tr><tr><td align="left" /><td align="left" /><td align="left" /><td align="center" colspan="4">Estimated <inline-graphic href="" /> with true multidimensional values:</td><td align="center" colspan="2">Estimated <inline-graphic href="" /> with true multidimensional values:</td></tr><tr><td align="center">Form</td><td align="center"><inline-graphic href="" /></td><td align="center">Criteria</td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td></tr></tbody><tbody><tr><td align="left">2</td><td align="center">0</td><td align="left">MSE</td><td align="char">0.131</td><td align="char">0.537</td><td align="char">0.189</td><td align="char">0.110</td><td align="char">0.242</td><td align="char">0.104</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.123</td><td align="char">0.529</td><td align="char">0.181</td><td align="char">0.102</td><td align="char">0.239</td><td align="char">0.101</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.003</td><td align="char">0.003</td></tr><tr><td align="left" /><td align="char">.3</td><td align="left">MSE</td><td align="char">0.269</td><td align="char">0.689</td><td align="char">0.107</td><td align="char">0.256</td><td align="char">0.299</td><td align="char">0.150</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.258</td><td align="char">0.679</td><td align="char">0.096</td><td align="char">0.245</td><td align="char">0.297</td><td align="char">0.148</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.6</td><td align="left">MSE</td><td align="char">0.377</td><td align="char">0.801</td><td align="char">0.087</td><td align="char">0.366</td><td align="char">0.315</td><td align="char">0.167</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.364</td><td align="char">0.788</td><td align="char">0.074</td><td align="char">0.353</td><td align="char">0.313</td><td align="char">0.165</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.013</td><td align="char">0.013</td><td align="char">0.013</td><td align="char">0.013</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.9</td><td align="left">MSE</td><td align="char">0.620</td><td align="char">1.136</td><td align="char">0.126</td><td align="char">0.654</td><td align="char">0.319</td><td align="char">0.169</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.603</td><td align="char">1.120</td><td align="char">0.110</td><td align="char">0.638</td><td align="char">0.317</td><td align="char">0.167</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.017</td><td align="char">0.017</td><td align="char">0.017</td><td align="char">0.017</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left">3</td><td align="center">0</td><td align="left">MSE</td><td align="char">0.117</td><td align="char">0.535</td><td align="char">0.198</td><td align="char">0.102</td><td align="char">0.879</td><td align="char">0.566</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.110</td><td align="char">0.528</td><td align="char">0.190</td><td align="char">0.095</td><td align="char">0.876</td><td align="char">0.563</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.003</td><td align="char">0.003</td></tr><tr><td align="left" /><td align="char">.3</td><td align="left">MSE</td><td align="char">0.254</td><td align="char">0.707</td><td align="char">0.108</td><td align="char">0.257</td><td align="char">0.784</td><td align="char">0.498</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.244</td><td align="char">0.697</td><td align="char">0.098</td><td align="char">0.247</td><td align="char">0.782</td><td align="char">0.496</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.6</td><td align="left">MSE</td><td align="char">0.366</td><td align="char">0.795</td><td align="char">0.082</td><td align="char">0.357</td><td align="char">0.773</td><td align="char">0.493</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.354</td><td align="char">0.783</td><td align="char">0.069</td><td align="char">0.345</td><td align="char">0.770</td><td align="char">0.490</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.9</td><td align="left">MSE</td><td align="char">0.611</td><td align="char">1.135</td><td align="char">0.126</td><td align="char">0.649</td><td align="char">0.737</td><td align="char">0.465</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.593</td><td align="char">1.117</td><td align="char">0.107</td><td align="char">0.631</td><td align="char">0.735</td><td align="char">0.463</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.018</td><td align="char">0.018</td><td align="char">0.018</td><td align="char">0.018</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left">4</td><td align="center">0</td><td align="left">MSE</td><td align="char">0.115</td><td align="char">0.546</td><td align="char">0.197</td><td align="char">0.107</td><td align="char">0.991</td><td align="char">0.671</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.108</td><td align="char">0.538</td><td align="char">0.190</td><td align="char">0.100</td><td align="char">0.988</td><td align="char">0.668</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.003</td><td align="char">0.003</td></tr><tr><td align="left" /><td align="char">.3</td><td align="left">MSE</td><td align="char">0.254</td><td align="char">0.714</td><td align="char">0.108</td><td align="char">0.260</td><td align="char">0.849</td><td align="char">0.561</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.244</td><td align="char">0.704</td><td align="char">0.098</td><td align="char">0.250</td><td align="char">0.846</td><td align="char">0.558</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.003</td><td align="char">0.003</td></tr><tr><td align="left" /><td align="char">.6</td><td align="left">MSE</td><td align="char">0.367</td><td align="char">0.797</td><td align="char">0.081</td><td align="char">0.358</td><td align="char">0.828</td><td align="char">0.548</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.355</td><td align="char">0.785</td><td align="char">0.070</td><td align="char">0.346</td><td align="char">0.826</td><td align="char">0.546</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.9</td><td align="left">MSE</td><td align="char">0.608</td><td align="char">1.133</td><td align="char">0.123</td><td align="char">0.647</td><td align="char">0.785</td><td align="char">0.512</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.592</td><td align="char">1.117</td><td align="char">0.108</td><td align="char">0.631</td><td align="char">0.783</td><td align="char">0.510</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.002</td><td align="char">0.002</td></tr></tbody></table> </ephtml> *MSE of with is reported since and have the opposite directional meanings. <emph>Note.</emph> MSE = Mean Square Error, SB = Squared Bias, VAR = Variance</p> <p>Average MSE, SB, and VAR of the Estimated Unidimensional Item Parameters with the True Multidimensional Parameter for All 40 Items (The number of common items: 32 [80%])</p> <p> <ephtml> <table border="1" cellpadding="9"><tbody><tr><td align="left" /><td align="left" /><td align="left" /><td align="center" colspan="4">Discrimination</td><td align="center" colspan="2">Difficulty</td></tr><tr><td align="left" /><td align="left" /><td align="left" /><td align="center" colspan="4">Estimated <inline-graphic href="" /> with true multidimensional values:</td><td align="center" colspan="2">Estimated <inline-graphic href="" /> with true multidimensional values:</td></tr><tr><td align="left">Form</td><td align="center"><inline-graphic href="" /></td><td align="center">Criteria</td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td><td align="center"><inline-graphic href="" /></td></tr></tbody><tbody><tr><td align="left">2</td><td align="center">0</td><td align="left">MSE</td><td align="char">0.127</td><td align="char">0.533</td><td align="char">0.188</td><td align="char">0.106</td><td align="char">0.226</td><td align="char">0.095</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.120</td><td align="char">0.525</td><td align="char">0.180</td><td align="char">0.099</td><td align="char">0.223</td><td align="char">0.093</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.008</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.3</td><td align="left">MSE</td><td align="char">0.252</td><td align="char">0.703</td><td align="char">0.098</td><td align="char">0.254</td><td align="char">0.293</td><td align="char">0.147</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.241</td><td align="char">0.693</td><td align="char">0.088</td><td align="char">0.244</td><td align="char">0.291</td><td align="char">0.145</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.6</td><td align="left">MSE</td><td align="char">0.385</td><td align="char">0.802</td><td align="char">0.086</td><td align="char">0.370</td><td align="char">0.311</td><td align="char">0.165</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.373</td><td align="char">0.790</td><td align="char">0.074</td><td align="char">0.358</td><td align="char">0.310</td><td align="char">0.163</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.9</td><td align="left">MSE</td><td align="char">0.628</td><td align="char">1.160</td><td align="char">0.129</td><td align="char">0.671</td><td align="char">0.319</td><td align="char">0.169</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.612</td><td align="char">1.144</td><td align="char">0.113</td><td align="char">0.655</td><td align="char">0.317</td><td align="char">0.167</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.016</td><td align="char">0.001</td><td align="char">0.001</td></tr><tr><td align="left">3</td><td align="center">0</td><td align="left">MSE</td><td align="char">0.125</td><td align="char">0.522</td><td align="char">0.195</td><td align="char">0.100</td><td align="char">0.861</td><td align="char">0.555</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.118</td><td align="char">0.515</td><td align="char">0.189</td><td align="char">0.093</td><td align="char">0.858</td><td align="char">0.552</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.003</td><td align="char">0.003</td></tr><tr><td align="left" /><td align="char">.3</td><td align="left">MSE</td><td align="char">0.250</td><td align="char">0.715</td><td align="char">0.102</td><td align="char">0.259</td><td align="char">0.776</td><td align="char">0.493</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.241</td><td align="char">0.705</td><td align="char">0.093</td><td align="char">0.249</td><td align="char">0.774</td><td align="char">0.490</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.6</td><td align="left">MSE</td><td align="char">0.385</td><td align="char">0.801</td><td align="char">0.085</td><td align="char">0.369</td><td align="char">0.767</td><td align="char">0.489</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.373</td><td align="char">0.789</td><td align="char">0.074</td><td align="char">0.358</td><td align="char">0.766</td><td align="char">0.488</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.9</td><td align="left">MSE</td><td align="char">0.624</td><td align="char">1.165</td><td align="char">0.130</td><td align="char">0.671</td><td align="char">0.733</td><td align="char">0.463</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.608</td><td align="char">1.150</td><td align="char">0.115</td><td align="char">0.656</td><td align="char">0.732</td><td align="char">0.461</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.015</td><td align="char">0.015</td><td align="char">0.015</td><td align="char">0.015</td><td align="char">0.001</td><td align="char">0.001</td></tr><tr><td align="left">4</td><td align="center">0</td><td align="left">MSE</td><td align="char">0.125</td><td align="char">0.530</td><td align="char">0.195</td><td align="char">0.104</td><td align="char">0.971</td><td align="char">0.659</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.118</td><td align="char">0.523</td><td align="char">0.188</td><td align="char">0.097</td><td align="char">0.967</td><td align="char">0.656</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.007</td><td align="char">0.003</td><td align="char">0.003</td></tr><tr><td align="left" /><td align="char">.3</td><td align="left">MSE</td><td align="char">0.252</td><td align="char">0.719</td><td align="char">0.103</td><td align="char">0.262</td><td align="char">0.840</td><td align="char">0.555</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.242</td><td align="char">0.709</td><td align="char">0.093</td><td align="char">0.252</td><td align="char">0.837</td><td align="char">0.552</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.010</td><td align="char">0.003</td><td align="char">0.003</td></tr><tr><td align="left" /><td align="char">.6</td><td align="left">MSE</td><td align="char">0.388</td><td align="char">0.806</td><td align="char">0.086</td><td align="char">0.374</td><td align="char">0.823</td><td align="char">0.545</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.376</td><td align="char">0.795</td><td align="char">0.074</td><td align="char">0.362</td><td align="char">0.821</td><td align="char">0.543</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.012</td><td align="char">0.002</td><td align="char">0.002</td></tr><tr><td align="left" /><td align="char">.9</td><td align="left">MSE</td><td align="char">0.626</td><td align="char">1.167</td><td align="char">0.131</td><td align="char">0.673</td><td align="char">0.781</td><td align="char">0.510</td></tr><tr><td align="left" /><td align="left" /><td align="left">SB</td><td align="char">0.611</td><td align="char">1.152</td><td align="char">0.116</td><td align="char">0.658</td><td align="char">0.779</td><td align="char">0.508</td></tr><tr><td align="left" /><td align="left" /><td align="left">VAR</td><td align="char">0.015</td><td align="char">0.015</td><td align="char">0.015</td><td align="char">0.015</td><td align="char">0.002</td><td align="char">0.002</td></tr></tbody></table> </ephtml> *MSE of with is reported since and have the opposite directional meanings. <emph>Note.</emph> MSE = Mean Square Error, SB = Squared Bias, VAR = Variance</p> <hd id="AN0133291119-13">Discrimination</hd> <p>Overall, the average MSE of the item discrimination was smallest between the average of the true discrimination () values and the estimated unidimensional discrimination when , and smallest between true and the estimated discrimination when , more specifically . This trend was consistent for all forms and numbers of common items. This was clearly seen in Figure 2. The estimated discrimination tended to be a closer true than true for all forms, levels of correlation and the number of common items. This might be due to the larger number of items primarily measuring the first dimension; however, it should be noted that changes in item difficulty and correlation between dimensions did not alleviate the effect of the number of items within sets.</p> <p>PHOTO (COLOR): FIGURE 1 Neat Study Design, with and.</p> <p>PHOTO (COLOR): FIGURE 2 Average MSE of the estimated item discrimination parameter.</p> <p>As the level of correlation increased the MSE of the estimated discrimination with true and tended to increase on all forms and the all number of common items. However, the tendency for the MSE of the decreased as correlation increased from to ; however, the MSE increased as MDISC decreased when . In addition, the differences were not too big for all measures of discrimination (and) across test forms and the number of common items. This consistent trend across all forms indicated that the correlation had an effect on the estimates of the discrimination parameters, but that differences in difficulty of items within subsets primarily measuring one of the two dimensions had little effect on the accuracy of the estimated unidimensional discrimination.</p> <p>As the number of common items increased, the MSE of the estimated unidimensional discrimination with and tended to decrease in small quantities when . However, the MSE of the estimated discrimination increased as the number of common items increased when the correlation between equaled to 0.6 and 0.9. Moreover the patterns of MSE for all different forms, levels of correlation, and the number of common items in the estimated discrimination parameters were comparable in Figure 2 and the clear patterns in terms of correlation levels were found.</p> <hd id="AN0133291119-14">Difficulty</hd> <p>In the case of the difficulty parameter, the MSE was small between the estimated difficulty and the true than with true for all forms and the number of common items at all levels of correlation. The trend was displayed in Figure 2. Note, since the estimated unidimensional difficulty, , has the opposite interpretation as the true multidimensional difficulty, , the true value of was compared to the estimated difficulty.</p> <p>The confounding difficulty within dimensions on the multiple test forms had a strong effect on the estimated item difficulty. On form 2, which had an increase in overall test difficulty but did not have confounded difficulty within dimensions, the MSE of with and tended to increase slightly as correlation increased for datasets with a fixed number of common items. However, on forms 3 and 4, which did have confounding of difficulty within dimensions, the MSE of the difficulty parameters tended to decrease as correlation increased. Furthermore, the MSE of difficulty parameters tended to be much smaller on form 2 than on forms 3 and 4. This indicated that the correlation across dimensions and the number of common items had a slight effect on the accuracy of the difficulty estimations, but the confounding of difficulty within dimensions had a larger effect on the estimated difficulty.</p> <p>As the number of common items increased, the MSE of the estimated difficulty with and tended to decrease across all test forms and levels of correlation, although the drop was slight. Furthermore the patterns of MSE values for all different forms and levels correlation were similar across the number of common items as seen in Figure 3.</p> <p>PHOTO (COLOR): FIGURE 3 Average MSE of the estimated item difficulty parameter.</p> <hd id="AN0133291119-15">Systematic Error</hd> <p>The tendency of the systematic error, measured by the SB, of the item parameter estimates followed very similar trends as the results of MSE. Under all conditions, the systematic error accounted for at least 97% of the total error of the difficulty parameter and at least 83% of the total error of the discrimination parameter.</p> <hd id="AN0133291119-16">Latent Ability</hd> <p>When dimensions were highly correlated, the MSE between the estimated ability parameter and any true measure of ability ( and ) was very similar. Furthermore, the MSE of the estimated ability parameter with the true ability on which the test form had more items—the first dimension in this study, tended to be smaller than the MSE of the estimated ability parameter with the true ability on the second dimension, which had fewer items measuring that dimension. For forms having 40% and 80% of common items, the MSEs of the latent trait tended to be consistent across forms at same level of correlation, indicating little effect of the confounding of difficulty within dimensions or differences in total test difficulty across equated forms. As seen in Figure 4 and reported in Tables 5, 6, and 7, when the number of common items was small (20%), the MSEs between the estimated ability parameter and any true measure of ability was larger than other conditions with more common items (40% and 80% common items). Overall, in every condition, the systematic error was much larger than random error, and accounted for at least 51% of the total error. The MSE and SB values of ability parameters were larger than those of item parameters. This meant that the estimated ability parameters using unidimensional estimation and linking method could be biased when multidimensional response data sets with confounding difficulty within dimensions were used.</p> <p>PHOTO (COLOR): FIGURE 4 Average MSE of the estimated unidimensional ability parameter.</p> <p>Average MSE, SB, and VAR of the Estimated Unidimensional Ability with the True Multidimensional Ability on Each Dimension and the Average (The number of common items: 8 [20%])</p> <p> <ephtml> <table border="1" cellpadding="11"><tbody><tr><td align="left" /><td align="center" /><td align="center" colspan="3"><inline-graphic href="" /></td><td align="center" colspan="3"><inline-graphic href="" /></td><td align="center" colspan="3"><inline-graphic href="" /></td></tr><tr><td align="left">Form</td><td align="center"><inline-graphic href="" /></td><td align="center">MSE</td><td align="center">SB</td><td align="center">VAR</td><td align="center">MSE</td><td align="center">SB</td><td align="center">VAR</td><td align="center">MSE</td><td align="center">SB</td><td align="center">VAR</td></tr></tbody><tbody><tr><td align="center">2</td><td align="center">0</td><td align="center">2.399</td><td align="center">2.347</td><td align="center">0.052</td><td align="center">2.504</td><td align="center">2.452</td><td align="center">0.052</td><td align="center">1.949</td><td align="center">1.898</td><td align="center">0.052</td></tr><tr><td align="center" /><td align="center">.3</td><td align="center">2.287</td><td align="center">2.240</td><td align="center">0.047</td><td align="center">2.491</td><td align="center">2.444</td><td align="center">0.047</td><td align="center">2.024</td><td align="center">1.977</td><td align="center">0.047</td></tr><tr><td align="center" /><td align="center">.6</td><td align="center">4.225</td><td align="center">2.314</td><td align="center">1.953</td><td align="center">4.286</td><td align="center">2.375</td><td align="center">1.953</td><td align="center">4.064</td><td align="center">2.153</td><td align="center">1.953</td></tr><tr><td align="center" /><td align="center">.9</td><td align="center">3.867</td><td align="center">2.256</td><td align="center">1.645</td><td align="center">3.885</td><td align="center">2.274</td><td align="center">1.645</td><td align="center">3.827</td><td align="center">2.216</td><td align="center">1.645</td></tr><tr><td align="center">3</td><td align="center">0</td><td align="center">2.399</td><td align="center">2.349</td><td align="center">0.050</td><td align="center">2.517</td><td align="center">2.467</td><td align="center">0.050</td><td align="center">1.956</td><td align="center">1.906</td><td align="center">0.050</td></tr><tr><td align="center" /><td align="center">.3</td><td align="center">4.058</td><td align="center">2.346</td><td align="center">1.746</td><td align="center">4.267</td><td align="center">2.555</td><td align="center">1.746</td><td align="center">3.797</td><td align="center">2.085</td><td align="center">1.746</td></tr><tr><td align="center" /><td align="center">.6</td><td align="center">4.222</td><td align="center">2.323</td><td align="center">1.941</td><td align="center">4.282</td><td align="center">2.382</td><td align="center">1.941</td><td align="center">4.061</td><td align="center">2.161</td><td align="center">1.941</td></tr><tr><td align="center" /><td align="center">.9</td><td align="center">3.874</td><td align="center">2.258</td><td align="center">1.649</td><td align="center">3.893</td><td align="center">2.276</td><td align="center">1.649</td><td align="center">3.834</td><td align="center">2.218</td><td align="center">1.649</td></tr><tr><td align="center">4</td><td align="center">0</td><td align="center">4.304</td><td align="center">2.432</td><td align="center">1.905</td><td align="center">4.407</td><td align="center">2.535</td><td align="center">1.905</td><td align="center">3.854</td><td align="center">1.981</td><td align="center">1.905</td></tr><tr><td align="center" /><td align="center">.3</td><td align="center">4.011</td><td align="center">2.327</td><td align="center">1.719</td><td align="center">4.219</td><td align="center">2.534</td><td align="center">1.719</td><td align="center">3.750</td><td align="center">2.065</td><td align="center">1.719</td></tr><tr><td align="center" /><td align="center">.6</td><td align="center">4.185</td><td align="center">2.314</td><td align="center">1.911</td><td align="center">4.235</td><td align="center">2.364</td><td align="center">1.911</td><td align="center">4.019</td><td align="center">2.147</td><td align="center">1.911</td></tr><tr><td align="center" /><td align="center">.9</td><td align="center">3.812</td><td align="center">2.231</td><td align="center">1.614</td><td align="center">3.831</td><td align="center">2.249</td><td align="center">1.614</td><td align="center">3.772</td><td align="center">2.191</td><td align="center">1.614</td></tr></tbody></table> </ephtml> <emph>Note.</emph> MSE = Mean Square Error, SB = Squared Bias, VAR = Variance</p> <p>Average MSE, SB, and VAR of the Estimated Unidimensional Ability with the True Multidimensional Ability on Each Dimension and the Average (The number of common items: 16 [40%])</p> <p> <ephtml> <table border="1" cellpadding="11"><tbody><tr><td align="center" /><td align="center" /><td align="center" colspan="3"><inline-graphic href="" /></td><td align="center" colspan="3"><inline-graphic href="" /></td><td align="center" colspan="3"><inline-graphic href="" /></td></tr><tr><td align="center">Form</td><td align="center"><inline-graphic href="" /></td><td align="center">MSE</td><td align="center">SB</td><td align="center">VAR</td><td align="center">MSE</td><td align="center">SB</td><td align="center">VAR</td><td align="center">MSE</td><td align="center">SB</td><td align="center">VAR</td></tr></tbody><tbody><tr><td align="center">2</td><td align="center">0</td><td align="center">2.398</td><td align="center">2.346</td><td align="center">0.051</td><td align="center">2.505</td><td align="center">2.453</td><td align="center">0.051</td><td align="center">1.949</td><td align="center">1.898</td><td align="center">0.051</td></tr><tr><td align="center" /><td align="center">.3</td><td align="center">2.271</td><td align="center">2.224</td><td align="center">0.047</td><td align="center">2.476</td><td align="center">2.429</td><td align="center">0.047</td><td align="center">2.008</td><td align="center">1.961</td><td align="center">0.047</td></tr><tr><td align="center" /><td align="center">.6</td><td align="center">2.272</td><td align="center">2.226</td><td align="center">0.046</td><td align="center">2.322</td><td align="center">2.276</td><td align="center">0.046</td><td align="center">2.106</td><td align="center">2.060</td><td align="center">0.046</td></tr><tr><td align="center" /><td align="center">.9</td><td align="center">2.217</td><td align="center">2.173</td><td align="center">0.044</td><td align="center">2.232</td><td align="center">2.187</td><td align="center">0.044</td><td align="center">2.175</td><td align="center">2.131</td><td align="center">0.044</td></tr><tr><td align="center">3</td><td align="center">0</td><td align="center">2.396</td><td align="center">2.346</td><td align="center">0.049</td><td align="center">2.515</td><td align="center">2.465</td><td align="center">0.049</td><td align="center">1.953</td><td align="center">1.904</td><td align="center">0.049</td></tr><tr><td align="center" /><td align="center">.3</td><td align="center">2.277</td><td align="center">2.231</td><td align="center">0.045</td><td align="center">2.477</td><td align="center">2.432</td><td align="center">0.045</td><td align="center">2.011</td><td align="center">1.966</td><td align="center">0.045</td></tr><tr><td align="center" /><td align="center">.6</td><td align="center">2.276</td><td align="center">2.231</td><td align="center">0.045</td><td align="center">2.321</td><td align="center">2.276</td><td align="center">0.045</td><td align="center">2.107</td><td align="center">2.062</td><td align="center">0.045</td></tr><tr><td align="center" /><td align="center">.9</td><td align="center">2.213</td><td align="center">2.169</td><td align="center">0.044</td><td align="center">2.226</td><td align="center">2.183</td><td align="center">0.044</td><td align="center">2.170</td><td align="center">2.126</td><td align="center">0.044</td></tr><tr><td align="center">4</td><td align="center">0</td><td align="center">2.384</td><td align="center">2.334</td><td align="center">0.050</td><td align="center">2.500</td><td align="center">2.450</td><td align="center">0.050</td><td align="center">1.940</td><td align="center">1.890</td><td align="center">0.050</td></tr><tr><td align="center" /><td align="center">.3</td><td align="center">2.268</td><td align="center">2.222</td><td align="center">0.046</td><td align="center">2.468</td><td align="center">2.422</td><td align="center">0.046</td><td align="center">2.002</td><td align="center">1.956</td><td align="center">0.046</td></tr><tr><td align="center" /><td align="center">.6</td><td align="center">2.299</td><td align="center">2.224</td><td align="center">0.045</td><td align="center">2.343</td><td align="center">2.268</td><td align="center">0.045</td><td align="center">2.130</td><td align="center">2.055</td><td align="center">0.045</td></tr><tr><td align="center" /><td align="center">.9</td><td align="center">2.204</td><td align="center">2.160</td><td align="center">0.044</td><td align="center">2.217</td><td align="center">2.173</td><td align="center">0.044</td><td align="center">2.161</td><td align="center">2.117</td><td align="center">0.044</td></tr></tbody></table> </ephtml> <emph>Note.</emph> MSE = Mean Square Error, SB = Squared Bias, VAR = Variance</p> <p>Average MSE, SB, and VAR of the Estimated Unidimensional Ability with the True Multidimensional Ability on Each Dimension and the Average (The number of common items: 32 [80%])</p> <p> <ephtml> <table border="1" cellpadding="11"><tbody><tr><td align="center" /><td align="center" /><td align="center" colspan="3"><inline-graphic href="" /></td><td align="center" colspan="3"><inline-graphic href="" /></td><td align="center" colspan="3"><inline-graphic href="" /></td></tr><tr><td align="left">Form</td><td align="center"><inline-graphic href="" /></td><td align="center">MSE</td><td align="center">SB</td><td align="center">VAR</td><td align="center">MSE</td><td align="center">SB</td><td align="center">VAR</td><td align="center">MSE</td><td align="center">SB</td><td align="center">VAR</td></tr></tbody><tbody><tr><td align="center">2</td><td align="center">0</td><td align="center">2.413</td><td align="center">2.362</td><td align="center">0.051</td><td align="center">2.515</td><td align="center">2.465</td><td align="center">0.051</td><td align="center">1.960</td><td align="center">1.909</td><td align="center">0.051</td></tr><tr><td align="center" /><td align="center">.3</td><td align="center">2.269</td><td align="center">2.223</td><td align="center">0.046</td><td align="center">2.475</td><td align="center">2.429</td><td align="center">0.046</td><td align="center">2.008</td><td align="center">1.962</td><td align="center">0.046</td></tr><tr><td align="center" /><td align="center">.6</td><td align="center">2.271</td><td align="center">2.225</td><td align="center">0.046</td><td align="center">2.321</td><td align="center">2.275</td><td align="center">0.046</td><td align="center">2.105</td><td align="center">2.059</td><td align="center">0.046</td></tr><tr><td align="center" /><td align="center">.9</td><td align="center">2.207</td><td align="center">2.163</td><td align="center">0.044</td><td align="center">2.222</td><td align="center">2.178</td><td align="center">0.044</td><td align="center">2.165</td><td align="center">2.121</td><td align="center">0.044</td></tr><tr><td align="center">3</td><td align="center">0</td><td align="center">2.408</td><td align="center">2.359</td><td align="center">0.049</td><td align="center">2.522</td><td align="center">2.473</td><td align="center">0.049</td><td align="center">1.961</td><td align="center">1.911</td><td align="center">0.049</td></tr><tr><td align="center" /><td align="center">.3</td><td align="center">2.271</td><td align="center">2.226</td><td align="center">0.045</td><td align="center">2.475</td><td align="center">2.430</td><td align="center">0.045</td><td align="center">2.009</td><td align="center">1.964</td><td align="center">0.045</td></tr><tr><td align="center" /><td align="center">.6</td><td align="center">2.269</td><td align="center">2.225</td><td align="center">0.045</td><td align="center">2.316</td><td align="center">2.272</td><td align="center">0.045</td><td align="center">2.101</td><td align="center">2.057</td><td align="center">0.045</td></tr><tr><td align="center" /><td align="center">.9</td><td align="center">2.199</td><td align="center">2.155</td><td align="center">0.044</td><td align="center">2.212</td><td align="center">2.168</td><td align="center">0.044</td><td align="center">2.156</td><td align="center">2.112</td><td align="center">0.044</td></tr><tr><td align="center">4</td><td align="center">0</td><td align="center">2.395</td><td align="center">2.345</td><td align="center">0.050</td><td align="center">2.507</td><td align="center">2.457</td><td align="center">0.050</td><td align="center">1.946</td><td align="center">1.896</td><td align="center">0.050</td></tr><tr><td align="center" /><td align="center">.3</td><td align="center">2.262</td><td align="center">2.217</td><td align="center">0.045</td><td align="center">2.465</td><td align="center">2.419</td><td align="center">0.045</td><td align="center">2.000</td><td align="center">1.955</td><td align="center">0.045</td></tr><tr><td align="center" /><td align="center">.6</td><td align="center">2.263</td><td align="center">2.218</td><td align="center">0.045</td><td align="center">2.307</td><td align="center">2.262</td><td align="center">0.045</td><td align="center">2.093</td><td align="center">2.048</td><td align="center">0.045</td></tr><tr><td align="center" /><td align="center">.9</td><td align="center">2.190</td><td align="center">2.146</td><td align="center">0.044</td><td align="center">2.203</td><td align="center">2.159</td><td align="center">0.044</td><td align="center">2.147</td><td align="center">2.103</td><td align="center">0.044</td></tr></tbody></table> </ephtml> <emph>Note.</emph> MSE = Mean Square Error, SB = Squared Bias, VAR = Variance.</p> <hd id="AN0133291119-17">DISCUSSION</hd> <p>Many large-scale standardized tests are administered using several test forms. Thus testing companies (ETS, ACT, KICE, etc.) develop new test forms that assess multiple content areas with a set of common items, and new test forms should be linked and equated to the reference test form. This is done in order to put the parameters across forms on the same scale. Unfortunately, a unidimensional linking and/or equating method is often used in practice, although data are potentially multidimensional. While multidimensional linking and equating procedures have been developed, unidimensional analyses continue to be a standard application. The FIPC method is becoming an increasingly popular method to perform linking (Hickman et al., 2012; Kim et al., 2011; OECD, 2012; Wainer & Wang, 2001). A potential issue when administering forms that assess multiple sub-content areas is the potential of confounding difficulty within dimensions, even after using linking methods to adjust in total test difficulty. The purpose of this study was to investigate the effects of linking test forms with and without confounding difficulty within dimensions using the FIPC method at various levels of correlation and numbers of common items.</p> <p>Under conditions within this study, the estimated unidimensional discrimination was less affected by differing test forms, i.e., total test difficulty or confounding difficulty within dimensions, but it was affected by other conditions, such as varied correlation and proportion of common items. The estimated discrimination tended to be a closer to the true , a measure of multidimensional discrimination, than the individual true discriminations on each dimension (), when dimensions were correlated. When dimensions were uncorrelated, the estimated discrimination might be a close the average of the true discriminations. Furthermore, for the same test form structure, as the number of common items increased, the MSEs tended to decreased when the correlation was low. In contrast, as the number of common items increased, the MSE increased when the correlation was high. Thus, if the data are potentially multidimensional, this brings awareness to test developers and psychometricians to give special care to the interpretations according to the degree of correlation.</p> <p>The estimated unidimensional difficulty was slightly affected by unequal total test difficulty across multiple forms, and highly affected by confounding difficulty within dimensions. When multiple forms were constructed, with some forms having difficult items measuring first dimension and easy items measuring a second dimension but another form having easy items measuring the first dimension and hard items measuring the second dimension, the estimated unidimensional difficulty of items was likely to be highly biased. The estimated unidimensional difficulty parameter was only slightly affected by the number of common items, where forms with a larger number of common items produced a more stable estimated difficulty parameter across all levels of correlation.</p> <p>The estimated ability of examinees was less affected by different test specifications, and more affected by the correlation of the dimensions and the number of common items. As correlation increased, the MSE of the estimated ability with the average of true ability increased, but the MSE of the estimated ability with the true ability on either dimension decreased. The estimated ability tended to be closer to the examinee's true ability on the sub-content area having more items. The number of common items had a strong influence on the estimated ability parameters, where forms with a small number of common items produced estimates with higher MSEs. As a result, the results of this study recommend having more than 40% of the common items across forms for obtaining stable results.</p> <p>The implications of this study are important to the test developers of large-scale standardized testing companies. Especially, this study will be defensible and comfortable for users of the FIPC linking method. One may think that since the estimated scores of examinees are slightly affected by confounding difficulty within dimensions across test forms, that this issue is not a priority. However, item parameter estimates are of value to those developing tests that meet certain specifications. When building multiple test forms, developers pay close attention to total test difficulty of each form and maintaining the same numbers of items within sub-content areas across forms for purposes of reliability and validity. The results of this study also bring awareness to the need for attention to average item difficulty within sub-content areas. When the difficulty of items within sub-content areas differs across forms (See Table 1 for the differences in the magnitude of difficulty in this study), the estimated item difficulty was likely to be biased, which is important in the test development phase of form construction and item revision. At least 40% of the total number of total test items are needed to be common across forms in order to produce stable and accurate results in the FIPC linking procedure since the estimated ability parameters is greatly influenced by the number of common items.</p> <p>In conclusion, when the data set have the characteristic of multidimensionality with confounding difficulty within dimensions, but the unidimensional FIPC linking method is used, the estimated item and ability parameters should be interpreted carefully based on the degree of the correlation levels, the amount of confounding difficulty, and the number of common items. When new test forms are developed, common items with at least 40% of total test items are recommended to produce accurate and stable ability parameter estimates. Since the effect of the confounding difficulty within dimensions was more severe than the effect of differing total test difficulty across forms on estimated difficulty parameters, developer should be more aware of constructing test forms where the difficulty of items within sub-content area is consistent.</p> <p>In the future, studies of the effects of multidimensionality on unidimensional linking methods may also include other varied characteristics such as the number of items within the different sub-content areas, non-normal ability distributions, more dimensions, and comparisons to other linking procedures. Further studies may also include the application of multidimensional linking methods.</p> <ref id="AN0133291119-18"> <title> REFERENCES </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Ackerman, T. A. ( 1989 ). Unidimensional IRT calibration of compensatory and noncompensatory items. <emph>Applied Psychological Measurement</emph>, <emph>13</emph>, 113 - 127. doi: 10.1177/014662168901300201</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> Ackerman, T. A. ( 1994 ). Creating a test information profile in a two-dimensional latent space. <emph>Applied Psychological Measurement</emph>, <emph>18</emph>, 257 - 275. doi: 10.1177/014662169401800306</bibtext> </blist> <blist> <bibl id="bib3" type="bt">3</bibl> <bibtext> ACT. ( 2014 ). <emph>The ACT technical manual</emph>. Iowa City, IA : ACT, Inc.</bibtext> </blist> <blist> <bibl id="bib4" type="bt">4</bibl> <bibtext> Allen, N. A., Donoghue, J. R., & Schoeps, T. L. ( 2001 ). <emph>The NAEP 1998 Technical Report (NCES 2001-452)</emph>. Washington, DC : United States Department of Education, Institute of Education Sciences, Department of Education, Office for Educational Research and Improvement.</bibtext> </blist> <blist> <bibl id="bib5" type="bt">5</bibl> <bibtext> Andrews, B. J. ( 2011 ). <emph>Assessing first- and second-order equity for the common-item nonequivalent groups design using multidimensional IRT</emph>. ( Doctoral dissertation ). Retrieved from The University of Iowa, Iowa Research Online, <ulink href="http://ir.uiowa.edu/etd/1196">http://ir.uiowa.edu/etd/1196</ulink></bibtext> </blist> <blist> <bibl id="bib6" type="bt">6</bibl> <bibtext> Baker, F., & Kim, S-H. ( 2004 ). <emph>Item response theory: Parameter estimation techniques</emph> ( 2nd Ed.). New York, NY : Marcel Dekker, Inc.</bibtext> </blist> <blist> <bibl id="bib7" type="bt">7</bibl> <bibtext> Baldwin, S. G., Baldwin, P., & Nering, M. L. ( 2007 ). A comparison of IRT equating methods on recovering item parameters and growth in mixed-format tests. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref3" type="bt">8</bibl> <bibtext> Béguin, A. A., & Hanson, B. A. ( 2001 ). Effect of noncompensatory multidimensionality on separate and concurrent estimation in IRT observed score equating. Paper presented at the annual meeting of the National Council of Measurement in Education, Seattle, WA.</bibtext> </blist> <blist> <bibl id="bib9" type="bt">9</bibl> <bibtext> Béguin, A. A., Hanson, B. A., & Glas, C. A. W. ( 2000 ). <emph>Effect of noncompensatory multidimensionality on separate and concurrent estimation in IRT observed score equating</emph>. Paper presented at the annual meeting of the National Council of Measurement in Education, Seattle, WA.</bibtext> </blist> <blist> <bibtext> Birnbaum, A. ( 1968 ). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord, & M. R. Novick, (Eds.), <emph>Statistical theories of mental test scores</emph> (pp. 397 - 472 ). Reading, MA : Addison-Wesley.</bibtext> </blist> <blist> <bibtext> Bolt, D. M. ( 1999 ). Evaluating the effects of multidimensionality on IRT true-score equating. <emph>Applied Measurement in Education</emph>, <emph>12</emph> ( 4 ), 383 - 407. doi: 10.1207/S15324818AME1204_4</bibtext> </blist> <blist> <bibtext> Bolt, D. M., & Lall, V. F. ( 2003 ). Estimation of compensatory and noncompensatory multidimensional item response models using Markov Chain Monte Carlo. <emph>Applied Psychological Measurement</emph>, <emph>27</emph> ( 6 ), 395 - 414. doi: 10.1177/0146621603258350</bibtext> </blist> <blist> <bibtext> Camille, G., Wang, M., & Fesq, J. (1995). The effects of dimensionality on equating the Law School Admission Test. <emph>Journal of Educational Measurement</emph>, <emph>32</emph>, 79-96.</bibtext> </blist> <blist> <bibtext> Chalmers, P. ( 2012 ). mirt: A multidimensional item response theory package for the R environment. <emph>Journal of Statistical Software</emph>, <emph>48</emph> ( 6 ), 1 - 29. doi: 10.18637/jss.v048.i06</bibtext> </blist> <blist> <bibtext> Core Team, R. ( 2016 ). <emph>R: A language and environment for statistical computing</emph>. Vienna, Austria : R Foundation for Statistical Computing. Retrieved from <ulink href="http://222.R-project.org">http://222.R-project.org</ulink></bibtext> </blist> <blist> <bibtext> De Chaplain, A. F. ( 1996 ). The effect of multidimensionality on IRT true-score equating for subgroups of examinees. <emph>Journal of Educational Measurement</emph>, <emph>33</emph> ( 2 ), 181 - 201. doi: 10.1111/j.1745-3984.1996.tb00488.x</bibtext> </blist> <blist> <bibtext> Hanson, B. A., & Béguin, A. A. ( 1999 ). Obtaining a common scale for IRT item parameters using separate versus concurrent estimation in the common item nonequivalent groups equating design<emph>.</emph> Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.</bibtext> </blist> <blist> <bibtext> Harris, D. J., & Kolen, M. J. ( 1986 ). Effect of examinee group on equating relationships. <emph>Applied Psychological Measurement</emph>, <emph>10</emph> ( 1 ), 35 - 43. doi: 10.1177/014662168601000103</bibtext> </blist> <blist> <bibtext> Hickman, J. J., Fu, J., & Hill, H. C. ( 2012 ). <emph>Technical report: Creation and dissemination of upper-elementary mathematics assessment modules</emph>. Princeton, NJ: Educational Testing Service.</bibtext> </blist> <blist> <bibtext> Hu, H., Rogers, W. T., & Vukmirovic, Z. ( 2008 ). Investigation of IRT-based equating methods in the presence of outlier common items. <emph>Applied Psychological Measurement</emph>, <emph>32</emph> ( 4 ), 311 - 333. doi: 10.1177/0146621606292215</bibtext> </blist> <blist> <bibtext> Kang, T., & Petersen, N. ( 2009 ). Linking item parameters to a base scale. <emph>Asia Pacific Education Review</emph>, <emph>13</emph> ( 2 ), 331 - 321.</bibtext> </blist> <blist> <bibtext> Keller, L. A., Keller, R. R., & Parker, P. A. ( 2010 ). The examination of the classification of students into performance categories by two different equating methods. <emph>The Journal of Experimental Education</emph>, <emph>79</emph> ( 1 ), 30 - 52. doi: 10.1080/00220970903292959</bibtext> </blist> <blist> <bibtext> Keller, R. R., Keller, L. A., & Baldwin, S. ( 2007 ). The effect of changing equating methods on monitoring growth in mixed-format tests. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.</bibtext> </blist> <blist> <bibtext> Kim, S. ( 2006 ). A comparative study of IRT fixed parameter calibration methods. <emph>Journal of Educational Measurement</emph>, <emph>43</emph>, 355 - 381. doi: 10.1111/j.1745-3984.2006.00021.x</bibtext> </blist> <blist> <bibtext> Kim, S. ( 2008 ). IRT fixed parameter calibration and other approaches to maintaining item parameters on a common ability scale. Paper presented at the Measured Progress, Dover, NH.</bibtext> </blist> <blist> <bibtext> Kim, S., & Kang, T. ( 2012 ). A comparison of MCMC and MMLE/EM algorithms for fixed item parameter calibraion. <emph>Journal of Educational Evaluation</emph>, <emph>25</emph> ( 2 ), 337 - 350.</bibtext> </blist> <blist> <bibtext> Kim, S., & Kolen, M. J. ( 2006 ). Robustness to format effects on IRT linking methods for mixed-format tests. <emph>Applied Measurement in Education</emph>, <emph>19</emph> ( 4 ), 357 - 381. doi: 10.1207/s15324818ame1904_7</bibtext> </blist> <blist> <bibtext> Kim, G-H., Kim, W-S., Choi, I-B., Kim, H-G., Sang, G-A., & Shin, J. ( 2011 ). <emph>2011 National Assessment of Educational Achievement Technical Report</emph> (RRE 2011-2-5). Seoul, South Korea: Korea Institute for Curriculum and Evaluation.</bibtext> </blist> <blist> <bibtext> Kolen, M. J., & Brennan, R. L. ( 2014 ). <emph>Test equating, scaling, and linking: Methods and practice</emph> ( 3rd Edition). New York, NY : Springer.</bibtext> </blist> <blist> <bibtext> Lee, E., Lee, W. C., & Brennan, R. L. ( 2014 ). <emph>Equating multidimensional tests under a random groups design: A comparison of various equating procedures.</emph> Center for Advanced Studies in Measurement and Assessment Research Report Series (Number 40). University of Iowa, Iowa City, IA.</bibtext> </blist> <blist> <bibtext> Miller, T. R. ( 1991 ). <emph>Empirical estimation of standard errors of compensatory MIRT model parameters obtained from the NOHARM estimation program</emph>. Iowa City, IA : American College Testing Program.</bibtext> </blist> <blist> <bibtext> Muraki, E., & Bock, R. D. ( 2003 ). <emph>PARSCALE (Version 4.1) [Computer software]</emph>. Chicago, IL : Scientific Software International.</bibtext> </blist> <blist> <bibtext> OECD ( 2012 ). <emph>PISA 2009 technical report, OECD Publishing</emph>. Retrieved from <ulink href="http://dx.doi.org/">http://dx.doi.org/</ulink> 10.1787/9789264167872-en</bibtext> </blist> <blist> <bibtext> Paek, I., & Young, M. J. ( 2005 ). Investigation of student growth recovery in a fixed-item linking procedure with a fixed-person prior distribution for mixed-format test data. <emph>Applied Measurement in Education</emph>, <emph>18</emph>, 199 - 215. doi: 10.1207/s15324818ame1802_4</bibtext> </blist> <blist> <bibtext> Pang, X., Madera, E., Radwan, N., & Zhang, S. ( 2010 ). A comparison of four test equating methods. EQAO Research Report. Toronto, ON: EQAO Research.</bibtext> </blist> <blist> <bibtext> Reckase, M. D. ( 1985 ). The difficulty of test items that measure more than one ability. <emph>Applied Psychological Measurement</emph>, <emph>9</emph>, 401 - 412. doi: 10.1177/014662168500900409</bibtext> </blist> <blist> <bibtext> Reckase, M. D. ( 2009 ). <emph>Multidimensional item response theory</emph>. New York, NY : Springer.</bibtext> </blist> <blist> <bibtext> Reckase, M. D., & McKinley, R. L. ( 1991 ). The discriminating power of items that measure more than one dimension. <emph>Applied Psychological Measurement</emph>, <emph>15</emph> ( 4 ), 361 - 373. doi: 10.1177/014662169101500407</bibtext> </blist> <blist> <bibtext> Samuel, A. L. ( 2014 ). <emph>Equating test scores (without IRT)</emph> 2nd ed. Princeton, NJ: Educational Testing Service.</bibtext> </blist> <blist> <bibtext> Skorupski, W. P., Jodoin, M. G., Keller, L. A., & Swaminathan, H. ( 2003 ). An evaluation of item response theory equating procedures for capturing growth with tests composed of dichotomously scored items. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.</bibtext> </blist> <blist> <bibtext> Spence, P. D. ( 1996 ). The effect of multidimensionality on unidimensional equating with item response theory ( Doctoral dissertation ) ( UMI Number: 9703612 ). The University of Florida, Gainesville, FL.</bibtext> </blist> <blist> <bibtext> Spray, J. A. ( 1990 ). <emph>Comparison of two logistic multidimensional item response theory models</emph>. Iowa City, IA: American College Testing Program.</bibtext> </blist> <blist> <bibtext> Sympson, J. B. ( 1978 ). A model for testing with multidimensional items. In D. J. Weiss (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference. University of Minnesota, Minneapolis, MN.</bibtext> </blist> <blist> <bibtext> Wainer, H., & Wang, X. ( 2001 ). Using a new statistical model for testlets to score TOEFL. <emph>TOEFL Technical Report</emph> (Number TR-16, RR-01-09). Princeton, NJ: Educational Testing Service.</bibtext> </blist> <blist> <bibtext> Zhang, J., & Stout, W. F. ( 1999 ). Conditional covariance structure of generalized compensatory multidimensional item. <emph>Psychometrika</emph>, <emph>64</emph>, 129 - 152. doi: 10.1007/BF02294532</bibtext> </blist> </ref> <aug> <p>By Sohee Kim; Ki Lynn Cole and Mwarumba Mwavita</p> </aug>
Header DbId: eric
DbLabel: ERIC
An: EJ1198319
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: FIPC Linking across Multidimensional Test Forms: Effects of Confounding Difficulty within Dimensions
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Kim%2C+Sohee%22">Kim, Sohee</searchLink><br /><searchLink fieldCode="AR" term="%22Cole%2C+Ki+Lynn%22">Cole, Ki Lynn</searchLink><br /><searchLink fieldCode="AR" term="%22Mwavita%2C+Mwarumba%22">Mwavita, Mwarumba</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Testing%22"><i>International Journal of Testing</i></searchLink>. 2018 18(4):323-345.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 23
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2018
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Error+of+Measurement%22">Error of Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22Testing+Problems%22">Testing Problems</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/15305058.2018.1428980
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1530-5058
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study investigated the effects of linking potentially multidimensional test forms using the fixed item parameter calibration. Forms had equal or unequal total test difficulty with and without confounding difficulty. The mean square errors and bias of estimated item and ability parameters were compared across the various confounding tests. The estimated discrimination parameters were influenced by the levels of correlation between dimensions. The mean square errors (MSEs) of the average of the true discrimination parameters with the estimated value were smallest when the correlation equaled 0; however, the MSEs of the multidimensional discrimination parameter were smallest when the correlation was larger than 0. The estimated difficulty parameters were highly affected by different amount of confounding difficulty within dimensions. Furthermore, the MSEs of the average of the true ability parameters on the first and second dimensions with the estimated ability were smaller than those from the ability parameter on each dimension for all conditions. The pattern varied according to the number of common items, and the measures of MSE and squared bias were relatively consistent across forms at the same level of correlation, except for the condition where the correlation was 0 and the number of common items was 8.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 44
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2018
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1198319
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1198319
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/15305058.2018.1428980
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 23
        StartPage: 323
    Subjects:
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Test Construction
        Type: general
      – SubjectFull: Error of Measurement
        Type: general
      – SubjectFull: Testing Problems
        Type: general
    Titles:
      – TitleFull: FIPC Linking across Multidimensional Test Forms: Effects of Confounding Difficulty within Dimensions
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Kim, Sohee
      – PersonEntity:
          Name:
            NameFull: Cole, Ki Lynn
      – PersonEntity:
          Name:
            NameFull: Mwavita, Mwarumba
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2018
          Identifiers:
            – Type: issn-print
              Value: 1530-5058
          Numbering:
            – Type: volume
              Value: 18
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: International Journal of Testing
              Type: main
ResultId 1