View in EDS HTML Full Text PDF Full Text

Modeling Dimensions Converging at the Upper Anchor in Learning Progressions: An Example of Micro-Evolution

Saved in:
Bibliographic Details
Title: Modeling Dimensions Converging at the Upper Anchor in Learning Progressions: An Example of Micro-Evolution
Language: English
Authors: Mingfeng Xue (ORCID 0000-0002-4801-3754), Mark Wilson (ORCID 0000-0002-0425-5305)
Source: Applied Measurement in Education. 2024 37(1):71-87.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 17
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Descriptors: Learning Trajectories, Educational Assessment, Item Response Theory, Evolution, Measurement Techniques, Test Items, Multiple Choice Tests
DOI: 10.1080/08957347.2024.2311934
ISSN: 0895-7347
1532-4818
Abstract: Multidimensionality is common in psychological and educational measurements. This study focuses on dimensions that converge at the upper anchor (i.e. the highest acquisition status defined in a learning progression) and compares different ways of dealing with them using the multidimensional random coefficients multinomial logit model and scale alignment methods. Assumptions underlying the four approaches studied are (a) ignoring the convergence, (b) recognizing the convergence of the dimensions, (c) treating convergence as a new dimension, and (d) separating the within-category multidimensionality and treating convergence as a new dimension. A learning progression about micro-evolution is used as an example, including model fits, step difficulties, and associations between dimensions, Wright maps are drawn, and inferences are made under the four building blocks of measurement development. Finally, the usefulness and weaknesses of the four approaches are discussed.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1413499
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHh8yBDZ_90z0F1X6KefZZgAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDFg9P3utZ7W0xBLScQIBEICBm4FImJTgxmbdc1ZXWga_WiPLxfSoCIP0elSlFO-FUJj-ihNO8F63PACyCd_pQte5OiF1IYgbRs2ffc0ZfGPpmv4TIJydlSp4nN-GOo5Tz4HogW2JYBgb2PqHO6Hi8mw7JcaiEp5c0pug28mrtmR7INFMimobcWD7Uzojw1ChS3GugX7AEEQuI4IzdPKuRszJBsysoDEVPRA4m0je
Text:
  Availability: 1
  Value: <anid>AN0175570131;7lg01jan.24;2024Feb23.05:30;v2.2.500</anid> <title id="AN0175570131-1">Modeling Dimensions Converging at the Upper Anchor in Learning Progressions: An Example of Micro-Evolution </title> <p>Multidimensionality is common in psychological and educational measurements. This study focuses on dimensions that converge at the upper anchor (i.e. the highest acquisition status defined in a learning progression) and compares different ways of dealing with them using the multidimensional random coefficients multinomial logit model and scale alignment methods. Assumptions underlying the four approaches studied are a) ignoring the convergence, b) recognizing the convergence of the dimensions, c) treating convergence as a new dimension, and d) separating the within-category multidimensionality and treating convergence as a new dimension. A learning progression about micro-evolution is used as an example, including model fits, step difficulties, and associations between dimensions, Wright maps are drawn, and inferences are made under the four building blocks of measurement development. Finally, the usefulness and weaknesses of the four approaches are discussed.</p> <p>Unidimensionality is the underpinning assumption of item response theory (IRT). However, measurement practices often go beyond unidimensionality, for example, a test might consist of multiple components and an essay might be rated on several aspects. Therefore, multidimensional item response theory (MIRT) models pervade to deal with such complexities. There are generally two kinds of MIRT models depending on where the multidimensionality occurs, that is, between-item and within-item MIRT (Wang, Wilson, & Adams, [<reflink idref="bib26" id="ref1">26</reflink>]). The former assumes there are multiple dimensions at the test level, and each item assesses one of them only; while the latter assumes at least one item assesses more than one dimension. From a compensatory perspective, the former is non-compensatory if the dimensions are orthogonal, and the latter is compensatory if the dimensions are non-orthogonal (Reckase, [<reflink idref="bib20" id="ref2">20</reflink>]; Way, Ansley, & Forsyth, [<reflink idref="bib27" id="ref3">27</reflink>]). Both of them are widely applied in practical settings (e.g., Wu & Adams, [<reflink idref="bib34" id="ref4">34</reflink>]) because of mutual information borrowed from different dimensions in scaling, estimation of correlations among dimensions, practical requirements, and so on (Wang, Wilson, & Adams, [<reflink idref="bib26" id="ref5">26</reflink>]).</p> <p>Nevertheless, the complexities of measurement do not cease at a clear distinction between between-item and within-item multidimensionality. For example, this study focuses on a measurement context with dimensions converging at the so-called upper anchor, the most sophisticated level that learners are expected to reach in a learning progression (Sikorski, [<reflink idref="bib22" id="ref6">22</reflink>]; see Figure 3). Namely, the top level is assumed to be a within-level multidimensional situation, while the levels below are between-level multidimensional. For example, an arithmetic test might require four basic arithmetic operations at the top level of performance; or a reading test might require both content knowledge and reading skills at the top.</p> <p>Graph: Figure 3. The learning progression framework of the micro-evolution study.</p> <p>The peculiarities of this kind of situation call for guidance for models and interpretation. Moreover, the importance of this study can also manifest itself in the anchors of learning progressions. Specifically, lower and upper anchors define the boundaries of learning progressions, in which the upper one represents the most desirable goal that come with more abstractness (Duschl, Maeng, & Sezen, [<reflink idref="bib7" id="ref7">7</reflink>]; Sikorski, [<reflink idref="bib22" id="ref8">22</reflink>]), and our study helps shed light on how to disentangle this abstractness. Therefore, this study endeavors to compare different approaches in addressing dimensions that converge at the upper anchor by leveraging the multidimensional random coefficients multinomial logit model (MRCMLM; Adams, Wilson, & Wang, [<reflink idref="bib2" id="ref9">2</reflink>]) and scale alignment methods (Feuerstahler & Wilson, [<reflink idref="bib9" id="ref10">9</reflink>]), using data from a learning progression project of micro-evolution. Then the study moves on to demonstrate how to interpret the results and discuss their usefulness and drawbacks under the framework of the four building blocks of measurement development (Wilson, [<reflink idref="bib30" id="ref11">30</reflink>]).</p> <hd id="AN0175570131-2">1. The Four Building Blocks</hd> <p>A four-building-block approach to measurement development has been proposed by Wilson ([<reflink idref="bib30" id="ref12">30</reflink>]), which consists of (i) construct maps, (ii) items design, (iii) outcome space, and (iv) statistical models (see Figure 1). The first block is designing a construct map, where the definition and waypoints of the constructs to be measured are made explicitly based on substantive literature reviews, observation, interviews, and so on. Note that the term "waypoint" is used in the four building blocks approach to signify qualitatively different ordinal levels of the construct, which are used to attach interpretations to certain locations of the underlying empirical scale developed for the construct (Wilson, [<reflink idref="bib30" id="ref13">30</reflink>]). The term "level" is often used in learning progression research to have a similar, though more generic, meaning (e.g., Duschl, Maeng, & Sezen, [<reflink idref="bib7" id="ref14">7</reflink>]; Sikorski, [<reflink idref="bib22" id="ref15">22</reflink>]) but to be aligned with the four building blocks approach, the following sections will keep using the term "waypoints." The second block is the design of items that manifest the definition and waypoints created in the first block; the third one is the decision on scoring and classifying responses so as to reflect the construct map, aka, outcome space; the final block is applying responses to statistical models to calibrate item parameters, such as the step parameters, in order to provide empirical evidence to support the construct map. Typically, Wright maps will be generated at this step, which will be contrasted against the construct map to examine how empirical results are aligned with theoretical waypoints. Admittedly, this is not a one-off procedure and iteration is often needed, for instance, based on the interim results, we might need to revise the construct map, items, and outcome spaces, and Wright maps. This framework has been successfully applied in the development of numerous measurements, such as, mathematics education (Rittle-Johnson, Matthew, Taylor, & McEldoon, [<reflink idref="bib21" id="ref16">21</reflink>]), 21st-century skills (Wilson & Scalise, [<reflink idref="bib32" id="ref17">32</reflink>]), reading performance (Dray, Brown, Diakow, Lee, & Wilson, [<reflink idref="bib6" id="ref18">6</reflink>]), science education (Cardace, Wilson, & Metz, [<reflink idref="bib5" id="ref19">5</reflink>]), the concept of collaboration skills (Wilson, Scalise, & Gochyyev, [<reflink idref="bib33" id="ref20">33</reflink>]), and so on. It is also demonstrated to be well aligned with learning progression (Morell, Collier, Black, & Wilson, [<reflink idref="bib18" id="ref21">18</reflink>]; Wilson, [<reflink idref="bib29" id="ref22">29</reflink>]). This study focuses on the final block – how to set up MRCMLMs that accommodate the dimensions converging at the upper anchor and align calibration results across dimensions to verify the construct map.</p> <p>Graph: Figure 1. The four building blocks from Wilson ([<reflink idref="bib30" id="ref23">30</reflink>]).</p> <hd id="AN0175570131-3">2. Multidimensional Measurement</hd> <p>There have been long-standing debates about the dimensionality of educational and psychological measurements, which can fall into two categories in terms of the four building blocks framework. First, multidimensionality occurs because there is more than one construct to be measured, which is commonly seen in educational and psychological measurements, such as the big five personality assessment (Gosling, Rentfrow, & Swann, [<reflink idref="bib10" id="ref24">10</reflink>]), the verbal, quantitative, analytical components in Graduate Record Examinations (GRE, Kuncel, Hezlett, & Ones, [<reflink idref="bib13" id="ref25">13</reflink>]), and the <emph>fit</emph> and <emph>process</emph> dimensions in the exemplar assessment of this study. Second, unidimensionality is assumed at the construct map, but other latent traits – which are peripheral to the construct to be measured – may be inadvertently included in items. An item bundle, where items share a common context or materials, is an example of this kind of multidimensionality, as the additional bundle effect can be seen as an undesirable dimension and distorts item independence (Wainer, Bradlow, & Wang, [<reflink idref="bib24" id="ref26">24</reflink>]; Wilson & Adams, [<reflink idref="bib31" id="ref27">31</reflink>]). Response styles, consistent preferences for specific options regardless of item contents, are often modeled as additional dimensions and thus render possible multidimensionality of items intended to measure a single latent trait (e.g., Böckenholt & Meiser, [<reflink idref="bib3" id="ref28">3</reflink>]; Xue & Chen, [<reflink idref="bib35" id="ref29">35</reflink>]). Therefore, the second kind of multidimensionality is undesirable but sometimes unavoidable. Some researchers even argue that although measurement is always unidimensional, measurement tools need not be (Adams, Wilson, & Wang, [<reflink idref="bib2" id="ref30">2</reflink>]; Humphreys, [<reflink idref="bib12" id="ref31">12</reflink>]).</p> <p>Numerous IRT models have been proposed to address multidimensional measurements and this study focuses on just a subset of these many possibilities, so we detail more on the models for the first kind. Multidimensional extension of the Rasch model, the two-parameter logistic (2PL) model, the 3PL model, the generalized partial credit model, and the graded response model have been proposed (e.g., Reckase, [<reflink idref="bib20" id="ref32">20</reflink>]) and widely used (e.g., McKinley & Way, [<reflink idref="bib16" id="ref33">16</reflink>]; Thomas, [<reflink idref="bib23" id="ref34">23</reflink>]; Williams, Everaert, & Gotham, [<reflink idref="bib28" id="ref35">28</reflink>]). These models can be further extended and classified into either between-item or within-item models depending on where multidimensionality is designed.</p> <p>Between-item MIRT models are simple and straightforward as a single latent trait has been assumed to dominate the probability of responses within each item (see Figure 2) and the interpretation of item parameters remains similar to those in unidimensional IRT models. It is common in standard testing and large-scale assessment, such as the GRE (Kuncel, Hezlett, & Ones, [<reflink idref="bib13" id="ref36">13</reflink>]) and literacy measurements of the Programme for International Student Assessment (PISA; OECD, [<reflink idref="bib19" id="ref37">19</reflink>]). For illustration, we use the multidimensional Rasch model, which can take the form:</p> <p>Graph: Figure 2. Examples of between- and within-item models.</p> <p>(<reflink idref="bib1" id="ref38">1</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">P</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>y</mi><mrow><mi mathvariant="italic">ip</mi></mrow></msub></mrow><mo>=</mo><mn>1</mn><mrow><mo>|</mo></mrow><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">pd</mi></mrow></msub></mrow></mrow></mfenced><mo>=</mo><mrow><mfrac><mrow><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi mathvariant="italic">θ</mi><mrow><mi mathvariant="italic">pd</mi></mrow></msub></mrow><mo>−</mo><mrow><msub><mi>b</mi><mrow><mi mathvariant="italic">id</mi></mrow></msub></mrow></mrow></mfenced></mrow><mrow><mn>1</mn><mo>+</mo><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">pd</mi></mrow></msub></mrow><mo>−</mo><mrow><msub><mi>b</mi><mrow><mi mathvariant="italic">id</mi></mrow></msub></mrow></mrow></mfenced></mrow></mfrac></mrow></math> </ephtml> </p> <p>where <emph>i</emph> and <emph>p</emph> index item and person, respectively; <emph>θ</emph> and <emph>b</emph> represent latent trait and item difficulty, respectively. Note that a subscript <emph>d</emph> is added to differentiate latent traits. Moreover, associations between latent traits will be estimated by default. And it is worth noting that correlation coefficients among latent traits estimated directly from MIRT are more accurate (i.e., they are approximately disattenuated) than correlation based on latent traits estimated separately from unidimensional IRT models (Wang, [<reflink idref="bib25" id="ref39">25</reflink>]).</p> <p>Within-item MIRT models are more complicated than their between-item counterparts, where multiple latent traits determine the probability of responses to a single item and are combined through summation. As expressed in Equation (<reflink idref="bib2" id="ref40">2</reflink>), responses to item <emph>i</emph> are determined by the sum <emph>θ</emph><subs><emph>1</emph></subs> and <emph>θ</emph><subs><emph>2</emph></subs>. They are useful in modeling performance in complex tasks that cannot be explained by a single ability dimension (Hartig & Höhler, [<reflink idref="bib11" id="ref41">11</reflink>])</p> <p>(<reflink idref="bib2" id="ref42">2</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">P</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>y</mi><mrow><mi mathvariant="italic">ip</mi></mrow></msub></mrow><mo>=</mo><mn>1</mn><mrow><mo>|</mo></mrow><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">p</mi><mn>1</mn></mrow></msub></mrow><mo>,</mo><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">p</mi><mn>2</mn></mrow></msub></mrow></mrow></mfenced><mo>=</mo><mrow><mfrac><mrow><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi mathvariant="italic">θ</mi><mrow><mi mathvariant="italic">p</mi><mn>1</mn></mrow></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">p</mi><mn>2</mn></mrow></msub></mrow><mo>−</mo><mrow><msub><mi>b</mi><mi>i</mi></msub></mrow></mrow></mfenced></mrow><mrow><mn>1</mn><mo>+</mo><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">p</mi><mn>1</mn></mrow></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">p</mi><mn>2</mn></mrow></msub></mrow><mo>−</mo><mrow><msub><mi>b</mi><mi>i</mi></msub></mrow></mrow></mfenced></mrow></mfrac></mrow></math> </ephtml> </p> <p>However, compensation between <emph>θ</emph>s makes the interpretation of <emph>b</emph><subs>i</subs> in Equation (<reflink idref="bib2" id="ref43">2</reflink>) less straightforward than in Equation (<reflink idref="bib1" id="ref44">1</reflink>) because we cannot simply ascribe it to any single dimension. Thus, in a Wright map for a within-item model, the items that span dimensions may not have a straightforward interpretation. However, in our motivating example, only the upper anchor requires this kind of multidimensionality.</p> <p>Nevertheless, the distinction of multidimensionality at the item level ignores the complex reality that might occur at the category level. For example, <emph>45–6 × 7 =?</emph> is a partial credit item involving order of operations, multiplication, and subtraction. To get one credit, students need to realize multiplication should be executed before subtraction (i.e., they must apply an appropriate order of operation laws) and do the multiplication correctly. Next, they should do the subtraction correctly to get full credit. If we simply model it as a within-item multidimensional item, we ignore the fact that skills requirements vary across categories and falsely assume all three skills are used in every step, which might distort the estimation. This complexity also applies to our example of a microevolution study, where only the upper anchor entails two dimensions.</p> <p>To model the complexity within categories, we leverage the MRCMLM, a multidimensional extension to the random coefficients multinomial logit model, to illustrate how to model constructs that converge at the top. The equations for the MRCMLM are given and detailed in the Methods section.</p> <hd id="AN0175570131-4">3. Motivating Example: A Learning Progression About Micro-Evolution for Young Learners</hd> <p>The Development of the Conceptual Underpinnings of Evolution (CUE) project aims at integrating curriculum and assessment of evolution for children by leveraging cognition theory and the concept of a learning progression (Cardace, [<reflink idref="bib4" id="ref45">4</reflink>]). As part of this project, a learning progression about micro-evolution was developed for second and third graders using the four building blocks (Wilson, [<reflink idref="bib29" id="ref46">29</reflink>]), which comprised both curricula and assessments (Metz et al., [<reflink idref="bib17" id="ref47">17</reflink>]). After some iterations, two constructs were designed (see Figure 3). The first construct is <emph>fit</emph>, which centers on an increasingly sophisticated understanding of the fit between organisms and their environment and contains four waypoints from F1 to F4; the second construct is <emph>process</emph>, which focuses on an increasingly adequate explanation of how the population of organisms changes over time (Metz et al., [<reflink idref="bib17" id="ref48">17</reflink>]), and it contains 7 waypoints from P1 to P7. Note that at the upper anchor of the framework, the two dimensions converge and make a new waypoint, that is, <emph>NS</emph> (i.e., initials for natural selection). Integration at the <emph>NS</emph> waypoint means that student's responses need to convey how organisms fit in their environments in terms of their structure and function as well as how the population of organisms changes in terms of the process of natural selection (Cardace, [<reflink idref="bib4" id="ref49">4</reflink>]; Metz et al., [<reflink idref="bib17" id="ref50">17</reflink>]).</p> <p>Curricula were developed for young students, in which measurement serves both diagnostic and summative purposes. Thus, Figure 3 is the construct map guiding curricula as well as measurement development (Cardace, Wilson, & Metz, [<reflink idref="bib5" id="ref51">5</reflink>]; Wilson, [<reflink idref="bib29" id="ref52">29</reflink>]). Eighteen items were designed to assess the two constructs, with five of them eliciting <emph>NS</emph> responses. The results from the pre- and post-curricula measurements show that the curricula improve students' predictions and explanations of the population of organisms changes across multiple generations (Metz et al., [<reflink idref="bib17" id="ref53">17</reflink>]). It further suggests that the four building blocks can be successfully applied to establish curricula and measurement development for learning progression (Cardace, Wilson, & Metz, [<reflink idref="bib5" id="ref54">5</reflink>]; Wilson, [<reflink idref="bib29" id="ref55">29</reflink>]).</p> <p>However, in the series of estimation models used by Metz et al. ([<reflink idref="bib17" id="ref56">17</reflink>]) and Cardace, Wilson, and Metz ([<reflink idref="bib5" id="ref57">5</reflink>]), the convergence at the <emph>NS</emph> waypoint was ignored and only a simple between-item MIRT model was used, treating <emph>NS</emph> as a higher waypoint than F4 in the <emph>fit</emph> dimension or P7 in the <emph>process</emph> dimension. For example, an item, asking how and why cheetahs evolve into what they are now, belongs to the construct of <emph>process</emph> but also assesses the <emph>NS</emph> waypoint, and Metz, Cardace and their colleagues treated <emph>NS</emph> as a higher waypoint in the construct of <emph>process</emph> without considering the role of the construct of <emph>fit</emph>. In this case, ignoring the dual role of the <emph>NS</emph> upper anchor was largely due to the interpretation difficulty for within-item multidimensional models, as mentioned above. The between-item MIRT model oversimplifies the assumptions in the construct maps and fails to recognize the combination feature of the <emph>NS</emph> waypoint, which is one of the goals of the curriculum. Of course, a within-item MIRT model is more complex only because the <emph>NS</emph> waypoint assesses the two constructs simultaneously.</p> <hd id="AN0175570131-5">4. The Current Study</hd> <p>This study endeavors to use the motivational example to illustrate different ways of addressing dimensions that come together at the upper anchor of learning progressions via the MRCMLM (Adams, Wilson, & Wang, [<reflink idref="bib2" id="ref58">2</reflink>]) and scale alignment method (Feuerstahler & Wilson, [<reflink idref="bib9" id="ref59">9</reflink>]) and how to interpret the results under the framework of the four building blocks (Wilson, [<reflink idref="bib30" id="ref60">30</reflink>]), so validity evidence can be extracted. First, model fits are compared to evaluate how responses are in line with different assumptions; second, step difficulties for the <emph>NS</emph> waypoint from different models are examined to investigate the influence of different assumptions on item parameters; third, we inspect the correlations among the latent trait estimates from the four models to evaluate the influence from the person side; finally, the Wright maps for the different models are investigated. This paper contributes to the literature in three ways: (<reflink idref="bib1" id="ref61">1</reflink>) revealing how different assumptions underlying IRT models can be related to different assumptions for the upper anchors of learning progressions; (<reflink idref="bib2" id="ref62">2</reflink>) advancing knowledge in the effects of mis-specified multidimensional IRT models; and (<reflink idref="bib3" id="ref63">3</reflink>) providing guidance for dealing with one kind of abstractness in the upper anchor of learning progressions (i.e., convergence of two dimensions).</p> <hd id="AN0175570131-6">5. Methods</hd> <p></p> <hd id="AN0175570131-7">5.1. Measurement</hd> <p>Measurements took the form of a structured interview where an interviewer presented the student with an image of certain organisms in nature and asked what keeps them surviving. An interview protocol was developed to standardize the interview process and elicit students' responses in detail. The interviews were recorded on videos. Raters discussed the scoring guides until consensus was reached and co-rated some responses until they had 80% agreement before they coded responses individually. Finally, 61% of the responses were doubly-rated. One hundred and twenty-one students, who participated in the curricula, received the interview and 65 of them took both pre- and post-curricula interviews. In total, there are 2,570 responses collected from 242 interviews and 20 <emph>NS</emph> responses were observed.</p> <p>The 18 items were created to assess students' understanding of the microevolutionary concept with context covering both flora (e.g., "Why do you think kelp live there?") and fauna (e.g., "Why do you think otters live there?"). Seven items belong to the <emph>fit</emph> construct and two of them capture <emph>NS</emph> waypoint responses, and one <emph>fit</emph> item was dropped due to misfit; 11 items belong to the <emph>process</emph> construct and three of them capture the <emph>NS</emph> waypoint responses.</p> <hd id="AN0175570131-8">5.2. The Multidimensional Random Coefficients Multinomial Logit Model (MRCMLM)</hd> <p>The MRCMLM is a multidimensional extension of RCMLM and provides a general and flexible framework for modeling. For item <emph>i</emph> with <emph>K</emph><subs><emph>i</emph></subs><emph>+1</emph> categories (<emph>k</emph> = <emph>0, 1, ... , K</emph><subs><emph>i</emph></subs>), its responses can be dichotomized as</p> <p>(<reflink idref="bib3" id="ref64">3</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>X</mi><mrow><mi mathvariant="italic">ij</mi></mrow></msub></mrow><mo>=</mo><mfenced open="{" close=""><mrow><mtable rowspacing="4pt" columnspacing="1em"><mtr><mtd><mn>1</mn></mtd></mtr><mtr><mtd><mn>0</mn></mtd></mtr></mtable><mtable rowspacing="4pt" columnspacing="1em"><mtr><mtd><mrow><mrow><mrow><mi mathvariant="normal">if</mi><mtext /><mi mathvariant="normal">response</mi><mtext /><mi mathvariant="normal">to</mi><mtext /><mi mathvariant="normal">item</mi></mrow></mrow><mtext /><mi mathvariant="italic">i</mi><mrow><mrow><mtext /><mi mathvariant="normal">is</mi><mtext /><mi mathvariant="normal">in</mi><mtext /><mi mathvariant="normal">category</mi></mrow></mrow><mtext /><mi mathvariant="italic">j</mi></mrow></mtd></mtr><mtr><mtd><mrow><mrow><mrow><mi mathvariant="normal">otherwise</mi></mrow></mrow></mrow></mtd></mtr></mtable></mrow></mfenced></math> </ephtml> </p> <p>Then the model can be expressed as</p> <p>(<reflink idref="bib4" id="ref65">4</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">P</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>X</mi><mrow><mi mathvariant="italic">ij</mi></mrow></msub></mrow><mo>=</mo><mn>1</mn><mo>;</mo><mrow><mrow><mi mathvariant="bold">A</mi></mrow></mrow><mo>,</mo><mrow><msub><mrow><mrow><mi mathvariant="bold">B</mi></mrow></mrow><mrow><mrow><mi mathvariant="bold">i</mi></mrow></mrow></msub></mrow><mo>,</mo><mrow><msub><mi>ξ</mi><mi>i</mi></msub></mrow><mrow><mo>|</mo></mrow><mi mathvariant="italic">θ</mi></mrow></mfenced><mo>=</mo><mrow><mfrac><mrow><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><msubsup><mrow><mrow><mi mathvariant="bold">b</mi></mrow></mrow><mrow><mrow><mrow><mi mathvariant="bold">ij</mi></mrow></mrow></mrow><mrow><msup><mi /><mo>′</mo></msup></mrow></msubsup><mi mathvariant="italic">θ</mi><mo>+</mo><msubsup><mrow><mrow><mi mathvariant="bold">a</mi></mrow></mrow><mrow><mrow><mrow><mi mathvariant="bold">i</mi><mi mathvariant="bold">j</mi></mrow></mrow></mrow><mo>,</mo></msubsup><mrow><msub><mi>ξ</mi><mrow><mrow><mi mathvariant="bold">i</mi></mrow></mrow></msub></mrow></mrow></mfenced></mrow><mrow><msubsup><mrow><mo>∑</mo></mrow><mrow><mi mathvariant="italic">h</mi><mo>=</mo><mn>0</mn></mrow><mrow><mrow><msub><mi>K</mi><mi>i</mi></msub></mrow></mrow></msubsup><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><msubsup><mrow><mrow><mi mathvariant="bold">b</mi></mrow></mrow><mrow><mrow><mrow><mi mathvariant="bold">ih</mi></mrow></mrow></mrow><mrow><msup><mi /><mo>′</mo></msup></mrow></msubsup><mi mathvariant="italic">θ</mi><mo>+</mo><msubsup><mrow><mrow><mi mathvariant="bold">a</mi></mrow></mrow><mrow><mrow><mrow><mi mathvariant="bold">i</mi><mi mathvariant="bold">h</mi></mrow></mrow></mrow><mo>,</mo></msubsup><mrow><msub><mi>ξ</mi><mrow><mrow><mi mathvariant="bold">i</mi></mrow></mrow></msub></mrow></mrow></mfenced></mrow></mfrac></mrow></math> </ephtml> </p> <p>with <emph>b</emph><subs><emph>i0</emph></subs><emph>≡0</emph> and <emph>a</emph><subs><emph>i0</emph></subs><emph>≡0.</emph></p> <p>For a measurement consisting of <emph>N</emph> latent traits, <emph><bold>θ</bold></emph> = (<emph>θ</emph><subs><emph>1</emph></subs>, <emph>θ</emph><subs><emph>2</emph></subs>, <emph>..., θ</emph><subs><emph>N</emph></subs>), <bold><emph>ξ</emph></bold><subs><bold><emph>i</emph></bold></subs> is the item parameter vector that encompasses <emph>P</emph> elements. <bold>A</bold> is the design matrix and comprises <emph><bold>a</bold></emph><subs><emph><bold>ik</bold></emph></subs> that combines <emph><bold>ξ</bold></emph><subs><emph><bold>i</bold></emph></subs> for category <emph>k</emph>. For example, consider a 5-point item in the partial credit model (PCM), <emph>P</emph> = 4 and</p> <p>(<reflink idref="bib5" id="ref66">5</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">A</mi><mo>=</mo><mfenced open="[" close="]"><mrow><mtable rowspacing="4pt" columnspacing="1em"><mtr><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mrow><mo>−</mo><mn>1</mn></mrow></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mrow><mo>−</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>−</mo><mn>1</mn></mrow></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mrow><mo>−</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>−</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>−</mo><mn>1</mn></mrow></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mrow><mo>−</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>−</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>−</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>−</mo><mn>1</mn></mrow></mtd></mtr></mtable></mrow></mfenced><mo>.</mo></math> </ephtml> </p> <p> <bold>B</bold> <subs> <bold> <emph>i</emph> </bold> </subs> is the scoring matrix and consists of <emph><bold>b</bold></emph><subs><emph><bold>i0</bold></emph></subs>, <emph><bold>b</bold></emph><subs><emph><bold>i1</bold></emph></subs>, ... , <emph><bold>b</bold></emph><subs><emph><bold>id</bold></emph></subs>, which stands for the scoring weight for each dimension. For example, for a 5-point item in a 2-dimensional PCM,</p> <p>(<reflink idref="bib6" id="ref67">6</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>B</mi><mi>i</mi></msub></mrow><mo>=</mo><mfenced open="[" close="]"><mrow><mtable rowspacing="4pt" columnspacing="1em"><mtr><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mn>1</mn></mtd><mtd><mn>1</mn></mtd></mtr><mtr><mtd><mn>2</mn></mtd><mtd><mn>2</mn></mtd></mtr><mtr><mtd><mn>3</mn></mtd><mtd><mn>3</mn></mtd></mtr><mtr><mtd><mn>4</mn></mtd><mtd><mn>4</mn></mtd></mtr></mtable></mrow></mfenced><mo>.</mo></math> </ephtml> </p> <p>Note that we add an item subscript <emph>i</emph> denoting that scoring weights can vary across items, such as our data.</p> <p>Specifying different <bold>A</bold>, <bold>B</bold><subs><bold>i</bold></subs>, and <emph><bold>ξ</bold></emph><subs><emph><bold>i</bold></emph></subs> offers great flexibility in modeling the relationship between latent traits and each category. It not only can be used to formulate traditional between- and within-item PCMs, rating-scale models, and nominal response models but also can help build-up new models (Adams, Wilson, & Wang, [<reflink idref="bib2" id="ref68">2</reflink>]).</p> <p>For item parameters <emph><bold>ξ</bold></emph><subs><emph><bold>i</bold></emph></subs> and the design matrices with the structure of <bold>A</bold> (i.e., in Equation 5), our study retains the convention of the PCM, that is, there are K item parameters for each item, and <bold>A</bold> is used to accumulate item parameters for categories. In Equations (<reflink idref="bib7" id="ref69">7</reflink>) to (<reflink idref="bib19" id="ref70">19</reflink>), we only retain category subscripts for demonstration. <bold>B</bold><subs><bold>i</bold></subs>, the scoring matrix, is the focus of this study, which conveys our learning progression hypotheses into mathematical expressions. There are several possible scoring matrices for constructs that come together at the upper anchor depending on how we conceptualize the <emph>NS</emph> waypoint. Note that in Rasch family models, these parameters <emph><bold>ξ</bold></emph><subs><emph><bold>i</bold></emph></subs> are also sometimes referred to as "Andrich thresholds." We will instead follow the original usage by the developer of the PCM (Masters, [<reflink idref="bib15" id="ref71">15</reflink>]), and the continuing usage within the literature (e.g., Adams, Wilson, & Wang, [<reflink idref="bib2" id="ref72">2</reflink>]; Dray, Brown, Diakow, Lee, & Wilson, [<reflink idref="bib6" id="ref73">6</reflink>]; Wilson, [<reflink idref="bib30" id="ref74">30</reflink>]). In this study, we compare four potential MRCMLM models of dealing with dimensions coming together at the upper anchor, which differ in the specification of <bold>A</bold> and <bold>B</bold><subs><bold>i</bold></subs>.</p> <p>Table 1 demonstrates the scoring matrix for Model 1, which was employed in Metz et al. ([<reflink idref="bib17" id="ref75">17</reflink>]) and Cardace, Wilson, and Metz ([<reflink idref="bib5" id="ref76">5</reflink>]), where the meaning of <emph>NS</emph> waypoints varies across constructs. That is, the <emph>NS</emph> waypoint is treated as a higher waypoint than F4 in the <emph>fit</emph> construct items and a higher waypoint than P7 in the <emph>process</emph> construct items. As mentioned above, it is a between-item MIRT that is similar to Equation (<reflink idref="bib1" id="ref77">1</reflink>), and the within-item assumption is ignored. With <emph>θ</emph><subs><emph>f</emph></subs> = <emph>fit</emph>; <emph>θ</emph><subs><emph>p</emph></subs> = <emph>process</emph>; <emph>D</emph> denoting the sum of all possible nominators to normalize the equations; person and item subscripts being dropped for the sake of convenience, the probability of each waypoint of a <emph>fit</emph> item is:</p> <p>(<reflink idref="bib7" id="ref78">7</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">F</mi><mn>1</mn></mrow></mfenced><mo>=</mo><mn>1</mn><mrow><mo>/</mo></mrow><mi mathvariant="italic">D</mi><mtext mathcolor="red">\break</mtext><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">F</mi><mn>2</mn></mrow></mfenced><mo>=</mo><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>1</mn></msub></mrow></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">D</mi><mtext mathcolor="red">\break</mtext><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">F</mi><mn>3</mn></mrow></mfenced><mo>=</mo><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>1</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>2</mn></msub></mrow></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">D</mi><mtext mathcolor="red">\break</mtext><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">F</mi><mn>4</mn></mrow></mfenced><mo>=</mo><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>1</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>2</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>3</mn></msub></mrow></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">D</mi><mtext mathcolor="red">\break</mtext><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mspace width="negativethinmathspace" /><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mo>=</mo><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>1</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>2</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>3</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>4</mn></msub></mrow></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">D</mi><mo>.</mo></math> </ephtml> </p> <p>Table 1. Scoring matrix of model 1 that ignores the convergence.</p> <p> <ephtml> <table><thead><tr><td><italic>Fit</italic> items Waypoints</td><td>Scoring matrix</td><td><italic>Process</italic> items Waypoints</td><td>Scoring matrix</td></tr><tr><td><italic>Fit</italic></td><td><italic>Process</italic></td><td><italic>Fit</italic></td><td><italic>Process</italic></td></tr></thead><tbody><tr><td>F1</td><td>0</td><td>0</td><td>P1</td><td>0</td><td>0</td></tr><tr><td>F2</td><td>1</td><td>0</td><td>P2</td><td>0</td><td>1</td></tr><tr><td>F3</td><td>2</td><td>0</td><td>P3</td><td>0</td><td>2</td></tr><tr><td>F4</td><td>3</td><td>0</td><td>P4</td><td>0</td><td>3</td></tr><tr><td>NS</td><td>4</td><td>0</td><td>P5</td><td>0</td><td>4</td></tr><tr><td /><td /><td /><td>P6</td><td>0</td><td>5</td></tr><tr><td /><td /><td /><td>P7</td><td>0</td><td>6</td></tr><tr><td /><td /><td /><td>NS</td><td>0</td><td>7</td></tr></tbody></table> </ephtml> </p> <p>Thus,</p> <p>(<reflink idref="bib8" id="ref79">8</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">F</mi><mn>4</mn></mrow></mfenced><mo>=</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>4</mn></msub></mrow></math> </ephtml> </p> <p>In the same vein, the equations for probabilities of a <emph>process</emph> item are</p> <p>(<reflink idref="bib9" id="ref80">9</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mtable columnalign="left"><mtr><mtd><mi mathvariant="italic">f</mi><mrow><mo>(</mo><mrow><mi mathvariant="italic">P</mi><mn>1</mn></mrow><mo>)</mo></mrow><mo>=</mo><mn>1</mn><mo>/</mo><mi mathvariant="italic">D</mi></mtd></mtr><mtr><mtd><mi mathvariant="italic">f</mi><mrow><mo>(</mo><mrow><mi mathvariant="italic">P</mi><mn>2</mn></mrow><mo>)</mo></mrow><mo>=</mo><mi mathvariant="italic">exp</mi><mrow><mo>(</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>1</mn></msub></mrow><mo>)</mo></mrow><mo>/</mo><mi mathvariant="italic">D</mi></mtd></mtr><mtr><mtd><mi mathvariant="italic">f</mi><mrow><mo>(</mo><mrow><mi mathvariant="italic">P</mi><mn>3</mn></mrow><mo>)</mo></mrow><mo>=</mo><mi mathvariant="italic">exp</mi><mrow><mo>(</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>1</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>2</mn></msub></mrow><mo>)</mo></mrow><mo>/</mo><mi mathvariant="italic">D</mi></mtd></mtr><mtr><mtd><mi mathvariant="italic">f</mi><mrow><mo>(</mo><mrow><mi mathvariant="italic">P</mi><mn>4</mn></mrow><mo>)</mo></mrow><mo>=</mo><mi mathvariant="italic">exp</mi><mrow><mo>(</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>1</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>2</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>3</mn></msub></mrow><mo>)</mo></mrow><mo>/</mo><mi mathvariant="italic">D</mi></mtd></mtr><mtr><mtd><mi mathvariant="italic">f</mi><mrow><mo>(</mo><mrow><mi mathvariant="italic">P</mi><mn>5</mn></mrow><mo>)</mo></mrow><mo>=</mo><mi mathvariant="italic">exp</mi><mrow><mo>(</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>1</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>2</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>3</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>4</mn></msub></mrow><mo>)</mo></mrow><mo>/</mo><mi mathvariant="italic">D</mi></mtd></mtr><mtr><mtd><mi mathvariant="italic">f</mi><mrow><mo>(</mo><mrow><mi mathvariant="italic">P</mi><mn>6</mn></mrow><mo>)</mo></mrow><mo>=</mo><mi mathvariant="italic">exp</mi><mrow><mo>(</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>1</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>2</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>3</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>4</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>5</mn></msub></mrow><mo>)</mo></mrow><mo>/</mo><mi mathvariant="italic">D</mi></mtd></mtr><mtr><mtd><mi mathvariant="italic">f</mi><mrow><mo>(</mo><mrow><mi mathvariant="italic">P</mi><mn>7</mn></mrow><mo>)</mo></mrow><mo>=</mo><mi mathvariant="italic">exp</mi><mrow><mo>(</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>1</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>2</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>3</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>4</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>5</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>6</mn></msub></mrow><mo>)</mo></mrow><mo>/</mo><mi mathvariant="italic">D</mi></mtd></mtr><mtr><mtd><mi mathvariant="italic">f</mi><mo stretchy="false">(</mo><mi mathvariant="italic">NS</mi><mo stretchy="false">)</mo><mo>=</mo><mi mathvariant="italic">exp</mi><mrow><mo>(</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>1</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>2</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>3</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>4</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>5</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>6</mn></msub><mo>+</mo><msub><mi>θ</mi><mi>p</mi></msub><mo>−</mo><msub><mi>ξ</mi><mn>7</mn></msub></mrow><mo>)</mo></mrow><mo>/</mo><mi mathvariant="italic">D</mi></mtd></mtr></mtable></math> </ephtml> </p> <p>thus,</p> <p>(<reflink idref="bib10" id="ref81">10</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">P</mi><mn>7</mn></mrow></mfenced><mo>=</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>7</mn></msub></mrow><mo>.</mo></math> </ephtml> </p> <p>Model 1 does not accurately reflect the assumptions in the learning progression in Figure 3 because in the Figure <emph>θ</emph><subs><emph>f</emph></subs> and <emph>θ</emph><subs><emph>p</emph></subs> are intertwined at the <emph>NS</emph> waypoint, but in Model 1 what dominates <emph>f(NS)/f(F4)</emph> is <emph>θ</emph><subs><emph>f</emph></subs> only, and what dominates <emph>f(NS)/f(P7)</emph> is <emph>θ</emph><subs><emph>p</emph></subs> only. Thus, if the learning progression is accurate, then this might lead to poor estimation of the step difficulties for the <emph>NS</emph> waypoint. The problem will be more severe if the two dimensions have lower correlation, but earlier work shows a correlation of 0.71 between <emph>fit</emph> and <emph>process</emph> (Cardace, Wilson, & Metz, [<reflink idref="bib5" id="ref82">5</reflink>]).</p> <p>Table 2 shows the scoring matrix for Model 2, where the within-category multidimensionality is realized at the <emph>NS</emph> waypoint. It closely reflects the construct map. Note that Model 2 cannot be simply ascribed to within-item multidimensional model as only the <emph>NS</emph> waypoints involve two dimensions, but other categories only assess a single dimension. The probabilities follow Equation (<reflink idref="bib7" id="ref83">7</reflink>), except for the <emph>NS</emph> waypoint, whose function in a <emph>fit</emph> item is</p> <p>(<reflink idref="bib11" id="ref84">11</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mo>=</mo><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>1</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>2</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>3</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>4</mn></msub></mrow></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">D</mi><mo>.</mo></math> </ephtml> </p> <p>Table 2. Scoring matrix of model 2 that recognizes the convergence of the dimensions.</p> <p> <ephtml> <table><thead><tr><td><italic>Fit</italic> items Waypoints</td><td>Scoring matrix</td><td><italic>Process</italic> items Waypoints</td><td>Scoring matrix</td></tr><tr><td><italic>Fit</italic></td><td><italic>Process</italic></td><td><italic>Fit</italic></td><td><italic>Process</italic></td></tr></thead><tbody><tr><td>F1</td><td>0</td><td>0</td><td>P1</td><td>0</td><td>0</td></tr><tr><td>F2</td><td>1</td><td>0</td><td>P2</td><td>0</td><td>1</td></tr><tr><td>F3</td><td>2</td><td>0</td><td>P3</td><td>0</td><td>2</td></tr><tr><td>F4</td><td>3</td><td>0</td><td>P4</td><td>0</td><td>3</td></tr><tr><td>NS</td><td>3</td><td>1</td><td>P5</td><td>0</td><td>4</td></tr><tr><td /><td /><td /><td>P6</td><td>0</td><td>5</td></tr><tr><td /><td /><td /><td>P7</td><td>0</td><td>6</td></tr><tr><td /><td /><td /><td>NS</td><td>1</td><td>6</td></tr></tbody></table> </ephtml> </p> <p>thus,</p> <p>(<reflink idref="bib12" id="ref85">12</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">F</mi><mn>4</mn></mrow></mfenced><mo>=</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>4</mn></msub></mrow><mo>,</mo></math> </ephtml> </p> <p> <emph>ξ</emph> <subs> <emph>4</emph> </subs> is the step difficulty for P7 given that F4 is attained, so it is not as same as in the <emph>process</emph> dimension. Therefore, we need to align dimensions so comparability can be established across dimensions, and step parameters for the <emph>NS</emph> waypoint can be easily interpreted under the four building blocks. The probabilities follow Equation (<reflink idref="bib9" id="ref86">9</reflink>), except for the <emph>NS</emph> waypoint:</p> <p>(<reflink idref="bib13" id="ref87">13</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mo>=</mo><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>1</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>2</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>3</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>4</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>5</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>6</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>7</mn></msub></mrow></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">D</mi></math> </ephtml> </p> <p>thus,</p> <p>(<reflink idref="bib14" id="ref88">14</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">P</mi><mn>7</mn></mrow></mfenced><mo>=</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>7</mn></msub></mrow><mo>.</mo></math> </ephtml> </p> <p>Table 3 illustrates the scoring matrix for Model 3, where a new dimension named <emph>NS</emph> is hypothesized and only influences the <emph>NS</emph> waypoint. It represents another kind of combination of both dimensions. Equations for F1 to F4 and P1 to P7 remain the same as in Equations (<reflink idref="bib7" id="ref89">7</reflink>) and (<reflink idref="bib9" id="ref90">9</reflink>), except the <emph>NS</emph> waypoint in the <emph>fit</emph> items takes the form</p> <p>(<reflink idref="bib15" id="ref91">15</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mo>=</mo><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>1</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>2</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>f</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>3</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">NS</mi></mrow></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>4</mn></msub></mrow></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">D</mi></math> </ephtml> </p> <p>Table 3. Scoring matrix of model 3 that treats convergence as a new dimension.</p> <p> <ephtml> <table><thead><tr><td><italic>Fit</italic> item Waypoints</td><td>Scoring matrix</td><td><italic>Process</italic> item Waypoints</td><td>Scoring matrix</td></tr><tr><td><italic>Fit</italic></td><td><italic>Process</italic></td><td><italic>NS</italic></td><td><italic>Fit</italic></td><td><italic>Process</italic></td><td><italic>NS</italic></td></tr></thead><tbody><tr><td>F1</td><td>0</td><td>0</td><td>0</td><td>P1</td><td>0</td><td>0</td><td>0</td></tr><tr><td>F2</td><td>1</td><td>0</td><td>0</td><td>P2</td><td>0</td><td>1</td><td>0</td></tr><tr><td>F3</td><td>2</td><td>0</td><td>0</td><td>P3</td><td>0</td><td>2</td><td>0</td></tr><tr><td>F4</td><td>3</td><td>0</td><td>0</td><td>P4</td><td>0</td><td>3</td><td>0</td></tr><tr><td>NS</td><td>0</td><td>0</td><td>1</td><td>P5</td><td>0</td><td>4</td><td>0</td></tr><tr><td /><td /><td /><td /><td>P6</td><td>0</td><td>5</td><td>0</td></tr><tr><td /><td /><td /><td /><td>P7</td><td>0</td><td>6</td><td>0</td></tr><tr><td /><td /><td /><td /><td>NS</td><td>0</td><td>0</td><td>1</td></tr></tbody></table> </ephtml> </p> <p>thus,</p> <p>(<reflink idref="bib16" id="ref92">16</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">F</mi><mn>4</mn></mrow></mfenced><mo>=</mo><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">NS</mi></mrow></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>4</mn></msub></mrow><mo>.</mo></math> </ephtml> </p> <p>Likewise, the <emph>NS</emph> waypoint in the <emph>process</emph> item is given as</p> <p>(<reflink idref="bib17" id="ref93">17</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mo>=</mo><mi mathvariant="italic">exp</mi><mfenced open="(" close=")"><mrow><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>1</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>2</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>3</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>4</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>5</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mi>p</mi></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>6</mn></msub></mrow><mo>+</mo><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">NS</mi></mrow></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>7</mn></msub></mrow></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">D</mi><mo>,</mo></math> </ephtml> </p> <p>(<reflink idref="bib18" id="ref94">18</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">P</mi><mn>7</mn></mrow></mfenced><mo>=</mo><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">NS</mi></mrow></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>7</mn></msub></mrow><mo>.</mo></math> </ephtml> </p> <p>Inspired by the between-item MIRT of Model 1 and the additional <emph>NS</emph> dimension of Model 3, we designed the Model 4 to separate the within-category multidimensionality. First, we created five binary <emph>NS</emph> items based on the five original items with <emph>NS</emph> responses, that is, 1 for <emph>NS</emph> response and 0 for non-NS response. Then, we recoded the <emph>NS</emph> waypoint to F4 in the <emph>fit</emph> dimension and to P7 in the process dimension. Thus, there are now 22 items with a between-item 3-dimensional structure. The scoring matrix is given in Table 4 and equations for F1 to F4 and P1 to P7 remain the same as in Equations 7 and 9, but the equation for the probability of <emph>NS</emph> is</p> <p>(<reflink idref="bib19" id="ref95">19</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NS</mi></mrow></mfenced><mrow><mo>/</mo></mrow><mi mathvariant="italic">f</mi><mfenced open="(" close=")"><mrow><mi mathvariant="italic">NonNS</mi></mrow></mfenced><mo>=</mo><mrow><msub><mi>θ</mi><mrow><mi mathvariant="italic">NS</mi></mrow></msub></mrow><mo>−</mo><mrow><msub><mi>ξ</mi><mn>1</mn></msub></mrow><mo>.</mo></math> </ephtml> </p> <p>Table 4. Scoring matrix of model 4 that separates the within-category multidimensionality and treats convergence as a new dimension.</p> <p> <ephtml> <table><thead><tr><td><italic>Fit</italic> items Waypoints</td><td>Scoring matrix</td><td><italic>Process</italic> items Waypoints</td><td>Scoring matrix</td><td><italic>NS</italic> items Waypoints</td><td>Scoring matrix</td></tr><tr><td><italic>Fit</italic></td><td><italic>Process</italic></td><td><italic>NS</italic></td><td><italic>Fit</italic></td><td><italic>Process</italic></td><td><italic>NS</italic></td><td><italic>Fit</italic></td><td><italic>Process</italic></td><td><italic>NS</italic></td></tr></thead><tbody><tr><td>F1</td><td>0</td><td>0</td><td>0</td><td>P1</td><td>0</td><td>0</td><td>0</td><td>Non-NS</td><td>0</td><td>0</td><td>0</td></tr><tr><td>F2</td><td>1</td><td>0</td><td>0</td><td>P2</td><td>0</td><td>1</td><td>0</td><td>NS</td><td>0</td><td>0</td><td>1</td></tr><tr><td>F3</td><td>2</td><td>0</td><td>0</td><td>P3</td><td>0</td><td>2</td><td>0</td><td /><td /><td /><td /></tr><tr><td>F4</td><td>3</td><td>0</td><td>0</td><td>P4</td><td>0</td><td>3</td><td>0</td><td /><td /><td /><td /></tr><tr><td /><td /><td /><td /><td>P5</td><td>0</td><td>4</td><td>0</td><td /><td /><td /><td /></tr><tr><td /><td /><td /><td /><td>P6</td><td>0</td><td>5</td><td>0</td><td /><td /><td /><td /></tr><tr><td /><td /><td /><td /><td>P7</td><td>0</td><td>6</td><td>0</td><td /><td /><td /><td /></tr></tbody></table> </ephtml> </p> <p>The four models are useful in different situations. First, if the two dimensions are highly correlated, Model 1 is not very problematic, as the two dimensions are approximately "exchangeable." Second, if there is a low correlation between the two dimensions, Model 1 might be problematic and we should consider Models 2, 3, and 4. Here, Model 2 closely reflects the construct map in Figure 3. It seems more suitable for our learning progression example, which maps out how F4 or P7 progresses into the <emph>NS</emph> upper anchor. Third, Model 3 seems more appropriate than Model 1 when (a) there are interactions between the two dimensions which are not captured in the sum, and (b) the two dimensions merge into a new dimension. Fourth, as mentioned above, the interpretation of Wright maps for within-item models can be difficult. Hence, Model 4 is advantageous in that it separates the within-category multidimensionality so that a between-item MIRT model can be used. On the other hand, a potential issue is that as a learning progression project, we care more about how students grow step by step while a transition from non-<emph>NS</emph> understanding to <emph>NS</emph> understanding might not be fine-grained enough. Finally, it is worth noting that there may often be relatively fewer responses at the highest waypoint than waypoints below, and thus separation of the <emph>NS</emph> waypoint or introduction of <emph>NS</emph> dimensions will likely result in estimates with large standard errors.</p> <hd id="AN0175570131-9">5.3. Scale Alignment Method</hd> <p>The scale alignment method was used in this study to establish comparability across dimensions and render the <emph>NS</emph> waypoints more interpretable under the four-building-block framework. It enforces the unique ordering of items across dimensions by linearly transforming item parameters within dimensions (Feuerstahler & Wilson, [<reflink idref="bib8" id="ref96">8</reflink>], [<reflink idref="bib9" id="ref97">9</reflink>]). That is, a dimension is treated as the reference, and item parameters of other dimensions are projected onto it. Specifically, the delta dimensional alignment (DDA) method is employed, which requires the calculation of slopes and intercepts for linear transformation (Feuerstahler & Wilson, [<reflink idref="bib9" id="ref98">9</reflink>]). Let <emph>sd</emph> and <emph>mn</emph> denote the standard deviation and mean, respectively, and dimension 1 be the reference dimension.</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover><mrow><mrow><mi mathvariant="italic">η</mi></mrow></mrow><mo>ˆ</mo></mover><mrow><mrow><mrow><mrow><mi mathvariant="italic">Ud</mi></mrow></mrow></mrow></mrow></msub></mrow></math> </ephtml> represents vector collecting item parameters (<emph>ξ</emph>) of dimension <emph>d</emph> obtained from unidimensional IRT model and</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover><mrow><mrow><mi mathvariant="italic">η</mi></mrow></mrow><mo>ˆ</mo></mover><mrow><mrow><mrow><mrow><mi mathvariant="italic">Md</mi></mrow></mrow></mrow></mrow></msub></mrow></math> </ephtml> stands for the item parameter vectors from MIRT model (i.e., Model 1 to 4). Equations for slopes take the form</p> <p>(<reflink idref="bib20" id="ref99">20</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover><mi>S</mi><mo stretchy="false">ˆ</mo></mover><mi mathvariant="italic">d</mi></msub></mrow><mo>=</mo><mrow><mfrac><mrow><mi>s</mi><mi>d</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">Ud</mi></mrow></msub></mrow></mrow></mfenced><mi mathvariant="italic">sd</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">M</mi><mn>1</mn></mrow></msub></mrow></mrow></mfenced></mrow><mrow><mi mathvariant="italic">sd</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">Md</mi></mrow></msub></mrow></mrow></mfenced><mi mathvariant="italic">sd</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">U</mi><mn>1</mn></mrow></msub></mrow></mrow></mfenced></mrow></mfrac></mrow></math> </ephtml> </p> <p>and equations for intercepts are</p> <p>(<reflink idref="bib21" id="ref100">21</reflink>)</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover><mi>i</mi><mo stretchy="false">ˆ</mo></mover><mi mathvariant="italic">d</mi></msub></mrow><mo>=</mo><mrow><mfrac><mrow><mi>m</mi><mi>n</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">ud</mi></mrow></msub></mrow></mrow></mfenced><mo>−</mo><mi mathvariant="italic">mn</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">U</mi><mn>1</mn></mrow></msub></mrow></mrow></mfenced><mo>+</mo><mrow><mfrac><mrow><mi>s</mi><mi>d</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">U</mi><mn>1</mn></mrow></msub></mrow></mrow></mfenced></mrow><mrow><mi mathvariant="italic">sd</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">M</mi><mn>1</mn></mrow></msub></mrow></mrow></mfenced></mrow></mfrac></mrow><mi mathvariant="italic">mn</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">M</mi><mn>1</mn></mrow></msub></mrow></mrow></mfenced><mo>−</mo><mrow><mfrac><mrow><mi>s</mi><mi>d</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">Ud</mi></mrow></msub></mrow></mrow></mfenced></mrow><mrow><mi mathvariant="italic">sd</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">Md</mi></mrow></msub></mrow></mrow></mfenced></mrow></mfrac></mrow><mi mathvariant="italic">mn</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">Md</mi></mrow></msub></mrow></mrow></mfenced></mrow><mrow><mrow><mfrac><mrow><mi>s</mi><mi>d</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">U</mi><mn>1</mn></mrow></msub></mrow></mrow></mfenced></mrow><mrow><mi mathvariant="italic">sd</mi><mfenced open="(" close=")"><mrow><mrow><msub><mrow><mover><mi>η</mi><mo stretchy="false">ˆ</mo></mover></mrow><mrow><mi mathvariant="italic">M</mi><mn>1</mn></mrow></msub></mrow></mrow></mfenced></mrow></mfrac></mrow></mrow></mfrac></mrow></math> </ephtml> </p> <p>Scale alignment methods require a between-item MIRT model in calculating</p> <p>Graph</p> <p> <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mover><mrow><mrow><mi mathvariant="italic">η</mi></mrow></mrow><mo>ˆ</mo></mover><mrow><mrow><mrow><mrow><mi mathvariant="italic">Md</mi></mrow></mrow></mrow></mrow></msub></mrow></math> </ephtml> , and only Models 1 and 4 satisfy this requirement. To address this issue, we temporarily ascribe parameters for <emph>NS</emph> waypoint in <emph>fit</emph> items to the <emph>fit</emph> dimension and parameters for <emph>NS</emph> waypoint in <emph>process</emph> items to the <emph>process</emph> dimension. The logics underlying this procedure are (a) when calibrating <emph>NS</emph> waypoint parameters in MRCMLM with scoring matrix in Tables 2 and 3, parameters for <emph>NS</emph> waypoint largely fall into the dimension that the item belongs to; and (b) after alignment, item parameters were cast into the reference dimension then it does not matter too much which dimensions item parameter belongs to.</p> <p>This research takes the <emph>process</emph> dimension as the reference because it has a larger item set than the <emph>fit</emph> dimension. Therefore, after scale alignment, all item parameters, including those for the <emph>NS</emph> waypoint, will be projected into the <emph>process</emph> dimension.</p> <hd id="AN0175570131-10">6. Results</hd> <p></p> <hd id="AN0175570131-11">6.1. Model Fits</hd> <p>Table 5 shows fit statistics for Models 1, 2, and 3. The results of Model 4 are not shown in the Table because it is not comparable to other models due to the changes in responses. Compared with Model 1, the better fits of Models 2 suggest that including additional dimensions at the upper anchor helps explain the data. Similarly, incorporating a dimension that integrates <emph>fit</emph> and <emph>process</emph> can improve the model fits as evidenced by comparison between Models 1 and 3. Moreover, comparing Model 2 to 3 in terms of AIC and BIC, even though the introduction of a new <emph>NS</emph> dimension decreases the log-likelihood, model complexity increases. That is, a simple sum of the two dimensions at the top leads to a more parsimonious model than Model 3 and provides better explanatory power than Model 1, which serves as evidence to support the construct maps that come together at the upper anchor. Thus, this comparison supports Model 2 as the best fit model among the four.</p> <p>Table 5. Model fits for model 1, 2, and 3.</p> <p> <ephtml> <table><thead><tr><td /><td>Log likelihood</td><td># of pars</td><td>AIC</td><td>BIC</td></tr></thead><tbody><tr><td>Model 1</td><td>−2798.002</td><td>76</td><td>5748.005</td><td>6013.164</td></tr><tr><td>Model 2</td><td>−2793.992</td><td>76</td><td>5739.984</td><td>6005.143</td></tr><tr><td>Model 3</td><td>−2792.218</td><td>79</td><td>5742.435</td><td>6018.061</td></tr></tbody></table> </ephtml> </p> <hd id="AN0175570131-12">6.2. Step Difficulty of the NS Waypoint After Alignment</hd> <p>Step difficulties (i.e., <emph><bold>ξ</bold></emph><subs><emph>i</emph></subs>) are direct item parameters estimates from the four MRCMLMs. In comparison to Model 1, Model 2 leads to an increase in the step difficulty for the <emph>NS</emph> waypoint of items belonging to the <emph>fit</emph> dimension and a decrease for items of <emph>process</emph> dimension (see Table 6). This seems to mean that the <emph>NS</emph> waypoint requires more understanding of <emph>process</emph> than <emph>fit</emph>. Furthermore, step difficulties of Model 3 suggest that it is relatively more difficult to obtain <emph>NS</emph> responses given that F4 is known than given that P7 is known. Model 4 produces the largest <emph>NS</emph> step difficulties because unlike Models 1, 2, and 3, where <emph>NS</emph> step difficulties are between F4 and P7, Model 4 is between the <emph>NS</emph> and all non-<emph>NS</emph> waypoints. To conclude, it is difficult to progress from not knowing <emph>NS</emph> to knowing <emph>NS</emph>, and understanding F4 or P7 can make it easier to handle <emph>NS</emph>, which is especially true for P7.</p> <p>Table 6. Step difficulties of the <emph>NS</emph> waypoints after alignment.</p> <p> <ephtml> <table><thead><tr><td>Dimension</td><td>Item</td><td>Model 1</td><td>Model 2</td><td>Model 3</td><td>Model 4</td></tr></thead><tbody><tr><td>Fit</td><td>CD1</td><td>1.842</td><td>1.866</td><td>2.790</td><td>4.056</td></tr><tr><td>DS</td><td>2.730</td><td>2.764</td><td>3.812</td><td>4.147</td></tr><tr><td>Process</td><td>CD2</td><td>.117</td><td>−.283</td><td>−2.223</td><td>3.696</td></tr><tr><td>GP1</td><td>1.817</td><td>1.426</td><td>−.519</td><td>6.299</td></tr><tr><td>GP3</td><td>3.888</td><td>3.537</td><td>2.236</td><td>6.296</td></tr></tbody></table> </ephtml> </p> <hd id="AN0175570131-13">6.3. Correlation Between Dimensions and Models</hd> <p>Table 7 presents correlation coefficients among latent traits obtained via each model. Clearly, the hypothesized <emph>NS</emph> dimension has higher correlations with <emph>process</emph> than <emph>fit</emph>. In terms of <emph>fit</emph> and <emph>process</emph>, their association declined in Model 3, especially in comparison to Model 1. That is because Model 1 falsely assumes that <emph>NS</emph> waypoints only assess the understanding of <emph>fit</emph> in <emph>fit</emph> items and the understanding of the <emph>process</emph> in <emph>process</emph> items, which lead to contaminated latent trait estimates. In Figure 3, the estimated latent trait of <emph>fit</emph> in Model 1 is not purely about the construct of <emph>fit</emph> but also the construct of <emph>process</emph>, and similarly for the estimates of the <emph>process</emph>. Therefore, we might consider the correlation coefficient in Model 1 to be an inflated estimate. The correlation coefficients between <emph>fit</emph> and <emph>process</emph> for Models 3 and 4 are closer to the corresponding coefficient for Model 2 than that for Model 3. This leads to a preference for Model 2.</p> <p>Table 7. Correlations among different dimensions.</p> <p> <ephtml> <table><thead><tr><td /><td><italic>Fit</italic> & <italic>Process</italic></td><td><italic>Fit</italic> & <italic>NS</italic></td><td><italic>Process</italic> & <italic>NS</italic></td></tr></thead><tbody><tr><td>Model 1</td><td>.759</td><td /><td /></tr><tr><td>Model 2</td><td>.706</td><td /><td /></tr><tr><td>Model 3</td><td>.661</td><td>.518</td><td>.908</td></tr><tr><td>Model 4</td><td>.718</td><td>.701</td><td>.938</td></tr></tbody></table> </ephtml> </p> <p>Tables 8 to 9 show the correlation coefficients of latent trait estimates between the four models for <emph>fit</emph>, <emph>process</emph>, and the additional <emph>NS</emph> dimension, respectively. They are very highly correlated. There is also a high correlation between the hypothesized <emph>NS</emph> dimension in Models 3 and 4 (i.e., 0.993). Thus, regardless of the findings in previous results, it may not matter much which model to choose in terms of latent trait estimates.</p> <p>Table 8. Correlation among models within <emph>fit</emph> dimension.</p> <p> <ephtml> <table><thead><tr><td /><td>Model 1</td><td>Model 2</td><td>Model 3</td><td>Model 4</td></tr></thead><tbody><tr><td>Model 1</td><td /><td /><td /><td /></tr><tr><td>Model 2</td><td>.998</td><td /><td /><td /></tr><tr><td>Model 3</td><td>.964</td><td>.971</td><td /><td /></tr><tr><td>Model 4</td><td>.999</td><td>.999</td><td>.969</td><td /></tr></tbody></table> </ephtml> </p> <p>Table 9. Correlation among models within <emph>process</emph> dimension.</p> <p> <ephtml> <table><thead><tr><td /><td>Model 1</td><td>Model 2</td><td>Model 3</td><td>Model 4</td></tr></thead><tbody><tr><td>Model 1</td><td /><td /><td /><td /></tr><tr><td>Model 2</td><td>.999</td><td /><td /><td /></tr><tr><td>Model 3</td><td>.997</td><td>.997</td><td /><td /></tr><tr><td>Model 4</td><td>.998</td><td>.998</td><td>.994</td><td /></tr></tbody></table> </ephtml> </p> <hd id="AN0175570131-14">6.4. Wright Maps</hd> <p>Figures 4-7 demonstrate the Wright maps for Model 1 to 4, respectively, whose Y-axes extend from −2.5 logits to 6 logits, to provide a comparable visualization. On these Wright maps, the items are represented by their Thurstonian thresholds (Adams et al., [<reflink idref="bib1" id="ref101">1</reflink>]), which are interpretative cut-points defined as the location where respondents have equal probability in choosing successive categories (Linacre & Wright, [<reflink idref="bib14" id="ref102">14</reflink>]). These are calculated based on the estimated step parameter estimates following the procedures described in Adams et al. ([<reflink idref="bib1" id="ref103">1</reflink>]).</p> <p>Graph: Figure 4. The Wright map, showing thurstone thresholds, for Model 1 that ignores the convergence at the NS waypoint.</p> <p>Graph: Figure 5. The Wright map, showing thurstone thresholds, for Model 2 that recognizes the convergence at the NS waypoint.</p> <p>Graph: Figure 6. The Wright map, showing thurstone thresholds, for Model 3 that treats convergence as a new dimension.</p> <p>MAP: Figure 7. The Wright map of Model 4 that separates the within-category multidimensionality and treats convergence as a new dimension.</p> <p>Looking at these Figures, one can see that as expected, P7 is generally higher than F4. Nevertheless, there are some "messy middles" as often seen in learning progressions (Sikorski, [<reflink idref="bib22" id="ref104">22</reflink>]). Note that, the meaning of <emph>NS</emph> thresholds differs across figures: (a) in Figure 4, <emph>NS</emph> in the <emph>fit</emph> dimension is the transition from F4 to F4+P7, where the <emph>process</emph> dimension is projected onto the <emph>fit</emph> dimension, and <emph>NS</emph> in the <emph>process</emph> dimension is the transition from P7 to F4+P7, where the <emph>fit</emph> dimension is projected onto the <emph>process</emph> dimension; (b) in Figure 5, <emph>NS</emph> in the <emph>fit</emph> dimension is the transition from F4 to F4+P7, and <emph>NS</emph> in the <emph>process</emph> dimension is the transition from P7 to F4+P7; (c) in Figure 6, <emph>NS</emph> thresholds represent transitions from the compromise of F4 and P7 to F4+P7; and (d) in Figure 7, <emph>NS</emph> thresholds are transition from all non-<emph>NS</emph> waypoints to <emph>NS</emph> waypoints.</p> <p>Figure 4 is a replication of Metz et al. ([<reflink idref="bib17" id="ref105">17</reflink>]) except an item is dropped due to misfit. As a Wright map of between-item MIRT analysis, it failed to recognize the <emph>process</emph> dimension at the <emph>NS</emph> waypoint of items pertaining to <emph>fit</emph> items and the <emph>fit</emph> dimension at the <emph>NS</emph> waypoint of items pertaining to <emph>process</emph> dimension. On the other hand, Figure 5 is closest to the assumptions of the construct map in Figure 3. After scale alignment, a small difference was found between the corresponding <emph>NS</emph> thresholds of Model 1 and 2. That is largely because a high correlation exists between <emph>fit</emph> and <emph>process</emph> dimension, buffering the inappropriate setting of Model 1.</p> <p>In Figures 6 and 7, <emph>NS</emph> thresholds are closer to P7 than F4, which suggests that <emph>NS</emph> thresholds are more dominated by the <emph>process</emph> dimension, which is consistent with the result that the hypothesized <emph>NS</emph> dimension is more closely related to the process dimension than the fit dimension in Table 7. That is, when responding to an item, students with P7 responses to the <emph>process</emph> dimensions are likely to give F4 responses to the <emph>fit</emph> dimension (combined to be <emph>NS</emph> responses), but not vice versa.</p> <hd id="AN0175570131-15">7. Discussion</hd> <p>Upper anchors are the highest (i.e., most sophisticated) waypoints in learning progressions and contain more complexity and abstractness than the waypoints below (Duschl, Maeng, & Sezen, [<reflink idref="bib7" id="ref106">7</reflink>]; Sikorski, [<reflink idref="bib22" id="ref107">22</reflink>]). This study concentrates on comparing and interpreting different ways of modeling constructs that come together at the upper anchor via MRCMLMs and scale alignment methods based on a learning progression assessment of micro-evolution. Four models, varying on how to treat the <emph>NS</emph> waypoint (the upper anchor of our exemplar learning progression), are considered to address this complexity, that is, ignoring the convergence, recognizing the convergence of the two dimensions, hypothesizing a new <emph>NS</emph> dimension, and separating the within-category multidimensionality and then hypothesizing a new <emph>NS</emph> dimension.</p> <p>First, we compared model fits across models and found that the inclusion of both dimensions or the introduction of an additional <emph>NS</emph> dimension at the <emph>NS</emph> waypoint can improve fit indices. This comparison is a meaningful step to investigate the usefulness of an additional dimension at the upper anchor. Second, we compared the step difficulties for <emph>NS</emph> given F4 or P7, which echoed the equations about <emph>NS</emph> waypoints in the method section. This helps us investigate what construct dominates more on the <emph>NS</emph> waypoint and the relationship between F4 and P7. In our example, the comparison of Model 1, 2, and 3 suggests the transition from P7 to <emph>NS</emph> is easier than F4 to <emph>NS</emph>. Third, scale alignment methods serve as a useful tool in addressing the between-item issue for interpreting Wright maps (in particular for the <emph>NS</emph> waypoints in Models 2 and 3). This resolves the issue of having more than a single latent trait for the item by projecting all the item parameters onto the reference dimension and rendering them comparable. Fourth, Wright maps, important outputs of the fourth building block, were shown for each model. They are a visualization of the empirical findings. The Thurstonian thresholds across items show the <emph>NS</emph> thresholds are generally located nearer to P7 thresholds than F4 thresholds. However, as noted above, the meaning of <emph>NS</emph> thresholds varies across models. In our learning progression example, the Wright map of Model 2 in Figure 6 is more desirable, as it reflects the convergence of the <emph>NS</emph> waypoint, whose threshold is connected to the F4 and P7 categories, and thus supports the curriculum. Specifically, to facilitate an <emph>NS</emph> understanding, P7 would be more challenging than F4 and should be given more attention in teaching the upper anchor.</p> <p>However, if constructs merge into a new one at the upper anchor, Model 3 and 4 will be more desirable. From a learning progression point of view, Model 3 is better than Model 4 because Model 3 illustrates how students progress from an easy construct to a difficult construct and Model 4 is modeling the change from lack of understanding of the construct to understanding of the highest waypoint of the construct.</p> <p>Finally, this study calls for more subtle MIRT models than simple within- and between-item MIRT models, particularly, for the upper anchor in learning progressions, which often involve more sophisticated interplay between various dimensions. As for the construct map in our motivating example, the multidimensionality occurs between items and at the <emph>NS</emph> upper anchor, and neither within- nor between-item MIRT models truly reflect the construct map. Therefore, a category or waypoint consideration would be beneficial in modeling responses. It also affirms the importance of the reflection between the construct map and measurement models.</p> <hd id="AN0175570131-16">8. Limitations and Future Directions</hd> <p>As the <emph>NS</emph> waypoint is the top performance and interviews limit our number of participants, we observed very few <emph>NS</emph> responses. Larger differences across models would be found if there were more data. Moreover, the learning progression in our motivation example ends at the <emph>NS</emph> waypoint because it focuses on young students. It would be interesting to know how students progress after the <emph>NS</emph> waypoint and whether we should treat <emph>NS</emph> as a new dimension after that.</p> <hd id="AN0175570131-17">Disclosure statement</hd> <p>We have no known conflicts of interest to disclose.</p> <ref id="AN0175570131-18"> <title> References </title> <blist> <bibl id="bib1" idref="ref38" type="bt">1</bibl> <bibtext> Adams, R. J., Cloney, D., Wu, M., Osses, A., Schwantner, V., & Vista, A. (2022). ACER conquest manual. Australia : Australian Council for Educational Research. https://conquestmanual.acer.org/</bibtext> </blist> <blist> <bibl id="bib2" idref="ref9" type="bt">2</bibl> <bibtext> Adams, R., Wilson, M., & Wang, W. C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21 (1), 1 – 23. doi: 10.1177/0146621697211001</bibtext> </blist> <blist> <bibl id="bib3" idref="ref28" type="bt">3</bibl> <bibtext> Böckenholt, U., & Meiser, T. (2017). Response style analysis with threshold and multi‐process IRT models: A review and tutorial. British Journal of Mathematical and Statistical Psychology, 70 (1), 159 – 181. doi: 10.1111/bmsp.12086</bibtext> </blist> <blist> <bibl id="bib4" idref="ref45" type="bt">4</bibl> <bibtext> Cardace, A. (2018). The Affordances of Item Response Theory: The Case of a Complex Learning Progression about Evolution [ Doctoral dissertation ]. University of California : Berkeley.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref19" type="bt">5</bibl> <bibtext> Cardace, A., Wilson, M., & Metz, K. E. (2021). Designing a learning progression about micro-evolution to inform instruction and assessment in elementary science. Education Sciences, 11 (10), 609. doi: 10.3390/educsci11100609</bibtext> </blist> <blist> <bibl id="bib6" idref="ref18" type="bt">6</bibl> <bibtext> Dray, A. J., Brown, N. J. S., Diakow, R., Lee, Y., & Wilson, M. (2019). A construct modeling approach to the assessment of reading comprehension for adolescent readers. Reading Psychology, 40 (2), 191 – 241. doi: 10.1080/02702711.2019.1614125</bibtext> </blist> <blist> <bibl id="bib7" idref="ref7" type="bt">7</bibl> <bibtext> Duschl, R., Maeng, S., & Sezen, A. (2011). Learning progressions and teaching sequences: A review and analysis. Studies in Science Education, 47 (2), 123 – 182. doi: 10.1080/03057267.2011.604476</bibtext> </blist> <blist> <bibl id="bib8" idref="ref79" type="bt">8</bibl> <bibtext> Feuerstahler, L., & Wilson, M. (2019). Scale alignment in between-item multidimensional Rasch models. Journal of Educational Measurement, 56 (2), 280 – 301. doi: 10.1111/jedm.12209</bibtext> </blist> <blist> <bibl id="bib9" idref="ref10" type="bt">9</bibl> <bibtext> Feuerstahler, L., & Wilson, M. (2021). Scale alignment in the between-item multidimensional partial credit model. Applied Psychological Measurement, 45 (4), 268 – 282. doi: 10.1177/01466216211013103</bibtext> </blist> <blist> <bibtext> Gosling, S. D., Rentfrow, P. J., & Swann, W. B., Jr. (2003). A very brief measure of the big-five personality domains. Journal of Research in Personality, 37 (6), 504 – 528. doi: 10.1016/S0092-6566(03)00046-1</bibtext> </blist> <blist> <bibtext> Hartig, J., & Höhler, J. (2009). Multidimensional IRT models for the assessment of competencies. Studies in Educational Evaluation, 35 (2–3), 57 – 63. doi: 10.1016/j.stueduc.2009.10.002</bibtext> </blist> <blist> <bibtext> Humphreys, L. G. (1985). General intelligence: An integration of factor, test, and simplex theory. In B. B. Wolman (Ed.), Handbook of intelligence (pp. 201 – 224). New York : Wile.</bibtext> </blist> <blist> <bibtext> Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001). A comprehensive meta-analysis of the predictive validity of the graduate record examinations: Implications for graduate student selection and performance. Psychological Bulletin, 127 (1), 162. doi: 10.1037/0033-2909.127.1.162</bibtext> </blist> <blist> <bibtext> Linacre, J. M., & Wright, B. (1995). Rasch measurement transactions. Chicago : MESA Press.</bibtext> </blist> <blist> <bibtext> Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47 (2), 149 – 174. doi: 10.1007/BF02296272</bibtext> </blist> <blist> <bibtext> McKinley, R. L., & Way, W. D. (1992). The feasibility of modeling secondary TOEFL ability dimensions using multidimensional IRT models. ETS Research Report Series, 1992 (1), i – 22. doi: 10.1002/j.2333-8504.1992.tb01448.x</bibtext> </blist> <blist> <bibtext> Metz, K. E., Cardace, A., Berson, E., Ly, U., Wong, N. ... Wilson, M. (2019). Primary grade children's capacity to understand microevolution: The power of leveraging their fruitful intuitions and engagement in scientific practices. Journal of the Learning Sciences, 28 (4–5), 556 – 615. doi: 10.1080/10508406.2019.1667806</bibtext> </blist> <blist> <bibtext> Morell, L., Collier, T., Black, P., & Wilson, M. (2017). A construct-modeling approach to develop a learning progression of how students understand the structure of matter. Journal of Research in Science Teaching, 54 (8), 1024 – 1048. doi: 10.1002/tea.21397</bibtext> </blist> <blist> <bibtext> OECD. (2019). PISA 2018: Insights and interpretations. Paris : OECD Publishing.</bibtext> </blist> <blist> <bibtext> Reckase, M. D. (2009). Multidimensional item response theory models (pp. 79 – 112). New York : Springer.</bibtext> </blist> <blist> <bibtext> Rittle-Johnson, B., Matthew, P. G., Taylor, R. S., & McEldoon, K. L. (2011). Assessing knowledge of mathematical equivalence: A construct-modeling approach. Journal of Educational Psychology, 103 (1), 85 – 104. doi: 10.1037/a0021334</bibtext> </blist> <blist> <bibtext> Sikorski, T. R. (2019). Context-dependent "upper anchors" for learning progressions. Science & Education, 28 (8), 957 – 981. doi: 10.1007/s11191-019-00074-w</bibtext> </blist> <blist> <bibtext> Thomas, M. L. (2019). Advances in applications of item response theory to clinical assessment. Psychological Assessment, 31 (12), 1442. doi: 10.1037/pas0000597</bibtext> </blist> <blist> <bibtext> Wainer, H., Bradlow, E. T., & Wang, X. (2007). Testlet response theory and its applications. Cambridge : Cambridge University Press.</bibtext> </blist> <blist> <bibtext> Wang, W. C. (1999). Direct estimation of correlations among latent traits within IRT framework. Methods of Psychological Research Online, 4 (2), 47 – 70.</bibtext> </blist> <blist> <bibtext> Wang, W. C., Wilson, M., & Adams, R. J. (1997). Rasch models for multidimensionality between items and within items. Objective Measurement: Theory into Practice, 4, 139 – 155.</bibtext> </blist> <blist> <bibtext> Way, W. D., Ansley, T. N., & Forsyth, R. A. (1988). The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional IRT estimates. Applied Psychological Measurement, 12 (3), 239 – 252. doi: 10.1177/014662168801200303</bibtext> </blist> <blist> <bibtext> Williams, Z. J., Everaert, J., & Gotham, K. O. (2021). Measuring depression in autistic adults: Psychometric validation of the beck depression inventory–II. Assessment, 28 (3), 858 – 876. doi: 10.1177/1073191120952889</bibtext> </blist> <blist> <bibtext> Wilson, M. (2009). Measuring progressions: Assessment structures underlying a learning progression. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 46 (6), 716 – 730. doi: 10.1002/tea.20318</bibtext> </blist> <blist> <bibtext> Wilson, M. (2023). Constructing measures: An item response modeling approach. New York : Taylor & Francis.</bibtext> </blist> <blist> <bibtext> Wilson, M., & Adams, R. J. (1995). Rasch models for item bundles. Psychometrika, 60 (2), 181 – 198. doi: 10.1007/BF02301412</bibtext> </blist> <blist> <bibtext> Wilson, M., & Scalise, K. (2014). Assessment of learning in digital networks. In P. G. In & E. Care (Eds.), Assessment and teaching of 21st Century skills volume 2 - methods & approaches (pp. 57 – 84). Dordrecht : Springer.</bibtext> </blist> <blist> <bibtext> Wilson, M., Scalise, K., & Gochyyev, P. (2017). Modeling data from collaborative assessments: Learning in digital interactive social networks. Journal of Educational Measurement, 54 (1), 85 – 102. doi: 10.1111/jedm.12134</bibtext> </blist> <blist> <bibtext> Wu, M., & Adams, R. (2006). Modelling mathematics problem solving item responses using a multidimensional IRT model. Mathematics Education Research Journal, 18 (2), 93 – 113. doi: 10.1007/BF03217438</bibtext> </blist> <blist> <bibtext> Xue, M., & Chen, Y. (2023). A stan tutorial on Bayesian IRTree models: Conventional models and explanatory extension. Behavior Research Methods, 1 – 21. doi: 10.3758/s13428-023-02121-5</bibtext> </blist> </ref> <aug> <p>By Mingfeng Xue and Mark Wilson</p> <p>Reported by Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib26" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib20" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib27" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib34" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib22" firstref="ref6"></nolink> <nolink nlid="nl6" bibid="bib30" firstref="ref11"></nolink> <nolink nlid="nl7" bibid="bib21" firstref="ref16"></nolink> <nolink nlid="nl8" bibid="bib32" firstref="ref17"></nolink> <nolink nlid="nl9" bibid="bib33" firstref="ref20"></nolink> <nolink nlid="nl10" bibid="bib18" firstref="ref21"></nolink> <nolink nlid="nl11" bibid="bib29" firstref="ref22"></nolink> <nolink nlid="nl12" bibid="bib10" firstref="ref24"></nolink> <nolink nlid="nl13" bibid="bib13" firstref="ref25"></nolink> <nolink nlid="nl14" bibid="bib24" firstref="ref26"></nolink> <nolink nlid="nl15" bibid="bib31" firstref="ref27"></nolink> <nolink nlid="nl16" bibid="bib35" firstref="ref29"></nolink> <nolink nlid="nl17" bibid="bib12" firstref="ref31"></nolink> <nolink nlid="nl18" bibid="bib16" firstref="ref33"></nolink> <nolink nlid="nl19" bibid="bib23" firstref="ref34"></nolink> <nolink nlid="nl20" bibid="bib28" firstref="ref35"></nolink> <nolink nlid="nl21" bibid="bib19" firstref="ref37"></nolink> <nolink nlid="nl22" bibid="bib25" firstref="ref39"></nolink> <nolink nlid="nl23" bibid="bib11" firstref="ref41"></nolink> <nolink nlid="nl24" bibid="bib17" firstref="ref47"></nolink> <nolink nlid="nl25" bibid="bib15" firstref="ref71"></nolink> <nolink nlid="nl26" bibid="bib14" firstref="ref88"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1413499
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Modeling Dimensions Converging at the Upper Anchor in Learning Progressions: An Example of Micro-Evolution
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Mingfeng+Xue%22">Mingfeng Xue</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-4801-3754">0000-0002-4801-3754</externalLink>)<br /><searchLink fieldCode="AR" term="%22Mark+Wilson%22">Mark Wilson</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0425-5305">0000-0002-0425-5305</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Applied+Measurement+in+Education%22"><i>Applied Measurement in Education</i></searchLink>. 2024 37(1):71-87.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 17
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Learning+Trajectories%22">Learning Trajectories</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Assessment%22">Educational Assessment</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Evolution%22">Evolution</searchLink><br /><searchLink fieldCode="DE" term="%22Measurement+Techniques%22">Measurement Techniques</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/08957347.2024.2311934
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0895-7347<br />1532-4818
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Multidimensionality is common in psychological and educational measurements. This study focuses on dimensions that converge at the upper anchor (i.e. the highest acquisition status defined in a learning progression) and compares different ways of dealing with them using the multidimensional random coefficients multinomial logit model and scale alignment methods. Assumptions underlying the four approaches studied are (a) ignoring the convergence, (b) recognizing the convergence of the dimensions, (c) treating convergence as a new dimension, and (d) separating the within-category multidimensionality and treating convergence as a new dimension. A learning progression about micro-evolution is used as an example, including model fits, step difficulties, and associations between dimensions, Wright maps are drawn, and inferences are made under the four building blocks of measurement development. Finally, the usefulness and weaknesses of the four approaches are discussed.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1413499
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1413499
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/08957347.2024.2311934
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 17
        StartPage: 71
    Subjects:
      – SubjectFull: Learning Trajectories
        Type: general
      – SubjectFull: Educational Assessment
        Type: general
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Evolution
        Type: general
      – SubjectFull: Measurement Techniques
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Multiple Choice Tests
        Type: general
    Titles:
      – TitleFull: Modeling Dimensions Converging at the Upper Anchor in Learning Progressions: An Example of Micro-Evolution
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Mingfeng Xue
      – PersonEntity:
          Name:
            NameFull: Mark Wilson
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0895-7347
            – Type: issn-electronic
              Value: 1532-4818
          Numbering:
            – Type: volume
              Value: 37
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Applied Measurement in Education
              Type: main
ResultId 1