View in EDS HTML Full Text PDF Full Text

Computational Concepts and Their Assessment in Preschool Students: An Empirical Study

Saved in:
Bibliographic Details
Title: Computational Concepts and Their Assessment in Preschool Students: An Empirical Study
Language: English
Authors: Marcos Jiménez (ORCID 0000-0003-4029-6144), María Zapata-Cáceres (ORCID 0000-0002-8817-5889), Marcos Román-González (ORCID 0000-0001-8506-1715), Gregorio Robles (ORCID 0000-0002-1442-6761), Jesús Moreno-León (ORCID 0000-0002-3821-5707), Estefanía Martín-Barroso (ORCID 0000-0001-5652-5592)
Source: Journal of Science Education and Technology. 2024 33(6):998-1020.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 23
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Descriptors: Computation, Thinking Skills, Student Evaluation, Preschool Children, Problem Solving, Cognitive Tests, Psychometrics, Test Reliability, Test Validity, Scoring, Skill Development
DOI: 10.1007/s10956-024-10142-8
ISSN: 1059-0145
1573-1839
Abstract: Computational thinking (CT) is a multidimensional term that encompasses a wide variety of problem-solving skills related to the field of computer science. Unfortunately, standardized, valid, and reliable methods to assess CT skills in preschool children are lacking, compromising the reliability of the results reported in CT interventions. To surpass this limitation, we validated in a sample of 700 preschool students (5-6 years old) the Beginners Computational Thinking test Short-Form (BCTt-SF), an unplugged 12-item instrument that measures three of the most common computational concepts assessed in preschool research: sequences, loops, and conditionals. The theoretical model underpinning the BCTt-SF was supported by dimensionality assessment, which suggested that preschool students can be distinguished in terms of four specific abilities (i.e., sequences, simple loops, nested loops, and conditionals) and that all of these abilities were related by a general factor. We modeled this hierarchical structure with a bi-factor model that presented excellent psychometric properties, from good statistical fit indices to adequate reliability of the general ability. To take full advantage of this model, we created an online application in the Shiny platform (https://computationalthinkingtests.shinyapps.io/SF-BCTt/) for the seamless scoring of examinees by any teacher or researcher who uses the BCTt-SF to assess CT skills in preschool children. Finally, we demonstrated how the BCTt-SF can be used to test the impact of educational interventions for improving CT skills in preschoolers.
Abstractor: As Provided
Notes: https://osf.io/3xn7a/?view_only=7190e194469f44119aa9174b768a406d
Entry Date: 2024
Accession Number: EJ1446117
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwETxvgE7xduBvOvV-6copWUAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDMubwp3r96k7tyT5UwIBEICBm3q9RJPUSKC2v5yXPkvImrQviOp-A4jEaooeewj5u687h1-2MqWiACyF0Gozq7lRIoLc_48dKVnJQxaQBxTFKHIkJojO0UnbCcSMQVhWK9bF9sqzHtZ7KBDUrnt4IkhdhdsZy4drg3lBSbS8lQBu8655C7mZiB20KmWYrXq71MpY4U7GvyCc_X91ioGnd0o6CjbJd9vODgGbQTFF
Text:
  Availability: 1
  Value: <anid>AN0180550747;4n601dec.24;2024Nov01.03:52;v2.2.500</anid> <title id="AN0180550747-1">Computational Concepts and their Assessment in Preschool Students: An Empirical Study </title> <p>Computational thinking (CT) is a multidimensional term that encompasses a wide variety of problem-solving skills related to the field of computer science. Unfortunately, standardized, valid, and reliable methods to assess CT skills in preschool children are lacking, compromising the reliability of the results reported in CT interventions. To surpass this limitation, we validated in a sample of 700 preschool students (5–6 years old) the Beginners Computational Thinking test Short-Form (BCTt-SF), an unplugged 12-item instrument that measures three of the most common computational concepts assessed in preschool research: sequences, loops, and conditionals. The theoretical model underpinning the BCTt-SF was supported by dimensionality assessment, which suggested that preschool students can be distinguished in terms of four specific abilities (i.e., sequences, simple loops, nested loops, and conditionals) and that all of these abilities were related by a general factor. We modeled this hierarchical structure with a bi-factor model that presented excellent psychometric properties, from good statistical fit indices to adequate reliability of the general ability. To take full advantage of this model, we created an online application in the Shiny platform (https://computationalthinkingtests.shinyapps.io/SF-BCTt/) for the seamless scoring of examinees by any teacher or researcher who uses the BCTt-SF to assess CT skills in preschool children. Finally, we demonstrated how the BCTt-SF can be used to test the impact of educational interventions for improving CT skills in preschoolers.</p> <p>Keywords: Computational thinking; Early years education; Evaluation methodologies; Psychometric validation; 21st century abilities</p> <p>Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s10956-024-10142-8.</p> <hd id="AN0180550747-2">Introduction</hd> <p>Many instances of problem solving in daily life require mastering abilities closely related to terms used in computer science. For example, pattern recognition is an important skill for driving a vehicle to identify obstacles in traffic flow and adjust the route accordingly, whereas interpreting conditional rules (e.g., stopping <emph>if</emph> encountering a stop sign or at a yield sign <emph>while</emph> other vehicles keep approaching from either direction) are also essential aspects for driving that are reminiscent concepts from computer science.</p> <p>However, the application of computational thinking (CT) skills is not restricted to highly standardized activities. Contrary, it comprises "the thought processes involved in formulating problems so their solutions can be represented as computational steps and algorithms" (Aho, [<reflink idref="bib1" id="ref1">1</reflink>]). As such, these processes can be transferred to nonprogramming domains and more nuanced situations (Weintrop et al., [<reflink idref="bib57" id="ref2">57</reflink>]). For instance, a person who is planning a vacation must break the problem down into smaller, manageable parts (e.g., destination, transportation, accommodations, activities, budget, timing), which is an ability termed <emph>decomposition</emph> (Shute et al., [<reflink idref="bib47" id="ref3">47</reflink>]).</p> <p>Graph: Fig. 1 Depictions of four items belonging to the BCTt, each one taken from a different computational concept block. Upper-left: Sequence block; bottom-left: Simple Loop block; upper-right: Nested Loop block; bottom-right: Conditional block</p> <p>The fact that everyone uses CT abilities on a routinely basis and with different degrees of success has motivated many educators to suggest that CT should be a compulsory subject as worthy as reading, writing, and arithmetic (Wing, [<reflink idref="bib58" id="ref4">58</reflink>]) and that it should be assessed at all levels of education (El-Hamamsy et al., [<reflink idref="bib14" id="ref5">14</reflink>]; Relkin & Bers, [<reflink idref="bib37" id="ref6">37</reflink>]; Stamatios, [<reflink idref="bib48" id="ref7">48</reflink>]), including preschool grades (Georgiou & Angeli, [<reflink idref="bib20" id="ref8">20</reflink>]; Stamatios, [<reflink idref="bib49" id="ref9">49</reflink>]). As a consequence, there has been a significant progress over the last years to promote CT in compulsory education (Kong, [<reflink idref="bib27" id="ref10">27</reflink>]; Manches & Plowman, [<reflink idref="bib31" id="ref11">31</reflink>]; Saxena et al., [<reflink idref="bib46" id="ref12">46</reflink>]; Wang et al., [<reflink idref="bib56" id="ref13">56</reflink>]; Weintrop et al., [<reflink idref="bib57" id="ref14">57</reflink>]), starting as early as primary school in some countries, such as New Zealand, South Korea, Canada, or Singapore, and in preschool, such as in Australia and Japan (Bocconi et al., [<reflink idref="bib7" id="ref15">7</reflink>]).</p> <p>With the advent of this new academic curricula in which students acquire CT skills, instruments for monitoring their learning progress have been proposed recently. The assessment of CT skills is a highly valuable endeavor because it helps to assess strengths and gaps in the curricula, to identify children who need further support, and to provide feedback to teachers on the effectiveness of their teaching methods. Additionally, assessing CT skills can also serve to identify areas where students are excelling, prompting opportunities for enrichment and challenge for these students (Román-González et al., [<reflink idref="bib44" id="ref16">44</reflink>]). Alternatively, it may promote social equity by helping children with disabilities or special needs to prepare them for higher levels of education (Taylor, [<reflink idref="bib52" id="ref17">52</reflink>]).</p> <p>Up to date, instruments for measuring CT skills span a wide audience, ranging from preschoolers to secondary school students (El-Hamamsy et al., [<reflink idref="bib14" id="ref18">14</reflink>]; Relkin & Bers, [<reflink idref="bib37" id="ref19">37</reflink>]; Relkin et al., [<reflink idref="bib38" id="ref20">38</reflink>]; Román-González et al., [<reflink idref="bib44" id="ref21">44</reflink>]; Zapata-Cáceres et al., [<reflink idref="bib60" id="ref22">60</reflink>], [<reflink idref="bib61" id="ref23">61</reflink>]). Notwithstanding, the development of rigorous testing procedures for the former group remains scarcer. This is a problem because an instrument designed for a specific age range is likely to be too difficult for those immediately younger, differing also in its capacity for disentangling students with different levels of skill. And more worrisome, a recent review of assessment methods in CT research for preschool children found that most of these methods were not standardized but created ad-hoc and did not undergo a proper validation and reliability analysis (Tang et al., [<reflink idref="bib51" id="ref24">51</reflink>]).</p> <p>To overcome these limitations, we made a psychometric adaptation for preschool students of the Beginners Computational Thinking test (BCTt; Zapata-Cáceres et al., [<reflink idref="bib60" id="ref25">60</reflink>]), an instrument originally designed for children aged 5 to 9 years old that has been validated for measuring the understanding of three computational concepts in lower-primary school: sequences, loops, and conditionals. Sequences are sets of instructions or steps that need to be taken in turn; loops are rules to run the same sequence a given number of times, either by repeating each time a sequential term (i.e., simple loop) or by repeating a number of times a block of sequential terms (i.e., nested loop). Finally, conditionals are indications to perform an action when either a specific circumstance is satisfied (i.e., if-then), when a condition is satisfied and another action when it is not (i.e., if-then-else), or as long as a condition remains satisfied (i.e., while).</p> <p>In the BCTt, these three broad computational concepts are put into test in 25 items, categorized as either grid format (21 items) or canvas format (4 items). In the grid format items, the examinees are instructed to move an object (i.e., a baby chicken) across a grid to reach a target (i.e., the mother of the baby chicken) by spotting the correct sequential pattern from a list of four alternatives. Figure 1 provides an illustrative example of a typical grid item format in the BCTt, encompassing each of the aforementioned dimensions. Conversely, in canvas format items, the examinees are presented with a simple path drawing over a canvas and must select the correct sequential pattern that corresponds to the given path from another list of four alternatives. An important note is that, although the BCTt is mainly focused on the assessment of these computational concepts (e.g., sequences, loops, and conditionals), it is not limited to them. Contrary, computational practices such as testing and debugging are also required to solve the questions of the BCTt because the examinees need to inspect the four response alternatives (i.e., A, B, C, and D) to spot the erroneous ones.</p> <p>The BCTt bears the potential to be applied in preschool settings because it was found to be most suited for low-percentile students in primary school (Zapata-Cáceres et al., [<reflink idref="bib61" id="ref26">61</reflink>]). However, the average time of completion in lower-primary school students was about 40 min (Zapata-Cáceres et al., [<reflink idref="bib61" id="ref27">61</reflink>]). Therefore, reducing the test length is necessary for preschool children to complete it, as they need a longer time to answer most of the items and are more prone to fatigue. Such, the aim of this study is three-fold:</p> <p></p> <ulist> <item> First, to identify the nonoverlapping computational concepts that the students can differentially understand and upon which can be differentiated.</item> <p></p> <item> Second, to select the subset of items from the BCTt that consistently measures the target computational concepts in preschool students, discarding those that are tangential or measure spurious features. We also intend to shorten further the test by selecting the items that best encompass all the computational dimensions while retaining adequate psychometric properties of reliability and content validity, creating the Short-Form of the Beginners Computational Thinking test (BCTt-SF).</item> <p></p> <item> Third, to create an online application where teachers can upload the responses of their students and automatically obtain the corresponding scores on all the computational dimensions assessed by the BCTt-SF.</item> </ulist> <hd id="AN0180550747-3">Computational Thinking in Preschoolers</hd> <p>Although no consensus definition of CT currently exists, there is agreement in that CT is a multidimensional term that encompasses a wide variety of problem-solving skills, including logical thinking, development of algorithms, data representation and manipulation, computer programming, and more (Shute et al., [<reflink idref="bib47" id="ref28">47</reflink>]). In other words, CT is not just about solving problems in a computational environment, but applying thought processes that lead to problem evaluation, simplification, and elaboration of strategies to find a solution using a set of skills that are related to computational science. However, the existence of multiple definitions, categorizations, and assessment proposals of CT abilities makes difficult for educators to decide what specific competences should be addressed in a given age group and how to do it. For instance, while some authors warn about the scarcity of research with regards to the specific skills that preschoolers candominate (Relkin & Bers, [<reflink idref="bib37" id="ref29">37</reflink>]), others remark the lack of consensus about what computational concepts and practices should be taught to children at these early ages (McCormick & Hall, [<reflink idref="bib33" id="ref30">33</reflink>]).</p> <p>According to Piaget's theory of cognitive development, preschool children fall under the preoperational stage of development that spans between 2 and 7 years old. This is a period when children acquire language and symbolic thinking. They learn to represent objects and events mentally without their physical presence and through images and words, prompting the appearance of abilities such as sequencing, pattern recognition, and algorithm design (Saxena et al., [<reflink idref="bib46" id="ref31">46</reflink>]).</p> <p>In this vein, there exists some research in preschool children in relation to the specific concepts of sequences, loops, and conditionals that the BCTt measures. Specifically, there is consensus in that preschoolers can solve with accuracy simple loops problems (Angeli & Valanides, [<reflink idref="bib3" id="ref32">3</reflink>]; Bers et al., [<reflink idref="bib6" id="ref33">6</reflink>]; Saxena et al., [<reflink idref="bib46" id="ref34">46</reflink>]), although they struggle more than with complex sequences (Elkin et al., [<reflink idref="bib15" id="ref35">15</reflink>]). In any case, it is considered that this ability is already mastered in early primary school (Relkin & Bers, [<reflink idref="bib37" id="ref36">37</reflink>]). With regards to their performance in solving problems with loops, research also shows that they can understand and apply with success this concept (Angeli & Valanides, [<reflink idref="bib3" id="ref37">3</reflink>]; Bers et al., [<reflink idref="bib6" id="ref38">6</reflink>]; Relkin & Bers, [<reflink idref="bib37" id="ref39">37</reflink>]). Lastly, there is also evidence that preschoolers can solve certain tasks involving conditional statements (Andrews et al., [<reflink idref="bib2" id="ref40">2</reflink>]; Diamond et al., [<reflink idref="bib12" id="ref41">12</reflink>]). In fact, it seems that they can do it much before than what it was previously thought (Ling et al., [<reflink idref="bib28" id="ref42">28</reflink>]), indicating that it is a concept that preschoolers can understand at an age of 3 years old.</p> <hd id="AN0180550747-4">Test Development in Computational Thinking Research for Preschoolers</hd> <p>A systematic review conducted by Bakala et al. ([<reflink idref="bib4" id="ref43">4</reflink>]) confirmed that sequences, loops, and conditionals were, in fact, the computational concepts most commonly assessed in the literature for the purpose of testing educational interventions. However, despite the recent development of many appealing educational programs and curricula for promoting the learning of CT skills in preschool children, the lack of valid instruments for testing them remains as one of the biggest problems in the field (Bakala et al., [<reflink idref="bib4" id="ref44">4</reflink>]; Tang et al., [<reflink idref="bib51" id="ref45">51</reflink>]). Moreover, in order to make an objective comparison between the many different educational interventions that have been developed in the field, it is necessary to operationalize the computational concepts and practices that are relevant for different age groups and create standardized instruments accordingly (McCormick & Hall, [<reflink idref="bib33" id="ref46">33</reflink>]). Thus, we currently do not know to what extent the rubrics, tasks, and tests that conform these assessment methods measure the intended computational constructs and, ultimately, if they are valid for the purpose of scoring students. As a consequence of this lack of validated instruments, the fair comparison of different CT interventions is neither possible. Moreover, the samples recruited in these studies were generally small, precluding even the application of reliable psychometric analyses.</p> <p>Another problem is that most of the reviewed instruments were plugged (i.e., they involved tasks where students needed to interact with electronic devices, write or interpret code, program robots, and perform other activities that required specific computational knowledge). Unfortunately, these kinds of plugged assessments have drawbacks. When using technological environments to measure CT skills, children who lack the necessary coding knowledge to participate cannot be assessed, prompting the risk of conflating CT abilities with programming skills. Moreover, some children may outperform others because of more familiarity or prior exposure to technological environments and a more developed fine-motor functioning. An additional disadvantage of plugged methods for assessment is that they are costly because they require materials (e.g., physical kits) and the individual observation of each child. Furthermore, plugged tasks are often assessed with portfolios and rubrics, which can also be problematic due to the subjectivity of the teacher who scores the behavior of the children.</p> <p>For these reasons, many of the new psychometric instruments that are being developed take an unplugged design based on traditional tests (El-Hamamsy et al., [<reflink idref="bib14" id="ref47">14</reflink>]; Li et al., [<reflink idref="bib29" id="ref48">29</reflink>]; Relkin & Bers, [<reflink idref="bib37" id="ref49">37</reflink>]; Relkin et al., [<reflink idref="bib39" id="ref50">39</reflink>], [<reflink idref="bib38" id="ref51">38</reflink>]; Román-González et al., [<reflink idref="bib42" id="ref52">42</reflink>]; Zapata-Cáceres et al., [<reflink idref="bib61" id="ref53">61</reflink>]; Zhang & Wong, [<reflink idref="bib62" id="ref54">62</reflink>]). Consider the TechCheck (Relkin et al., [<reflink idref="bib39" id="ref55">39</reflink>]), an unplugged 15-item test designed for preschool students that measures six different computational concepts and practices: algorithms, modularity, debugging, hardware/software, control structures, and representation. The TechCheck does not require any coding knowledge and can be easily scored. In addition, it was adapted to the population of preschool students using a sample of 395 students, yielding the revised version TechCheck-K (Relkin & Bers, [<reflink idref="bib37" id="ref56">37</reflink>]; Relkin et al., [<reflink idref="bib38" id="ref57">38</reflink>]).</p> <p>Table 1 Review of Computational Thinking tests directly related to the BCTt-SF in chronological order</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Test</p></th><th align="left"><p>Format</p></th><th align="left"><p>Target group</p></th><th align="left"><p>CT concepts</p></th><th align="left"><p>Psychometric assessment</p></th><th align="left"><p>Sample</p></th></tr></thead><tbody><tr><td align="left"><p>CTt (Román-González et al., <xref ref-type="bibr" rid="bibr42">2015</xref>, <xref ref-type="bibr" rid="bibr43">2017</xref>, <xref ref-type="bibr" rid="bibr44">2018</xref>; Guggemos et al., <xref ref-type="bibr" rid="bibr24">2022</xref>)</p></td><td align="left"><p>28 items</p></td><td align="left"><p>Secondary school (10-16 years old)</p></td><td align="left"><p>Sequences, Finite Loops, Loops over a condition, Conditionals (i.e., If-then, If-else, and While statements), and Functions</p></td><td align="left"><p>Expert validation, Reliability estimation within CTT (i.e., Cronbach alpha) and Item Response Theory, Criterion validity, Predictive validity</p></td><td align="left"><p>400 students (Román-González et al., <xref ref-type="bibr" rid="bibr42">2015</xref>) 1251 students (Román-González et al., <xref ref-type="bibr" rid="bibr43">2017</xref>) 314 students (Román-González et al., <xref ref-type="bibr" rid="bibr44">2018</xref>; Guggemos et al., <xref ref-type="bibr" rid="bibr24">2022</xref>)</p></td></tr><tr><td align="left"><p>BCTt (Zapata-Cáceres et al., <xref ref-type="bibr" rid="bibr60">2020</xref>)</p></td><td align="left"><p>25 items</p></td><td align="left"><p>lower-Primary school (5-7 years old)</p></td><td align="left"><p>Sequences, Simple Loops, Nested Loops, and Conditionals (i.e., If-then, If-else, and While statements)</p></td><td align="left"><p>Reliability estimation within CTT (i.e., Cronbach alpha)</p></td><td align="left"><p>299 students</p></td></tr><tr><td align="left"><p>cCTt (El-Hamamsy et al., <xref ref-type="bibr" rid="bibr13">2022a</xref>)</p></td><td align="left"><p>Three versions of 25, 17, and 15 items</p></td><td align="left"><p>upper-Primary school (7-9 years old)</p></td><td align="left"><p>Sequences, Simple Loops, Nested Loops, Conditionals (i.e., If-then, If-else, and While statements), Complex (i.e., combinations of the previous concepts)</p></td><td align="left"><p>Expert Validation, Model fit assessment with CFA, Item calibration and reliability estimation within IRT (i.e., item information), Reliability estimation within CTT (i.e., Cronbach alpha)</p></td><td align="left"><p>1519 students</p></td></tr><tr><td align="left"><p>BCTt-SF (current study)</p></td><td align="left"><p>Selection of 20 items Final version of 12 items</p></td><td align="left"><p>Preschool students (4-6 years old)</p></td><td align="left"><p>Sequences, Simple Loops, Nested Loops, Conditionals (i.e., If-then and If-then-else)</p></td><td align="left"><p>Dimensionality assessment, Model fit assessment with CFA, Item calibration and reliability estimation with MIRT (i.e., item information), and automatic test scoring via an online application</p></td><td align="left"><p>700 students</p></td></tr></tbody></table> </ephtml> </p> <p> <emph>Note</emph>. CTt: Computational Thinking test; BCTt: Beginners Computational Thinking test; cCTt: competent Computational Thinking test; BCTt-SF: Short-Form of the Beginners Computational Thinking test; CTT: Classical Test Theory</p> <p>Another example is the BCTt (Zapata-Cáceres et al., [<reflink idref="bib61" id="ref58">61</reflink>]). As pointed out by Bakala et al. ([<reflink idref="bib4" id="ref59">4</reflink>]), instruments like the BCTt are advantageous because many of the activities that were assessed in the literature were closely related to the tasks used during the educational program, whereas the BCTt offers measures that are independent from the devices used during the intervention, providing information about the transferable properties of CT skills. Moreover, the BCTt is an instrument created from adaptations of previous tests widely used in higher education levels that have undergone validity and reliability assessments (Table 1). In fact, we are not the first authors who use the BCTt to create an adaptation for students of a different age group. The BCTt has already been successfully adapted for upper-primary school students by increasing item difficulty, giving raise to the competent Computational Thinking test (cCTt; El-Hamamsy et al., [<reflink idref="bib13" id="ref60">13</reflink>]). As such, the theory underpinning the BCTt is well consolidated, and the BCTt itself serves as a benchmark for other tests as well as a template for the development of new tests (El-Hamamsy et al., [<reflink idref="bib13" id="ref61">13</reflink>]).</p> <p>To determine the internal validity of the BCTt and to accomplish a reliable adaptation of the BCTt with a suitable test length for preschool students, we pursued the three objectives outlined above: First, we determined which computational concepts can be distinctly assessed in a sample of preschool students. Second, we undertook a purification process in which screening out the items that do not measure the intended computational concepts. As a result, we selected the most informative items while maintaining adequate validity and reliability properties, resulting in a shortened version of the test (BCTt-SF). Third, we provided an online application for the automatic and seamless scoring of any preschool examinee.</p> <hd id="AN0180550747-5">Methods</hd> <p>In this section, we describe the details of the data collection and the psychometric analyses that we carried out to develop the adaptation of the BCTt for preschool students.</p> <hd id="AN0180550747-6">Participants and Data Collection</hd> <p>A sample of 700 preschool students aged 5 to 6 years old recruited from 8 classrooms across several Spanish regions participated in the current study. The sample was divided in two groups: quasi-experimental and control. The quasi-experimental group encompassed 595 children whose teachers subscribed voluntarily to an educational program promoted by the Spanish Ministry of Education and Vocational Training.[<reflink idref="bib1" id="ref62">1</reflink>]4 The aim of this project was investigating the impact of the unplugged CT activities in the development of the CT skills of children. The educational intervention, termed <emph>School of Computational Thinking and Artificial Intelligence</emph>, included a training phase for the teachers (i.e., from 13 November 2020 to 1 February 2021) and a curriculum implementation phase for the children (i.e., from 1 March to 31 May 2021). The training phase was composed by four blocks. In the first block, CT was introduced to the teachers, who learned the definitions of several concepts such as sequences, algorithms, logical thinking, abstraction, decomposition, parallelism; in the second block, they completed different unplugged activities involving these concepts to acquire familiarity with them. In the third block, they learned how to analyze the didactic value of the unplugged activities using <emph>Cody & Roby</emph>,[<reflink idref="bib2" id="ref63">2</reflink>] a game in which Cody (i.e., the programmer) creates instructions that Roby (i.e., the robot) must execute. The final block was dedicated to the presentation of unplugged activities for the teaching of Artificial Intelligence (Touretzky et al., [<reflink idref="bib53" id="ref64">53</reflink>]). In total, the teachers dedicated 30 h to complete all of these training blocks.</p> <p>Table 2 Item features and error profiles for each response alternative. The proportion in which each alternative was chosen is also presented within parentheses</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left" /><th align="left" /><th align="left" colspan="4"><p>Response alternatives</p></th><th align="left" colspan="3"><p>Missingness</p></th><th align="left" colspan="6"><p>Error Profiles</p></th></tr><tr><th align="left"><p> Block</p></th><th align="left"><p>Q</p></th><th align="left"><p>Format</p></th><th align="left"><p>A</p></th><th align="left"><p>B</p></th><th align="left"><p>C</p></th><th align="left"><p>D</p></th><th align="left"><p>Prop</p></th><th align="left"><p>WD</p></th><th align="left"><p>Diff</p></th><th align="left"><p>1</p></th><th align="left"><p>2</p></th><th align="left"><p>3</p></th><th align="left"><p>4</p></th><th align="left"><p>5</p></th><th align="left"><p>6</p></th></tr></thead><tbody><tr><td align="left"><p>Sequences</p></td><td align="left"><p>1</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>3</mn><mo>×</mo><mn>3</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq1.gif" />)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq2.gif" /> (.89)</p></td><td align="left"><p>1 (.05)</p></td><td align="left"><p>6 (.03)</p></td><td align="left"><p>1 (.03)</p></td><td align="left"><p>.00</p></td><td align="left"><p>.00</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq3.gif" /></p></td><td align="left"><p>Wrong</p></td><td align="left"><p>Insufficient</p></td><td align="left"><p>Fails to avoid</p></td><td align="left"><p>Fails to pick</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq4.gif" /></p></td><td align="left"><p>Other error</p></td></tr><tr><td align="left" /><td align="left"><p>2</p></td><td align="left"><p>Canvas</p></td><td align="left"><p>1 (.05)</p></td><td align="left"><p>1 (.05)</p></td><td align="left"><p>1 (.05)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq5.gif" /> (.81)</p></td><td align="left"><p>.04</p></td><td align="left"><p>.00</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq6.gif" /></p></td><td align="left"><p>direction</p></td><td align="left"><p>steps</p></td><td align="left"><p>the obstacle</p></td><td align="left"><p>the object</p></td><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>3</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>3</mn><mo>×</mo><mn>3</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq7.gif" />)</p></td><td align="left"><p>3 (.05)</p></td><td align="left"><p>2|3 (.04)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq8.gif" /> (.82)</p></td><td align="left"><p>1 (.05)</p></td><td align="left"><p>.03</p></td><td align="left"><p>.00</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq9.gif" /></p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>4</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>3</mn><mo>×</mo><mn>3</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq10.gif" />)</p></td><td align="left"><p>2 (.06)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq11.gif" /> (.77)</p></td><td align="left"><p>3|4 (.05)</p></td><td align="left"><p>4 (.09)</p></td><td align="left"><p>.04</p></td><td align="left"><p>.00</p></td><td align="left"><p>-0.58</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>5</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq12.gif" />)</p></td><td align="left"><p>3|4 (.07)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq13.gif" /> (.77)</p></td><td align="left"><p>3 (.07)</p></td><td align="left"><p>2|3 (.07)</p></td><td align="left"><p>.02</p></td><td align="left"><p>.00</p></td><td align="left"><p>-0.10</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>6</p></td><td align="left"><p>Canvas</p></td><td align="left"><p>1 (.06)</p></td><td align="left"><p>1 (.06)</p></td><td align="left"><p>2 (.09)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq14.gif" /> (.72)</p></td><td align="left"><p>.07</p></td><td align="left"><p>.01</p></td><td align="left"><p>1.06</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>Simple Loop</p></td><td align="left"><p>7</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>3</mn><mo>×</mo><mn>3</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq15.gif" />)</p></td><td align="left"><p>2 (.12)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq16.gif" /> (.76)</p></td><td align="left"><p>5 (.06)</p></td><td align="left"><p>5 (.02)</p></td><td align="left"><p>.04</p></td><td align="left"><p>.01</p></td><td align="left"><p>3.89</p></td><td align="left"><p>Wrong</p></td><td align="left"><p>Insufficient</p></td><td align="left"><p>Fails to avoid</p></td><td align="left"><p>Fails to pick</p></td><td align="left"><p>Leaves</p></td><td align="left"><p>Other error</p></td></tr><tr><td align="left" /><td align="left"><p>8</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>3</mn><mo>×</mo><mn>3</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq17.gif" />)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq18.gif" /> (.74)</p></td><td align="left"><p>5 (.10)</p></td><td align="left"><p>5 (.08)</p></td><td align="left"><p>1 (.04)</p></td><td align="left"><p>.05</p></td><td align="left"><p>.01</p></td><td align="left"><p>0.89</p></td><td align="left"><p>direction</p></td><td align="left"><p>steps</p></td><td align="left"><p>the obstacle</p></td><td align="left"><p>the object</p></td><td align="left"><p>the grid</p></td><td align="left" /></tr><tr><td align="left" /><td align="left"><p>9</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>3</mn><mo>×</mo><mn>3</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq19.gif" />)</p></td><td align="left"><p>3 (.11)</p></td><td align="left"><p>3 (.06)</p></td><td align="left"><p>6 (.05)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq20.gif" /> (.73)</p></td><td align="left"><p>.05</p></td><td align="left"><p>.02</p></td><td align="left"><p>2.19</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>10</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq21.gif" />)</p></td><td align="left"><p>2 (.09)</p></td><td align="left"><p>2 (.05)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq22.gif" /> (.75)</p></td><td align="left"><p>2 (.04)</p></td><td align="left"><p>.07</p></td><td align="left"><p>.03</p></td><td align="left"><p>0.12</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>11</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq23.gif" />)</p></td><td align="left"><p>4 (.18)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq24.gif" /> (.63)</p></td><td align="left"><p>6 (.10)</p></td><td align="left"><p>6 (.04)</p></td><td align="left"><p>.06</p></td><td align="left"><p>.03</p></td><td align="left"><p>-</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>Nested Loop</p></td><td align="left"><p>12</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq25.gif" />)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq26.gif" /> (.61)</p></td><td align="left"><p>2 (.13)</p></td><td align="left"><p>2 (.05)</p></td><td align="left"><p>2 (.13)</p></td><td align="left"><p>.08</p></td><td align="left"><p>.03</p></td><td align="left"><p>1.56</p></td><td align="left"><p>Wrong</p></td><td align="left"><p>Insufficient</p></td><td align="left"><p>Fails to avoid</p></td><td align="left"><p>Fails to pick</p></td><td align="left"><p>Leaves</p></td><td align="left"><p>Other error</p></td></tr><tr><td align="left" /><td align="left"><p>13</p></td><td align="left"><p>Canvas</p></td><td align="left"><p>1 (.09)</p></td><td align="left"><p>1 (.06)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq27.gif" /> (.68)</p></td><td align="left"><p>1 (.09)</p></td><td align="left"><p>.09</p></td><td align="left"><p>.04</p></td><td align="left"><p>0.09</p></td><td align="left"><p>direction</p></td><td align="left"><p>steps</p></td><td align="left"><p>the obstacle</p></td><td align="left"><p>the object</p></td><td align="left"><p>the grid</p></td><td align="left" /></tr><tr><td align="left" /><td align="left"><p>14</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq28.gif" />)</p></td><td align="left"><p>3 (.26)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq29.gif" /> (.41)</p></td><td align="left"><p>3|5 (.05)</p></td><td align="left"><p>2 (.17)</p></td><td align="left"><p>.11</p></td><td align="left"><p>.05</p></td><td align="left"><p>0.72</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>15</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq30.gif" />)</p></td><td align="left"><p>4 (.32)</p></td><td align="left"><p>2 (.12)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq31.gif" /> (.23)</p></td><td align="left"><p>2 (.21)</p></td><td align="left"><p>.13</p></td><td align="left"><p>.07</p></td><td align="left"><p>7.81</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>16</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq32.gif" />)</p></td><td align="left"><p>4 (.22)</p></td><td align="left"><p>4|5 (.15)</p></td><td align="left"><p>2 (.09)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq33.gif" /> (.38)</p></td><td align="left"><p>.17</p></td><td align="left"><p>.10</p></td><td align="left"><p>-0.24</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>17</p></td><td align="left"><p>Canvas</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq34.gif" /> (.69)</p></td><td align="left"><p>1 (.06)</p></td><td align="left"><p>1 (.04)</p></td><td align="left"><p>1 (.04)</p></td><td align="left"><p>.16</p></td><td align="left"><p>.11</p></td><td align="left"><p>1.59</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>18</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq35.gif" />)</p></td><td align="left"><p>2 (.23)</p></td><td align="left"><p>3 (.15)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq36.gif" /> (.33)</p></td><td align="left"><p>4 (.11)</p></td><td align="left"><p>.17</p></td><td align="left"><p>.11</p></td><td align="left"><p>3.67</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>If-then</p></td><td align="left"><p>19</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>3</mn><mo>×</mo><mn>3</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq37.gif" />)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq38.gif" /> (.51)</p></td><td align="left"><p>1 (.11)</p></td><td align="left"><p>2 (.08)</p></td><td align="left"><p>5 (.11)</p></td><td align="left"><p>.19</p></td><td align="left"><p>.13</p></td><td align="left"><p>2.35</p></td><td align="left"><p>Follows a</p></td><td align="left"><p>Wrong direction</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq39.gif" /></p></td><td align="left"><p>Other error</p></td><td align="left" /></tr><tr><td align="left" /><td align="left"><p>20</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq40.gif" />)</p></td><td align="left"><p>1 (.27)</p></td><td align="left"><p>2 (.14)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq41.gif" /> (.27)</p></td><td align="left"><p>2 (.25)</p></td><td align="left"><p>.25</p></td><td align="left"><p>.15</p></td><td align="left"><p>1.75</p></td><td align="left"><p>simple sequence</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>If-then-else</p></td><td align="left"><p>21</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq42.gif" />)</p></td><td align="left"><p>2 (.15)</p></td><td align="left"><p>3 (.21)</p></td><td align="left"><p>2 (.14)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq43.gif" /> (.25)</p></td><td align="left"><p>.25</p></td><td align="left"><p>.18</p></td><td align="left"><p>0.30</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq44.gif" /></p></td><td align="left"><p>Wrong direction</p></td><td align="left"><p>Insufficient steps</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq45.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq46.gif" /></p></td><td align="left" /></tr><tr><td align="left" /><td align="left"><p>22</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq47.gif" />)</p></td><td align="left"><p>2 (.27)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq48.gif" /> (.25)</p></td><td align="left"><p>2 (.09)</p></td><td align="left"><p>2 (.11)</p></td><td align="left"><p>.28</p></td><td align="left"><p>.20</p></td><td align="left"><p>-0.18</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>While</p></td><td align="left"><p>23</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>3</mn><mo>×</mo><mn>3</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq49.gif" />)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq50.gif" /> (.51)</p></td><td align="left"><p>2 (.11)</p></td><td align="left"><p>2 (.08)</p></td><td align="left"><p>2 (.06)</p></td><td align="left"><p>.24</p></td><td align="left"><p>.22</p></td><td align="left"><p>1.80</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq51.gif" /></p></td><td align="left"><p>Wrong direction</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq52.gif" /></p></td><td align="left"><p>Fails to move from</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo xmlns="">-</mo></math><inline-graphic href="10956_2024_10142_Article_IEq53.gif" /></p></td><td align="left" /></tr><tr><td align="left" /><td align="left"><p>24</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq54.gif" />)</p></td><td align="left"><p>4 (.30)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq55.gif" /> (.26)</p></td><td align="left"><p>2 (.09)</p></td><td align="left"><p>2 (.07)</p></td><td align="left"><p>.28</p></td><td align="left"><p>.25</p></td><td align="left"><p>1.89</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left"><p>starting point</p></td><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>25</p></td><td align="left"><p>Grid (<math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq56.gif" />)</p></td><td align="left"><p>2 (.16)</p></td><td align="left"><p>2 (.13)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mo stretchy="false" xmlns="">✓</mo></math><inline-graphic href="10956_2024_10142_Article_IEq57.gif" /> (.31)</p></td><td align="left"><p>2 (.10)</p></td><td align="left"><p>.31</p></td><td align="left"><p>.31</p></td><td align="left"><p>-</p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr></tbody></table> </ephtml> </p> <p> <emph>Note</emph>. Q: Question; Prop: proportion of missing responses; WD: cumulative proportion of withdrawal from the item onward; Diff: Difference between the sum score of the examinees that answered all the questions to the sum score of those who withdrew after answering item; Wrong direction: Moves in an opposite direction or to a farther place with respect to the target; Insufficient steps: The number of steps are not enough to reach the target; Fails to avoid the obstacle: Passes through the black cat; Fails to pick the object: Does not pick the flower</p> <p>Once the training phase was completed, the teachers implemented their custom educative intervention in at least five lectures during the following months. This intervention was integrated into standard subjects of the curriculum (i.e., mathematics, linguistics, natural and social sciences, physical education, foreign language, and music) following the guide proposed by AI4K12[<reflink idref="bib3" id="ref65">3</reflink>] (Nishida et al., [<reflink idref="bib35" id="ref66">35</reflink>]). In this manner, the CT activities also contributed to the learning of specific content from these subjects. During the implementation phase, the children participated in five to ten CT teaching sessions that included unplugged CT activities. Only the children whose teachers completed both phases of the program were considered in the study.</p> <p>On the other hand, the control group was composed by 105 students recruited with the help of the Education Departments of different Spanish regions. The teachers in the control group also participated voluntarily in the study, but they did not undergo the educational intervention. The only requirement for the students in the control group to participate in the study was the absence of unplugged CT activities in their daily teaching during the current and previous academic courses.</p> <p>After completing the implementation phase in the participating schools, all students, both in the quasi-experimental group and the control group, took the BCTt to measure their CT skills. The BCTt is an instrument that has been widely used in primary school students (Zapata-Cáceres et al., [<reflink idref="bib60" id="ref67">60</reflink>]; Zapata-Cáceres & Fanchamps, [<reflink idref="bib59" id="ref68">59</reflink>]) and has been found to display good reliability in terms of internal consistency, with a Cronbach's alpha coefficient of <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>.</mo><mn>824</mn></mrow></math> </ephtml> (Zapata-Cáceres et al., [<reflink idref="bib60" id="ref69">60</reflink>]). As such, the BCTt may be a reliable measure to assess the CT skills of the students in the current study.</p> <p>Regarding the test administration, the BCTt was taken during a class period, with students having a total of 45 min to answer the questions, in addition to a few minutes of prior explanation about the test scoring. The BCTt is a paper-based test, so each student had an individual printed copy of the test. To ensure that the instructions given to the students before taking the test were as similar as possible in all the schools, a video was recorded[<reflink idref="bib4" id="ref70">4</reflink>] explaining each type of question of the test. This video was made available to teachers so they could show it to their students in class just before they took the test. The teachers were also provided with an action protocol and a document with answers to frequently asked questions.</p> <p>In the end, the educational intervention used in our study was not designed to teach the children any particular concept included in the BCTt. Consequently, we expect that the results from the BCTt indicate their overall CT learning, rather than being affected by specific knowledge acquired from test preparation.</p> <hd id="AN0180550747-7">Data Analysis and Psychometric Validation</hd> <p>We first inspected the data with descriptive statistics (Table 2). These included the proportion in which every option to each item was selected and the cumulative missingness at each item. Table 2 also displays the qualitative characteristics of the items composing the BCTt, which includes their format (i.e., grid vs. canvas) and the particular mistakes that produce the selection of erroneous response alternatives (i.e., error profiles). We identified six error profiles across all the sequences and loop blocks: Moving in an opposite direction or to a farther place with respect to the target (profile 1); taking an insufficient number of steps to reach the target (profile 2); failing to avoid an obstacle (profile 3); failing to pick a required object (profile 4); leaving the grid (profile 5); and committing another kind of error (profile 6). The error profiles identified in the items from the conditional block are more idiosyncratic and do not overlap with the previous ones. Although the error profiles will not be involved in any quantitative analysis, we included these profiles because we think they would help teachers to identify problematic patterns of reasoning in children, facilitating the design of teaching methods focused on mitigating these errors.</p> <p>Graph: Fig. 2 Diagram representation of the statistical model underlying the BCTt-SF. G = General Factor of Computational Concepts; S 1 = Sequence factor; S 2 = Simple Loop factor; S 3 = Nested Loop factor; S 4 = Conditional factor</p> <p>The second step consisted of identifying the dimensionality and structural configuration of the test in the sample. Dimensionality assessment is a particular complex task since model misspecification is an important obstacle towards understanding the underlying organization of the data (MacCallum, [<reflink idref="bib30" id="ref71">30</reflink>]). In other words, all statistical models ignore many of the nuances that explain some item variance (e.g., fatigue, distraction, pressure, acquiescence). As a result, empirical data is messy, difficulting its alignment with theory even when the main features of such a theory are true. Fortunately, Exploratory Graph Analysis (EGA; Golino & Empskamp, [<reflink idref="bib23" id="ref72">23</reflink>]) is a recent dimensionality tool that can be used to uncover the underlying organization of the traits underpinning the responses to a set of items. It employs a regularization procedure designed to <emph>clean</emph> these nuances from the data. More specifically, EGA fits a Gaussian Graphical Model (GGM) with GLASSO regularization (Friedman et al., [<reflink idref="bib16" id="ref73">16</reflink>]; Golino & Epskamp, [<reflink idref="bib23" id="ref74">23</reflink>]) to estimate sparse correlations between the items (i.e., their pair-wise linear association after taking out the shared variance with the remaining items) and then applies a clustering algorithm (e.g., Louvain) to classify each item in different, nonoverlapping groups. The result of this analysis is usually visualized as a graph, where each node is an item, the node color represents a cluster, and the thickness of the edges connecting them indicates the strength of their linear relationship after conditioning on all the remaining items.</p> <p>Once this first dimensionality analysis was completed, a hierarchical version of EGA was performed (hierEGA; Jiménez et al., [<reflink idref="bib26" id="ref75">26</reflink>]). A score on each cluster was computed for each participant, and the GGM with Louvain clustering was applied again using these scores as input. As a result, the clusters computed in the first step (i.e., first-order clusters) were themselves clustered in higher-order clusters. This last step was used to identify the hierarchical organization of the data. In other words, we checked whether the first-order clusters were embedded within one or more higher-order clusters. Overall, we expected to find well-defined first-order clusters composed by items related to sequences, loops, and conditionals, which are the three broad blocks of computational concepts used to design the items of the BCTt, as well as a single higher-order cluster (i.e., general factor) accounting for the relationships between them.</p> <p>To obtain a more fine-grained analysis of the items and to develop a proper item scoring method, Multidimensional Item Response Theory (MIRT) was used. More concretely, we used a bayesian Three-Parameter Logistic (3PL) bi-factor model to estimate the probability of succeeding an item. According to the bayesian 3PL bi-factor model, the conditional probability of an examinee correctly scoring an item <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>j</mi></math> </ephtml> is defined by the inverse logit function (Cai et al., [<reflink idref="bib11" id="ref76">11</reflink>]),</p> <p>1 <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mtable><mtr><mtd columnalign="right"><mrow><msub><mi>p</mi><mi>j</mi></msub><mo>=</mo><msub><mi>c</mi><mi>j</mi></msub><mo>+</mo><mfrac><mrow><mn>1</mn><mo>-</mo><msub><mi>c</mi><mi>j</mi></msub></mrow><mrow><mn>1</mn><mo>+</mo><mo>exp</mo><mo stretchy="false">(</mo><mo>-</mo><mrow><mo stretchy="false">[</mo><msub><mi>θ</mi><mi>g</mi></msub><msub><mi>α</mi><mrow><mi mathvariant="italic">gj</mi></mrow></msub><mo>+</mo><msub><mi>θ</mi><mi>s</mi></msub><msub><mi>α</mi><mi>j</mi></msub><mo>+</mo><msub><mi>d</mi><mi>j</mi></msub><mo stretchy="false">]</mo></mrow><mo>+</mo><mi>G</mi><mo>×</mo><msub><mi>η</mi><mi>j</mi></msub><mo stretchy="false">)</mo></mrow></mfrac><mo>,</mo></mrow></mtd></mtr></mtable></mrow></math> </ephtml></p> <p>Graph</p> <p>where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>c</mi><mi>j</mi></msub></math> </ephtml> is a parameter indicating the basal probability of succeeding item <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>j</mi></math> </ephtml> by guessing, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>θ</mi><mi>g</mi></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>θ</mi><mi>s</mi></msub></math> </ephtml> are the values of the general and specific abilities with respective slope or <emph>discrimination</emph> parameters <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>α</mi><mrow><mi mathvariant="italic">gj</mi></mrow></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>α</mi><mi>j</mi></msub></math> </ephtml> , and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>d</mi><mi>j</mi></msub></math> </ephtml> is a location parameter for item <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>j</mi></math> </ephtml> , usually termed the <emph>easiness</emph> parameter. <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>G</mi></math> </ephtml> is an indicator variable that is 1 if the examinee belongs to the quasi-experimental group and 0 otherwise. This manner, the <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>η</mi><mi>j</mi></msub></math> </ephtml> indicated the difference in location between the item characteristic curves of the quasi-experimental and control groups, taking into account the effect of the educational intervention in the quasi-experimental group. Thus, these <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>η</mi><mi>j</mi></msub></math> </ephtml> parameters are akin to the ones that are commonly used to test for uniform differential item functioning.</p> <p>To ascertain that the model represented accurately the observed data, we performed posterior predictive checks of the total score distribution for the quasi-experimental and control groups and for the probability of succeeding each item. Regarding the model parameter estimates, we flagged guessing estimates higher than.25, as they would imply that the basal probability of correctly answering the item is compatible with choosing a response alternative by chance.</p> <p>Taking together the results of the analyses, we selected three items from each block of computational concepts, focusing on the items that were most informative for the general factor across a range of abilities that were not too high or too low. We also considered desirable that the items loaded on the specific dimensions and did not have high guessing probabilities. The selected items conformed the BCTt-SF, whose underlying bi-factor structure is anticipated in the diagram of Fig. 2. To assess the adequacy of the final 12-item selection, we repeated all the psychometric analyses on the items comprising the BCTt-SF. Finally, to take full advantage of the final statistical model, we created an application in the Shiny platform to automatically score examinees from their patterns of success/failure responses (https://computationalthinkingtests.shinyapps.io/SF-BCTt/). This scoring method was based on Expected a Posteriori Estimation (EAP).</p> <p>In the end, to estimate the effect of the educational intervention on the measured CT skills, we used the posterior samples of the ability estimates for the quasi-experimental and control groups. For each sample of the posterior distribution of the general ability, we computed the Cohen's d between the groups. As a result, we obtained the Expected a Posteriori value of the Cohen's d for the educational intervention and corresponding credible interval at the 95% level.</p> <p>Graph: Fig. 3 Histograms of total scores in the BCTt for the quasi-experimental and control groups</p> <p>Graph: Fig. 4 Proportion of correct responses in the BCTt by item and group. The dotted lines indicate the proportions at.90 and.30, respectively, whereas the segments through each data point indicate the limits of the 95% confidence intervals</p> <p>Graph: Fig. 5 Item depuration of the BCTt. (a): Exploratory graph analysis of the 25 items of the BCTt; (b) Exploratory graph analysis of the 20-item selection from the BCTt; (c) Exploratory graph analysis with the final selection of 12 items composing the BCTt-SF</p> <hd id="AN0180550747-8">Transparency and Openness</hd> <p>All the analyses were performed in the R programming language. For conducting the EGA analyses, we employed the EGAnet package (Golino & Christensen, [<reflink idref="bib21" id="ref77">21</reflink>]), version 1.1.1. We used the package lavaan (Rosseel, [<reflink idref="bib45" id="ref78">45</reflink>]), version 0.6-13, for fitting the CFA models described in the appendix and the package cmdstanr (Gabry & Češnovar, [<reflink idref="bib17" id="ref79">17</reflink>]), version, 0.4.0, to execute the bayesian bi-factor MIRT model. All the materials, including the full data and scripts to reproduce the psychometric analyses, are available in the Open Science Framework repository at https://osf.io/3xn7a/?view_only=7190e194469f44119aa9174b768a406d.</p> <hd id="AN0180550747-9">Results</hd> <p>The results of the analyses carried out with the sample of 700 preschool students are presented next. We first present the results of the preliminary analyses involving descriptive statistics, followed by the results of the dimensionality assessment and bi-factor MIRT model. Finally, we review the psychometric properties of the shortened version of the BCTt.</p> <p>Graph: Fig. 6 Posterior predictive checks for the total scores in the quasi-experimental and control groups</p> <hd id="AN0180550747-10">Preliminary Analyses</hd> <p>The distributions of the total scores were bell-shaped (Fig. 3), slightly right-skewed for the quasi-experimental group and left-skewed for the control group. Neither floor nor ceiling effects were observed. The most common score in the quasi-experimental group was 16, whereas in the control group, it was 12. The quasi-experimental group also presented a higher median and interquartile range (i.e., median = 15; IQR = [<reflink idref="bib11" id="ref80">11</reflink>, 18]) than the control group (i.e., median = 12; IQR = [<reflink idref="bib9" id="ref81">9</reflink>, 16]). The mean scores for the quasi-experimental and control groups were closer, 14.32 and 12.66, respectively. A Welch two samples <emph>t</emph>-test showed that this difference was significant at the 95% confidence level (<emph>t</emph> = 3.09; <emph>p</emph>-value =.0024).</p> <p>The difficulty of the items was also progressive, with the highest proportion of success happening in the sequence block (.79), followed by the simple loop block (.72), the nested loop block (.48), and the block of conditionals (.34). The pattern of proportions of success was similar in the quasi-experimental and control groups, but the former usually presented higher proportions of success (Fig. 4), especially for the items in the sequence and simple loop blocks.</p> <p>We tested the significance of the difference between the proportions of correct responses between the quasi-experimental and control groups ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi mathvariant="italic">diff</mi></mrow></math> </ephtml> ). At a confidence level of 95%, we found significant differences in items 2 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mi>i</mi><mi>f</mi><mi>f</mi><mo>=</mo><mo>.</mo><mn>17</mn><mo>;</mo><msup><mi>χ</mi><mn>2</mn></msup><mo>=</mo><mn>16.32</mn></mrow></math> </ephtml> ; <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo>=</mo><mo>.</mo><mn>000</mn></mrow></math> </ephtml> ), 5 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mi>i</mi><mi>f</mi><mi>f</mi><mo>=</mo><mo>.</mo><mn>15</mn><mo>;</mo><msup><mi>χ</mi><mn>2</mn></msup><mo>=</mo><mn>10.393</mn></mrow></math> </ephtml> ; <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo>=</mo><mo>.</mo><mn>001</mn></mrow></math> </ephtml> ), 6 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mi>i</mi><mi>f</mi><mi>f</mi><mo>=</mo><mo>.</mo><mn>20</mn><mo>;</mo><msup><mi>χ</mi><mn>2</mn></msup><mo>=</mo><mn>16.25</mn></mrow></math> </ephtml> ; <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo>=</mo><mo>.</mo><mn>000</mn></mrow></math> </ephtml> ), 7 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mi>i</mi><mi>f</mi><mi>f</mi><mo>=</mo><mo>.</mo><mn>10</mn><mo>;</mo><msup><mi>χ</mi><mn>2</mn></msup><mo>=</mo><mn>4.40</mn></mrow></math> </ephtml> ; <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo>=</mo><mo>.</mo><mn>036</mn></mrow></math> </ephtml> ), 9 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mi>i</mi><mi>f</mi><mi>f</mi><mo>=</mo><mo>.</mo><mn>12</mn><mo>;</mo><msup><mi>χ</mi><mn>2</mn></msup><mo>=</mo><mn>5.86</mn></mrow></math> </ephtml> ; <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo>=</mo><mo>.</mo><mn>016</mn></mrow></math> </ephtml> ), 10 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mi>i</mi><mi>f</mi><mi>f</mi><mo>=</mo><mo>.</mo><mn>11</mn><mo>;</mo><msup><mi>χ</mi><mn>2</mn></msup><mo>=</mo><mn>4.75</mn></mrow></math> </ephtml> ; <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo>=</mo><mo>.</mo><mn>029</mn></mrow></math> </ephtml> ), 14 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mi>i</mi><mi>f</mi><mi>f</mi><mo>=</mo><mo>.</mo><mn>16</mn><mo>;</mo><msup><mi>χ</mi><mn>2</mn></msup><mo>=</mo><mn>8.87</mn></mrow></math> </ephtml> ; <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo>=</mo><mo>.</mo><mn>003</mn></mrow></math> </ephtml> ), and 16 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>d</mi><mi>i</mi><mi>f</mi><mi>f</mi><mo>=</mo><mo>.</mo><mn>11</mn><mo>;</mo><msup><mi>χ</mi><mn>2</mn></msup><mo>=</mo><mn>4.20</mn></mrow></math> </ephtml> ; <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>p</mi><mo>=</mo><mo>.</mo><mn>040</mn></mrow></math> </ephtml> ), all of them in favor of the quasi-experimental group. Interestingly, no item from the conditional block presented a significant difference in the proportions of correct response between the groups.</p> <p>Four children attained the maximum possible score (i.e., 25), and four obtained the minimum score (i.e, 0). From the latter group, two examinees provided only one (incorrect) answer to the first item and did not respond to the others. This prompted us to take a closer look at missing data. The proportion of missing responses (Prop) was similar between the control and quasi-experimental groups. The overall Prop increased progressively from item 1 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mtext>Prop</mtext><mo>=</mo><mn>0</mn></mrow></math> </ephtml> ) to item 25 ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mtext>Prop</mtext><mo>=</mo><mo>.</mo><mn>31</mn></mrow></math> </ephtml> ) with no steep increase from one item to the next (Table 2). This observation was compatible with examinees progressively withdrawing the test, so we also calculated the cumulative proportion of withdrawal (WD; Table 2) when faced each item. Indeed, this was the main reason behind missing responses. After item 14, most of the missing responses in each of the subsequent items came from examinees that had withdrawn the test. This turning point could be explained by the fact that item 15 was significantly more difficult than all the previous ones, possibly causing despair in examinees with lower scores. Thereby, to check whether test withdrawal was related to item difficulty, we computed the difference in sum scores between the examinees that answered all the items and those who withdrew after answering each item (Diff; Table Table 2). As expected, the highest difference was found after answering item 15, when the examinees that withdrew, on average, had <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>7.81</mn></mrow></math> </ephtml> less correct answers than those who completed the whole test. In most of the remaining items, the ratio was also positive but smaller.</p> <p>Table 3 Expected a posteriori estimates and 95% credible intervals (within parentheses) for the bi-factor MIRT model using the 20-item selection</p> <p> <ephtml> <table frame="hsides" rules="groups"><tbody><tr><td align="left"><p><inline-graphic href="MediaObjects/10956_2024_10142_Figa_HTML.png" /></p></td><td align="left" /></tr></tbody></table> </ephtml> </p> <hd id="AN0180550747-11">Hierarchical Exploratory Graph Analysis</hd> <p>The hierEGA results on the full test found six dimensions that closely resembled the theory upon which the BCTt was developed (Fig. 5a). In addition, the six dimensions were themselves clustered in a single general factor. Only seven problematic items did not fall within their theoretical dimensions. First, the sequence block was split in two clusters, the first containing items 1, 3, 4, 5 and 12 and the second one containing items 2 and 6. Item 12 displayed very weak partial correlations with the remaining items of the nested loop block, whereas items 2 and 6 were canvas-type items and share at least two response alternatives with a similar feature, named taking a wrong direction (error profile 1). These commonalities between items 2 and 6 may have been the reason for their strong association and subsequent grouping into a unique cluster. Second, two items from the nested loop block (i.e., 13 and 17) and two from the conditional block (i.e., 19, 23) merged in a single cluster. Items 13 and 17 were also canvas-type and appeared to be weakly related to the remaining items in the nested loop cluster. An inspection of their response alternatives revealed that they shared the same error profile than the previous canvas-type items (i.e., error profile 1). Items 19 and 23, on the hand, were more similar in that they were built upon a <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>3</mn><mo>×</mo><mn>3</mn></mrow></math> </ephtml> grid, whereas the other items in the conditional block were in a <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>4</mn><mo>×</mo><mn>4</mn></mrow></math> </ephtml> grid. However, the relationship between items 13 and 17 with items 19 and 23 was not obvious, and we could not explain why they grouped into the same cluster. Furthermore, items 19 and 23 appeared to be closely related to the remaining items of the conditional cluster to which they theoretically belong. For these reasons, we decided to keep items 19 and 23 in the forthcoming analyses and discarded all the other problematic items (i.e., 2, 6, 12, 13, and 17). All the remaining items from the sequence block (i.e., 1, 3, 4, 5, and 7), the simple loop block (i.e., 7, 8, 9, 10, and 11), nested loop block (i.e., 14, 15, 16, and 18), and conditional block (i.e., 20, 21, 22, 24, and 25) were grouped into well-differentiated clusters and thus were also considered in the forthcoming analyses.</p> <p>Graph: Fig. 7 Posterior distribution of Cohen's d for the difference between the quasi-experimental and control groups with respect to their mean general factor abilities</p> <p>A subsequent analysis with hierEGA on the reduced 20-item selection test revealed that the conceptual skills of preschool children could be separated into four well-defined dimensions (i.e., sequences, simple loops, nested loops, and conditionals) and a single general ability indicating a general factor of computational concepts. This time, all the items lied on their expected clusters, confirming that this 20-item selection was accurate enough to ensure a theoretically meaningful structure (Fig. 5b). Moreover, a single general dimension was again suggested in this second dimensionality analysis.</p> <p>To further reduce the length of the test, we assessed the properties of the items to retain the most informative ones. For this purpose, we used the bi-factor MIRT model.</p> <hd id="AN0180550747-12">Multidimensional Item Response Theory Model</hd> <p>We fitted the bi-factor MIRT model in Eq. 1 using the 20-item selection of the BCTt.[<reflink idref="bib5" id="ref82">5</reflink>] The model revealed that the location parameters of most of the items from the sequence, simple loop, and nested loop factors were higher in the quasi-experimental group. In particular, for items 1, 3, 5, 7, 8, 9, 10, 14, and 16, the credible intervals for the effect of the intervention either excluded the zero value or was close to excluding it, meaning that it is very probable that these items were easier for the quasi-experimental than for the control group.</p> <p>The posterior predictive checks indicated a good model fit for both the quasi-experimental and control groups. The predicted probabilities of success of the model matched the observed ones or were close to them (Table S3), and the posterior predictive distribution of the total scores resembled those of the quasi-experimental and control groups (Fig. 6).</p> <p>The estimated parameters of the bi-factor MIRT model are presented in Table 3. The discrimination parameter estimates presented higher values on the specific factors for more difficult items and higher discrimination values on the general factor for easier items. The location parameter estimates indicated that the items were most discriminant for children in the modest-to-low percentiles of the sequence and simple loop ability levels and in the moderate-to-high percentiles of the nested loop and conditional ability levels.</p> <p>To compare the quasi-experimental and control groups, we extracted the latent ability estimates of both groups. On average, the quasi-experimental group had higher scores on the general factor than the control group (Fig. 7; Cohen's d = 0.15; 95% credible interval = <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mo>-</mo></math> </ephtml> 0.02 <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mo>-</mo></math> </ephtml> 0.32). Specifically, the probability that the effect was larger than zero was.96. On the other hand, the differences between the specific latent abilities were close to zero.</p> <p>With respect to the probability of succeeding an item by random chance (i.e., guessing parameters), we found estimates ranging from.09 (items 5 and 21) to.49 (item 23). Interestingly, item 19 also had a similar high guessing parameter estimate (.48). This explains why items 23 and 19 were easy in comparison to the remaining items from the conditional factor. On the one hand, the easiness of item 23 could be explained by response alternatives that were too easy to spot. On the other hand, the correct alternative for item 19 was confounded with a simple sequence. In other words, ignoring the conditional statements and simply following the sequential pattern yielded the correct response. Finally, item 1 also displayed a high guessing probability (.31).</p> <p>Based on the overall results, we proceeded to select the best three items from each computational factor. From the sequence factor, we selected items 3, 4, and 5, as they presented good discrimination values of the general factor, are informative for a range of specific abilities higher than that of item 1, and did not display high guessing probabilities (guessing <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>≤</mo><mo>.</mo><mn>21</mn></mrow></math> </ephtml> ). From the simple loop factor, we decided to retain items 9, 10, and 11 by discarding items 7 and 8, which had the largest guessing probabilities in the simple loop factor (i.e., guessing =.17 and guessing =.18, respectively). From the nested loop block, we selected items 14, 16, and 18. We decided to discard item 15 because it was informative for the general factor only for very high-percentile students. Additionally, the descriptive analyses indicated that the abrupt difficulty of item 15 could be responsible for test withdrawal. Finally, we selected items 20, 21, and 22 from the conditional factor because they had medium-to-high discrimination values of the general factor and the remaining items either presented high guessing probabilities (i.e.,.48 and.49 for items 19 and 23, respectively) or were weaker indicators of the general factor (i.e., discrimination = 0.74 for item 24 and discrimination = 1.26 for item 25).</p> <p>Table 4 Expected a posteriori estimates and 95% credible intervals (within parentheses) for the bi-factor MIRT model of the BCTt-SF</p> <p> <ephtml> <table frame="hsides" rules="groups"><tbody><tr><td align="left"><p><inline-graphic href="MediaObjects/10956_2024_10142_Figb_HTML.png" /></p></td><td align="left" /></tr></tbody></table> </ephtml> </p> <hd id="AN0180550747-13">Psychometric Properties of the Short-Form of the Begginers Computational Thinking Test</hd> <p>To confirm the adequate psychometric properties of this final 12-item selection, we repeated the dimensionality assessment with hierEGA (Fig. 5c) and estimated the bi-factor MIRT model in Eq. 1 (Table 4). As expected, hierEGA yielded a graph whose clusters corresponded to the four theoretical computational concepts and a single general factor of computational concepts. The estimated parameters from the bi-factor MIRT model were similar to the ones estimated in the 20-item version. No item had a high guessing probability. Again, with high probability, items 3, 5, 9, 10, 14, and 16 were easier for the quasi-experimental group. The item characteristic and information curves for the BCTt-SF for both the quasi-experimental and control groups are presented in Figures S2-S5 from the Supplemental material.</p> <p>Finally, we used the calibration of the bi-factor MIRT model to create an online application in the Shiny platform that can be used to automatically score examinees. The app can be accessed at https://computationalthinkingtests.shinyapps.io/SF-BCTt/. The user only needs to upload a file in <emph>csv</emph> format with a matrix of dichotomous responses (i.e., 1 for a correct response and 0 for an incorrect response), where the participants are in rows and the items are in columns.</p> <hd id="AN0180550747-14">Discussion</hd> <p>CT is now widely recognized as a fundamental subject that should be incorporated into compulsory education at all grades (Georgiou & Angeli, [<reflink idref="bib20" id="ref83">20</reflink>]; Stamatios, [<reflink idref="bib49" id="ref84">49</reflink>]). An important part of the process of teaching CT is the explanation and application of computational concepts. These concepts extend beyond the realm of technology and are relevant in various aspects of everyday life. In this context, sequences, loops, and conditionals are three of the most popular computational concepts in teaching programs for preschool students (McCormick & Hall, [<reflink idref="bib33" id="ref85">33</reflink>]). However, the instruments that are currently used to assess these concepts are usually created ad-hoc and did not undergo a proper psychometric validation (Tang et al., [<reflink idref="bib51" id="ref86">51</reflink>]). To overcome this lack of reliable CT instruments in preschool research, we developed a short-form version of the BCTt that displayed excellent psychometric qualities. More concretely, we successfully performed item revision to select the best three items from each computational dimension assessed while maintaining optimal reliability properties, yielding a short-form version with 12 items (BCTt-SF).</p> <p>During the data analysis process, we were able to elucidate questions regarding the computational concepts that can be assessed at a preschool age. We found that the four computational concepts measured by the BCTt (i.e., sequences, simple loops, nested loops, and conditionals) were differentially defined, confirming that preschoolers can be differentiated according to these four concepts (e.g., some of them understand the concepts and others do not).</p> <p>We replicated previous research regarding the good performance of preschoolers when dealing with sequences and interpreting simple loops (Angeli & Valanides, [<reflink idref="bib3" id="ref87">3</reflink>]; Relkin & Bers, [<reflink idref="bib37" id="ref88">37</reflink>]; Saxena et al., [<reflink idref="bib46" id="ref89">46</reflink>]), but, more interestingly, we found that nested loops and conditionals were much more difficult and only some preschoolers could understand them. This finding is consistent with previous research showing that preschoolers' ability to display accurate conditional reasoning can vary depending on the specific context of the task (Andrews et al., [<reflink idref="bib2" id="ref90">2</reflink>]; Diamond et al., [<reflink idref="bib12" id="ref91">12</reflink>]).</p> <p>In addition to the four computational concepts identified, a general factor reflecting an overall ability of computational concepts was supported. This result was highly desirable as it justified the computation of a single test score, which is much more reliable than the scores of each specific computational concept. We also found that the items from the sequence and simple loop blocks were more related to the general factors than those from the nested loop and conditional blocks. That is, the general factor seemed to be more involved in the solution of easier rather than more difficult items. This result was sensical because nested loops and conditionals are much more nuanced concepts and hence that may require specific skills to understand them correctly.</p> <p>We also tested the effectiveness of an educational intervention based on unplugged methodologies. Unplugged approaches to teaching CT are crucial in preschool settings because the plugged ones could be too difficult for them: some children may not have enough prior exposure to technological devices and may not adapt well to coding activities within a computational environment. Additionally, economic constraints may limit their access to such technologies. Given these challenges, unplugged methods provide an excellent alternative. Specifically, we determined which computational concepts are more likely to be fostered after an unplugged educational intervention. We found that the quasi-experimental group was consistently superior to the control group in the understanding of sequences, simple, and nested loops. However, the educational intervention seemed to be more beneficial for improving the understanding of easier rather than difficult concepts. The highest improvements were observed for sequences and simple loops, whereas the improvement in the nested loop block was observed to a lesser extent and no improvement was observed for the conditional block. This result suggested that understanding conditional statements may require more specific training.</p> <p>Some limitations are worth noting. First, sample demographics are lacking, so differential item functioning depending on characteristics like sex could not be assessed. Notwithstanding, the fact that the bi-factor model already presented excellent fit indices and the organization of the data largely matched the theoretical dimensionality of the test are signs that these characteristics would not play a major explanatory role in the model.</p> <p>Second, the pattern of missing responses was mostly driven by test withdrawal probably due to both fatigue and discouragement of low-performing examinees. This could introduce some misspecification in the statistical model that we fitted. A possible solution to this could be to generalize the bi-factor model to explicitly model the probability of withdrawal and relate it to the ability of the examinees. Another problem of progressive test withdrawal is that it could prompt systematic correlations between items that are proximal in the test, a phenomenon that was also ignored. Unfortunately, the current study neither provides information on how the results can be linked to specific teaching methods and the teachers' characteristics.</p> <p>Third, we did not elucidate whether the general factor of computational concepts assessed by the BCTt-SF actually corresponds to a measure of general intelligence (i.e., discriminant validity). Previous research showed that CT skills are correlated with measures of intelligence like spatial, reasoning, and problem-solving abilities (Román-González et al., [<reflink idref="bib43" id="ref92">43</reflink>], [<reflink idref="bib44" id="ref93">44</reflink>]) and also relate to students' numerical and verbal abilities (Tsarava et al., [<reflink idref="bib54" id="ref94">54</reflink>]).</p> <p>A final limitation of this work is that the BCTt-SF is focused on computational concepts and practices and the assessment of computational perspectives is lacking (Brennan & Resnick, [<reflink idref="bib8" id="ref95">8</reflink>]). Computational perspectives are abilities that allow children to share information and express ideas, allowing them to teach others, interact, collaborate, discuss, and bring into question the surrounding technological world. In other words, these abilities are the most socially oriented as they bear the purpose of entertaining, educating, engaging with others, and critically question what and how products can be improved. Eventually, the achievement of these goals will yield computational fluency, which is the capacity to understand computational concepts and applying their meaning to express oneself creatively and effortlessly with digital technologies (Stamatios, [<reflink idref="bib48" id="ref96">48</reflink>]). Therefore, additional methods for assessing the progress of computational perspectives and computational fluency are required to get a broader picture of CT skills (Román-González, [<reflink idref="bib41" id="ref97">41</reflink>]).</p> <p>Despite these limitations, the current study has several strengths: we used a sample of 700 children, the largest up to date in any CT research with preschool children. This allowed us to study the competence of children in the most common computational concepts assessed in the literature, named sequences, loops, and conditionals. We replicated previous results that reported that children can solve with high accuracy problems involving simple and complex sequences and we also found that they performed well with simple loops, but presented more difficulties to understand nested loops and conditionals. We also carried out several psychometric analyses to demonstrate the internal validity and adequate reliability of both the 20-item version of the BCTt and the 12-item version composing the BCTt-SF. Additionally, it is crucial to highlight that the educational intervention implemented in our study was not tailored to specifically target the content covered in the BCTt. Therefore, the results obtained from the BCTt should reflect the generalizable properties of CT learning rather than being influenced by specific knowledge gained through test training.</p> <p>The BCTt-SF can serve as a valuable tool to assess the effectiveness of new educational interventions and to compare them. It can provide insights into the efficacy of different approaches in fostering computational concepts. It is important to note that the BCTt-SF can be used to test the efficacy of new educational interventions or compare them in an objective way. It should be noted that the unplugged activities carried out during the educational intervention were not specifically designed to target the content covered in the BCTt (e.g., identifying patterns in a grid). In other words, the higher scores observed in the quasi-experimental group cannot be attributed to the similarity between the unplugged activities of the intervention and the BCTt items.</p> <p>Empirical studies concerning preschool students often use ad-hoc methods for assessing the computational thinking skills of the children (Bakala et al., [<reflink idref="bib4" id="ref98">4</reflink>]; McCormick & Hall, [<reflink idref="bib33" id="ref99">33</reflink>]; Tang et al., [<reflink idref="bib51" id="ref100">51</reflink>]). Given the lack of well-validated instruments for measuring these abilities in preschool students, the BCTt-SF sets an unprecedented benchmark towards the rigorous scoring of preschool students and for testing and comparing educational interventions in their understanding of sequences, loops, and conditionals (El-Hamamsy et al., [<reflink idref="bib13" id="ref101">13</reflink>]; Vourletsis & Politis, [<reflink idref="bib55" id="ref102">55</reflink>]). Whereas multiple tests exist for measuring CT skills in lower-primary school (Li et al., [<reflink idref="bib29" id="ref103">29</reflink>]; Relkin et al., [<reflink idref="bib39" id="ref104">39</reflink>]; Zapata-Cáceres et al., [<reflink idref="bib61" id="ref105">61</reflink>]; Zhang & Wong, [<reflink idref="bib62" id="ref106">62</reflink>]), they are lacking in preeschool settings and do not cover the aforementioned computational concepts (Relkin & Bers, [<reflink idref="bib37" id="ref107">37</reflink>]). Researchers and practitioners will benefit from this new assessment instrument for several reasons: (<reflink idref="bib1" id="ref108">1</reflink>) the BCTt-SF is a short yet reliable instrument; (<reflink idref="bib2" id="ref109">2</reflink>) the relevance and boundaries of the measured dimensions are clearly established; (<reflink idref="bib3" id="ref110">3</reflink>) the scoring system was designed according to the validated underlying statistical model and can be automated via an online application so that researchers and practitioners can seamlessly score their examinees; (<reflink idref="bib4" id="ref111">4</reflink>) the test is unplugged and independent from the activities and games commonly used in educational interventions like the Tangiblek Robotics Program (Bers et al., [<reflink idref="bib6" id="ref112">6</reflink>]), guarding against biases due to familiarity, practice, and coding knowledge.</p> <p>In future research, it would be valuable to explore additional forms of validity for the BCTt-SF, such as convergent and discriminant validity in relation to other computational thinking (CT) concepts. A potential approach could involve utilizing both the BCTt-SF and the TechCheck-K assessments in conjunction to gain a comprehensive understanding of students' CT competence. This would enable the examination of the interrelationships between all the computational concepts being measured. Another aspect worth investigating is the predictive validity of the BCTt-SF. For instance, gathering data on students' performance in standard curriculum subjects and correlating them with their CT scores could provide insights into how CT relates to academic achievement.</p> <p>Finally, for researchers and practitioners planning to use the BCTt-SF in a preschool setting, it is important to administer the test in a physical format (i.e., paper) and to precede the test with examples. For each computational concept, the evaluator should read a representative item aloud, explain how to choose an option, and demonstrate the reasoning behind the correct choice. We also recommend allowing ample time for the examinees to read and answer all the test items. Preschoolers may sometimes lose patience or motivation or may simply withdraw from the test due to fatigue. Therefore, it is crucial to encourage examinees to answer all the items, even if they are unsure of the correct answer. This approach is recommended because the statistical model that we used does not accommodate missing responses, so all answers must be coded as correct or incorrect. This is not problematic, as the model accounts for the likelihood of correct answers occurring by chance.</p> <hd id="AN0180550747-15">Conclusion</hd> <p>An important component of the CT learning curricula is the adequate monitoring of the student progress, for which standardized, valid, and reliable psychometric instruments are required. Unfortunately, instruments with such properties for the assessment of CT skills are lacking in preschool settings. For the first time, we explicitly addressed this issue by developing a 12-item test (BCTt-SF) in a sample of 700 preschool students, obtaining excellent internal validity and reliability properties. Using the best items from the BCTt, we confirmed four well-defined, nonoverlapping factors to be independently assessed in preschoolers (i.e., sequences, simple loops, nested loops, and conditionals) and a single general factor that may reflect an overall competence in the understanding of these computational concepts. From these factors, we found that an educational intervention based on unplugged methodologies may improve the understanding of sequences and simple loops, whereas improving the understanding of nested loops and conditional statements may require more specific training at this early age. Finally, to ensure the achievement of reliable scores for preschool students, the BCTt-SF test was made freely available and can be automatically scored via an online application at https://computationalthinkingtests.shinyapps.io/SF-BCTt/.</p> <hd id="AN0180550747-16">Funding</hd> <p>This research was supported by Grant 22.167L (Ministry of Education and Vocational Training, Spain). Project name: The study and analysis of the benefits in the learning and development of early childhood education and 1st, 2nd, and 3rd grade primary education students through the elaboration and implementation of unplugged activities (without a computer). This study was also supported by the INVESTIGO program within the Recovery, Transformation and Resilience Plan, funded by the European Union (Next Generation EU project). This work is also co-funded by the Erasmus+ project CoTEDI, which is also co-financed by the European Union under the call-key action 2023-1-NL01-KA220-SCH-000152037 - OID E10207981. The work of J. Moreno-León was partially supported by FEDER/Ministry of Science, Innovation and Universities/Junta de Andalucía/State Research Agency/CDTI with the following grants: Data-pl (PID2022-138486OB-I00), TASOVA PLUS research network (RED2022-134337-T), MIDAS (IDI-20230256) and AquaIA (GOPG-SE-23-0011).</p> <hd id="AN0180550747-17">Data Availability</hd> <p>The dataset used in this study and all the files necessary to reproduce the analyses are available at https://osf.io/3xn7a/?view_only=7190e194469f44119aa9174b768a406d.</p> <hd id="AN0180550747-18">Declarations</hd> <p></p> <hd id="AN0180550747-19">Conflict of Interest</hd> <p>The authors declare no competing interests.</p> <hd id="AN0180550747-20">Appendix</hd> <p></p> <hd id="AN0180550747-21">Confirmatory Factor Analyses</hd> <p>We sought to identify the most reliable and informative items. For this purpose, we used the graph obtained with hierEGA to define a model to be fitted with Confirmatory Factor Analysis (CFA).[<reflink idref="bib6" id="ref113">6</reflink>] Factor loadings with an absolute value lower than.30 were considered small, those between.30 and.50 were considerate medium, and larger values were deemed to be high. As the observed scores were dichotomous (1: correct response; 0: wrong response), we used the tetrachoric correlation matrix (i.e., the correlation matrix between the latent continuous scores) to fit the CFA models.[<reflink idref="bib7" id="ref114">7</reflink>]</p> <p>Table 5 Factor loading estimates and 95% confidence intervals (within parentheses) for the correlated factors and bi-factor models using the 20-item selection</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left" /><th align="left"><p>Correlated model</p></th><th align="left" colspan="2"><p>Bi-factor model</p></th></tr><tr><th align="left"><p> Item</p></th><th align="left"><p>Factor</p></th><th align="left"><p>Factor loading</p></th><th align="left"><p>General factor loading</p></th><th align="left"><p>Specific factor loading</p></th></tr></thead><tbody><tr><td align="left"><p>1</p></td><td align="left"><p>Sequence</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.72</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.61</mn><mo>-</mo><mn>0.82</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq96.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.56</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.45</mn><mo>-</mo><mn>0.67</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq97.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.49</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.31</mn><mo>-</mo><mn>0.67</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq98.gif" /></p></td></tr><tr><td align="left"><p>3</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.78</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.69</mn><mo>-</mo><mn>0.86</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq99.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.62</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.53</mn><mo>-</mo><mn>0.71</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq100.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.44</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.28</mn><mo>-</mo><mn>0.61</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq101.gif" /></p></td></tr><tr><td align="left"><p>4</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.73</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.65</mn><mo>-</mo><mn>0.82</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq102.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.57</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.48</mn><mo>-</mo><mn>0.67</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq103.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.46</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.30</mn><mo>-</mo><mn>0.63</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq104.gif" /></p></td></tr><tr><td align="left"><p>5</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.72</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.64</mn><mo>-</mo><mn>0.81</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq105.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.57</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.47</mn><mo>-</mo><mn>0.66</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq106.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.44</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.27</mn><mo>-</mo><mn>0.60</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq107.gif" /></p></td></tr><tr><td align="left"><p>7</p></td><td align="left"><p>Simple loop</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.73</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.66</mn><mo>-</mo><mn>0.81</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq108.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.72</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.64</mn><mo>-</mo><mn>0.80</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq109.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.08</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mo>-</mo><mn>0.14</mn><mo>-</mo><mn>0.30</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq110.gif" /></p></td></tr><tr><td align="left"><p>8</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.81</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.74</mn><mo>-</mo><mn>0.87</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq111.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.82</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.74</mn><mo>-</mo><mn>0.89</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq112.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.01</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mo>-</mo><mn>0.21</mn><mo>-</mo><mn>0.24</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq113.gif" /></p></td></tr><tr><td align="left"><p>9</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.79</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.72</mn><mo>-</mo><mn>0.86</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq114.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.75</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.67</mn><mo>-</mo><mn>0.83</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq115.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.27</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.06</mn><mo>-</mo><mn>0.49</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq116.gif" /></p></td></tr><tr><td align="left"><p>10</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.87</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.82</mn><mo>-</mo><mn>0.93</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq117.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.83</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.75</mn><mo>-</mo><mn>0.91</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq118.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.43</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.13</mn><mo>-</mo><mn>0.72</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq119.gif" /></p></td></tr><tr><td align="left"><p>11</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.61</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.53</mn><mo>-</mo><mn>0.69</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq120.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.57</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.48</mn><mo>-</mo><mn>0.66</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq121.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.28</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.04</mn><mo>-</mo><mn>0.51</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq122.gif" /></p></td></tr><tr><td align="left"><p>14</p></td><td align="left"><p>Nested loop</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.55</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.45</mn><mo>-</mo><mn>0.65</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq123.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.42</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.32</mn><mo>-</mo><mn>0.53</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq124.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.40</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.21</mn><mo>-</mo><mn>0.59</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq125.gif" /></p></td></tr><tr><td align="left"><p>15</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.61</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.50</mn><mo>-</mo><mn>0.72</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq126.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.48</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.36</mn><mo>-</mo><mn>0.60</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq127.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.35</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.16</mn><mo>-</mo><mn>0.54</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq128.gif" /></p></td></tr><tr><td align="left"><p>16</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.73</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.65</mn><mo>-</mo><mn>0.81</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq129.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.58</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.49</mn><mo>-</mo><mn>0.67</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq130.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.45</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.25</mn><mo>-</mo><mn>0.64</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq131.gif" /></p></td></tr><tr><td align="left"><p>18</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.68</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.59</mn><mo>-</mo><mn>0.77</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq132.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.55</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.46</mn><mo>-</mo><mn>0.64</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq133.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.37</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.19</mn><mo>-</mo><mn>0.54</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq134.gif" /></p></td></tr><tr><td align="left"><p>19</p></td><td align="left"><p>Conditional</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.61</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.53</mn><mo>-</mo><mn>0.70</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq135.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.35</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.25</mn><mo>-</mo><mn>0.45</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq136.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.50</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.40</mn><mo>-</mo><mn>0.61</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq137.gif" /></p></td></tr><tr><td align="left"><p>20</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.66</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.57</mn><mo>-</mo><mn>0.76</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq138.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.51</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.41</mn><mo>-</mo><mn>0.62</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq139.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.28</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.16</mn><mo>-</mo><mn>0.41</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq140.gif" /></p></td></tr><tr><td align="left"><p>21</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.76</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.68</mn><mo>-</mo><mn>0.84</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq141.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.45</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.34</mn><mo>-</mo><mn>0.55</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq142.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.62</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.52</mn><mo>-</mo><mn>0.72</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq143.gif" /></p></td></tr><tr><td align="left"><p>22</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.63</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.53</mn><mo>-</mo><mn>0.72</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq144.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.39</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.28</mn><mo>-</mo><mn>0.50</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq145.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.46</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.34</mn><mo>-</mo><mn>0.57</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq146.gif" /></p></td></tr><tr><td align="left"><p>23</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.68</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.60</mn><mo>-</mo><mn>0.75</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq147.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.37</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.28</mn><mo>-</mo><mn>0.47</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq148.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.61</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.51</mn><mo>-</mo><mn>0.71</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq149.gif" /></p></td></tr><tr><td align="left"><p>24</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.54</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.45</mn><mo>-</mo><mn>0.63</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq150.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.23</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.11</mn><mo>-</mo><mn>0.34</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq151.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.61</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.50</mn><mo>-</mo><mn>0.73</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq152.gif" /></p></td></tr><tr><td align="left"><p>25</p></td><td align="left" /><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.65</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.56</mn><mo>-</mo><mn>0.73</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq153.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.34</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.24</mn><mo>-</mo><mn>0.44</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq154.gif" /></p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.60</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mn>0.50</mn><mo>-</mo><mn>0.70</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq155.gif" /></p></td></tr></tbody></table> </ephtml> </p> <p>Two types of models were estimated with CFA: a correlated factor model, in which the first-order clusters (i.e., specific factors) detected by hierEGA were allowed to correlate, and a bi-factor model, a popular method to address within-item multidimensionality by decomposing systematic item variance in a general factor that explains the commonalities between all the items and specific factors that only affect a subset of the items (Reise, [<reflink idref="bib36" id="ref115">36</reflink>]). If a general factor existed according to hierEGA, then we would expect medium or high correlations among all the specific factor correlations from the correlated factors model. If so, the bi-factor model could be used to quantify how much information each item contributes to measure such a general factor. Finding a general factor is very desirable because it would mean that computing a single test score reflecting computational concepts knowledge is possible. Furthermore, the reliability of such a score would be much higher than any of the specific abilities because all the items would be expected to load on this general factor.</p> <p>In the bi-factor model, we assessed the reliability of the general factor score with omega hierarchical ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>ω</mi><mi>H</mi></msub></math> </ephtml> ; Garcia-Garzon et al., [<reflink idref="bib18" id="ref116">18</reflink>]), a statistic indicating the ratio of the variance accounted for by the general factor to the test variance. According to Rodriguez et al. ([<reflink idref="bib40" id="ref117">40</reflink>]), it is considered that values over <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>.</mo><mn>80</mn></mrow></math> </ephtml> indicate that the general factor is a very reliable source to create a single score, as most of the test variance is attributed to it.</p> <p>We used three fit indices to assess the adequacy of the CFA models: The unbiased root mean square residual (uSRMR; Maydeu-Olivares, [<reflink idref="bib32" id="ref118">32</reflink>]), the root mean square error of approximation (RMSEA; Steiger, [<reflink idref="bib50" id="ref119">50</reflink>]), and the Comparative Fit Index (CFI; Bentler, [<reflink idref="bib5" id="ref120">5</reflink>]). The uSRMR measures how close the CFA model predicts the observed correlation matrix between the items, whereas the RMSEA estimates the distance between the expected model correlation predictions and the true ones in the population. Lastly, the CFI is a measure comparing the fit of the estimated model with that of a null model where no latent factors are present. Values of uSRMR and RMSEA between.05 and.08 are usually regarded as indicating an acceptable fit and values lower than.05 indicate a good fit (Maydeu-Olivares, [<reflink idref="bib32" id="ref121">32</reflink>]; McDonald & Ho, [<reflink idref="bib34" id="ref122">34</reflink>]). On the other hand, the cutoff values for the CFI are.90 or greater for an acceptable fit and.95 or greater for a good fit (Hu & Bentler, [<reflink idref="bib25" id="ref123">25</reflink>]).</p> <p>Table 6 Factor correlation estimates in the correlated factors model using the 20-item selection</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Factor</p></th><th align="left"><p>Sequence</p></th><th align="left"><p>Simple loop</p></th><th align="left"><p>Nested loop</p></th><th align="left"><p>Conditional</p></th></tr></thead><tbody><tr><td align="left"><p>Sequence</p></td><td align="left"><p>−</p></td><td align="left"><p>−</p></td><td align="left"><p>−</p></td><td align="left"><p>−</p></td></tr><tr><td align="left"><p>Simple loop</p></td><td align="left"><p>.84</p></td><td align="left"><p>−</p></td><td align="left"><p>−</p></td><td align="left"><p>−</p></td></tr><tr><td align="left"><p>Nested loop</p></td><td align="left"><p>.48</p></td><td align="left"><p>.73</p></td><td align="left"><p>−</p></td><td align="left"><p>−</p></td></tr><tr><td align="left"><p>Conditional</p></td><td align="left"><p>.39</p></td><td align="left"><p>.52</p></td><td align="left"><p>.63</p></td><td align="left"><p>−</p></td></tr></tbody></table> </ephtml> </p> <p>An important caveat regarding these fit indices is that the cutoffs used to assess model's goodness-of-fit are somewhat arbitrary because they were established in specific contexts (e.g., for specific statistical models, estimators, number of items) and may not be adequate in many applied situations. Thereby, we also assessed the adequacy of the CFA models by inspecting the modification indices (MI) to detect possible sources of residual dependencies. MI are <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi>χ</mi><mn>2</mn></msup></math> </ephtml> statistics that indicate the amount of model fit improvement if a parameter that was not estimated was actually estimated in turn. These statistics may identify different types of model misspecification like residual dependencies between some items, items loading on multiple factors, and correlations between originally assumed orthogonal factors. A significant fit improvement is attained when the MI > 3.84, value that corresponds to the 95<emph>th</emph> percentile of the distribution of a <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi>χ</mi><mn>2</mn></msup></math> </ephtml> statistic with one degree of freedom. However, higher values are usually expected because of either sampling error (i.e., the number of false positives may be high due to the many parameters being tested) or external factors not considered in the model. Therefore, we only interpreted the MI that were significant at the 99% confidence level (MI > 6.63).</p> <p>Overall, the CFM presented excellent absolute and relative fit indices (uSRMR[CFM]:.040 (95%CI:.031-.049); RMSEA[CFM]:.019 (95%CI:.006-.028); CFI[CFM]:.994). The CFM revealed medium-to-high factor loadings on all the items (Table 5), ranging from.54 (i.e., item 24 on the conditional factor) to.87 (i.e., item 10 on the simple loop factor). The factor correlation estimates (Fcor; Table 6) indicated that the sequence factor was strongly correlated with the ability to solve simple loops problems (Fcor =.84), which was related to the nested loop factor (Fcor =.73). The latter was also correlated with the conditional factor (Fcor =.63), creating an interdependence among the factors that could be explained by the presence of a single general factor. In fact, the smallest correlation, between the sequence and conditional factors, was medium-sized (Fcor =.39).</p> <p>This result, coupled with the results obtained from the hierEGA, prompted us to estimate item loadings on a general factor by means of the BFM. The BFM also presented excellent fit values (uSRMR[CFM]:.047 (95% CI:.038–.057); RMSEA[CFM]:.026 (95% CI:.017–.035); CFI[CFA]:.990), complying with the standards of goodness-of-fit. The loadings on the general factor ranged from small to high (Table 5), the highest being in the simple loop factor (mean =.74), followed by the sequence factor (mean =.58), nested loop factor (mean =.51), and conditional factor (mean =.38). In other words, the pattern of loadings resembled those of the discrimination parameter estimates in the bi-factor MIRT model, with higher loadings on the specific factors for more difficult items and higher loadings on the general factor for easier items.</p> <p>The reliability of the general factor was adequate ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>ω</mi><mi>H</mi></msub><mo>=</mo><mo>.</mo><mn>79</mn></mrow></math> </ephtml> ), indicating that nearly 80% of the test variance could be summarized in a single general factor score. On the other hand, the loadings on the specific factors were small-to-medium, presenting higher values in the items from the conditional block, and higher in the nested loop block than in the simple loop block.</p> <p>The BFM revealed some candidate items for removal. The weakest-defined specific factor was the simple loop factor, where items 7 and 8 had loadings close to zero (i.e.,. 08 and.01, respectively) and items 9 and 11 displayed low loadings (i.e.,.27 an d.28, respectively). Item 20 also loaded weakly on the conditional factor (i.e.,.28). All the remaining items presented medium or high loadings on their respective specific factors. With regards to the general factor, all the items in the sequence, simple loop, and nested loop blocks had either medium or high loadings on the general factor, whereas most items from the conditional block displayed medium loadings. Only item 24 had a low loading on the general factor (i.e.,.23).</p> <p>Next, we looked at the MI of the BFM to identify possible sources of model misspecification. The MI indicated that the BFM could be significantly improved if the nested loop factor was allowed to correlate with the conditional factor (MI = 52.21). Similarly, the sequence factor could be correlated with both the loop nested factor (MI = 29.20) and the simple loop factor (MI = 20.70), whereas the conditional factor could be correlated with both the simple loop factor (MI = 9.10) and the sequence factor (MI = 7.64). The only factor correlation that could not significantly improve the fit was that between the simple loop and nested loop factors. As it was shown before, withdrawing the test became more common from the nested loop block onward, so residual dependencies between the nested loop and conditional factors could be explained by the presence of these many absent responses. On the other hand, if the correlation between the sequence and nested loop factors was estimated, then it would be negative, which is compatible with the fact that some low-performing children withdrew when arriving at the nested loop block.</p> <p>With regards to cross-loadings, the MIs indicated that estimating many of them could also significantly improve the fit of the bi-factor model. However, we could not figure out an explanation for them (e.g., allowing item 14 to load on the conditional factor would lead to a fit improvement of MI = 18.41, but item 14 does not contain any conditional instruction). Finally, we found residual correlations between items 20 and 18 (MI = 11.92), 19 and 15 (MI = 9.57), 20 and 5 (MI = 7.30), and 23 and 19 (MI = 6.86). After inspecting all these item pairs in search for similarities, we did not found any apparent relationship in terms of content or difficulty except for items 23 and 19. Both of these were much more easier than the remaining items from the conditional factor, possibly because the response alternatives were bad and children could spot the correct answer by guessing.</p> <p>Table 7 Factor loading estimates and 95% confidence intervals (within parentheses) for the bi-factor model of the final 12-item selection (BCTt-SF)</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Item</p></th><th align="left"><p>Factor</p></th><th align="left"><p>General factor loading</p></th><th align="left"><p>Specific factor loading</p></th></tr></thead><tbody><tr><td align="left"><p>3</p></td><td align="left"><p>Sequence</p></td><td align="left"><p>0.62 (0.52, 0.72)</p></td><td align="left"><p>0.45 (0.25, 0.65)</p></td></tr><tr><td align="left"><p>4</p></td><td align="left" /><td align="left"><p>0.57 (0.47, 0.67)</p></td><td align="left"><p>0.46 (0.26, 0.66)</p></td></tr><tr><td align="left"><p>5</p></td><td align="left" /><td align="left"><p>0.56 (0.46, 0.66)</p></td><td align="left"><p>0.45 (0.25, 0.65)</p></td></tr><tr><td align="left"><p>9</p></td><td align="left"><p>Simple loop</p></td><td align="left"><p>0.80 (0.71, 0.89)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.11</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mo>-</mo><mn>0.33</mn><mo>,</mo><mn>0.55</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq161.gif" /></p></td></tr><tr><td align="left"><p>10</p></td><td align="left" /><td align="left"><p>0.85 (0.76, 0.94)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.52</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mo>-</mo><mn>1.14</mn><mo>,</mo><mn>2.17</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq162.gif" /></p></td></tr><tr><td align="left"><p>11</p></td><td align="left" /><td align="left"><p>0.61 (0.50, 0.71)</p></td><td align="left"><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mn>0.14</mn><mspace width="3.33333pt" /><mo stretchy="false">(</mo><mo>-</mo><mn>0.41</mn><mo>,</mo><mn>0.68</mn><mo stretchy="false">)</mo></mrow></math><inline-graphic href="10956_2024_10142_Article_IEq163.gif" /></p></td></tr><tr><td align="left"><p>14</p></td><td align="left"><p>Nested loop</p></td><td align="left"><p>0.42 (0.31, 0.52)</p></td><td align="left"><p>0.50 (0.25, 0.76)</p></td></tr><tr><td align="left"><p>16</p></td><td align="left" /><td align="left"><p>0.58 (0.48, 0.67)</p></td><td align="left"><p>0.42 (0.20, 0.64)</p></td></tr><tr><td align="left"><p>18</p></td><td align="left" /><td align="left"><p>0.52 (0.42, 0.63)</p></td><td align="left"><p>0.34 (0.14, 0.53)</p></td></tr><tr><td align="left"><p>20</p></td><td align="left"><p>Conditional</p></td><td align="left"><p>0.50 (0.39, 0.61)</p></td><td align="left"><p>0.35 (0.18, 0.51)</p></td></tr><tr><td align="left"><p>21</p></td><td align="left" /><td align="left"><p>0.42 (0.31, 0.53)</p></td><td align="left"><p>0.69 (0.43, 0.95)</p></td></tr><tr><td align="left"><p>22</p></td><td align="left" /><td align="left"><p>0.38 (0.26, 0.50)</p></td><td align="left"><p>0.49 (0.28, 0.70)</p></td></tr></tbody></table> </ephtml> </p> <p>To confirm the adequate psychometric properties of this final 12-item selection, we refitted the BFM (Table 7). The BFM displayed again excellent absolute and relative fit indices (uSRMR:.51 (95%CI:.38–.65); RMSEA:.037 (95% CI:.022–.051, CFI: 0.986)) and the reliability of the general factor was also good ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>ω</mi><mi>H</mi></msub><mo>=</mo><mo>.</mo><mn>80</mn></mrow></math> </ephtml> ). Regarding the loadings, only items 9 and 11 had low loadings on their specific factor (i.e., values of.11 and.14 on the simple loop factor, respectively), whereas the remaining item loadings on the general and specific factors were medium or high.</p> <hd id="AN0180550747-22">Bi-Factor MIRT Model</hd> <p>An important detail is that missing responses should not have a parameter accounting for guessing because in these cases, the items were never tried, rendering the probability of succeeding the item to be zero. Furthermore, as many of the missing responses happened because the examinees withdrew the test, we impeded missing values to contribute to the likelihood in the bayesian IRT model. Let <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi mathvariant="bold">Θ</mi></mrow></math> </ephtml> be a parameter-vector collecting all the parameters of the model (<reflink idref="bib1" id="ref124">1</reflink>), so the (Bernoulli) probability mass function of a single observation can be written as</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>P</mi><mrow><mo stretchy="false">(</mo><mi>Y</mi><mo>=</mo><mi>y</mi><mo stretchy="false">|</mo><mrow><mi mathvariant="bold">Θ</mi></mrow><mo stretchy="false">)</mo></mrow><mo>=</mo><mfenced open="{"><mrow><mtable><mtr><mtd columnalign="left"><mrow><mi>p</mi><mo>,</mo></mrow></mtd><mtd columnalign="left"><mrow><mtext>if</mtext><mspace width="4pt" /><mi>y</mi><mo>=</mo><mn>1</mn><mo>;</mo></mrow></mtd></mtr><mtr><mtd columnalign="left"><mrow><mrow /><mn>1</mn><mo>-</mo><mi>p</mi><mo>,</mo></mrow></mtd><mtd columnalign="left"><mrow><mtext>if</mtext><mspace width="4pt" /><mi>y</mi><mo>=</mo><mn>0</mn><mo>;</mo></mrow></mtd></mtr><mtr><mtd columnalign="left"><mrow><mrow /><mn>0</mn><mo>,</mo></mrow></mtd><mtd columnalign="left"><mrow><mtext>if</mtext><mspace width="4pt" /><mi>y</mi><mspace width="4pt" /><mtext>is</mtext><mspace width="4pt" /><mtext>missing</mtext><mo>.</mo></mrow></mtd></mtr></mtable></mrow></mfenced></mrow></math> </ephtml> </p> <p>Graph</p> <p>The posterior distribution of the parameters is then proportional to</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>P</mi><mo stretchy="false">(</mo><mrow><mi mathvariant="bold">Θ</mi></mrow><mo stretchy="false">|</mo><mi>y</mi><mo stretchy="false">)</mo><mo>∝</mo><mi>P</mi><mo stretchy="false">(</mo><mi>y</mi><mo stretchy="false">|</mo><mrow><mi mathvariant="bold">Θ</mi></mrow><mo stretchy="false">)</mo><mi>P</mi><mo stretchy="false">(</mo><mrow><mi mathvariant="bold">Θ</mi></mrow><mo stretchy="false">)</mo><mo>,</mo></mrow></math> </ephtml> </p> <p>Graph</p> <p>where <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>P</mi><mo stretchy="false">(</mo><mrow><mi mathvariant="bold">Θ</mi></mrow><mo stretchy="false">)</mo></mrow></math> </ephtml> is the prior probability of the parameters, a quantity reflecting our epistemic knowledge before seeing the data.</p> <p>We defined priors on all the item discrimination parameters (i.e., <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>α</mi><mi>j</mi></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>α</mi><mrow><mi mathvariant="italic">gj</mi></mrow></msub></math> </ephtml> ) using a half-normal distribution with a location hyperparameter for each factor and one scale hyperparameter. Following the advice of Bürkner ([<reflink idref="bib9" id="ref125">9</reflink>]) of using a distribution with mode at zero and then decreasing the probability density for larger values. The discrimination hyperparameters had another half-normal distribution centered at zero and with standard deviation one. We also chose the same half-normal prior for the standard deviation of the location parameters ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>d</mi><mi>j</mi></msub></math> </ephtml> ) but a standard normal prior over the means for each factor. To resemble our uncertainty about the guessing parameters ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>c</mi><mi>j</mi></msub></math> </ephtml> ), we chose a beta distribution with hyperparameters <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>a</mi><mo>=</mo><mn>1</mn></mrow></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>b</mi><mo>=</mo><mn>3</mn></mrow></math> </ephtml> , meaning that from <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mn>4</mn></mrow></math> </ephtml> trials, we expected one success and three failures if the examinee answers the item without knowing the correct alternative. With this distribution, we a priori expected that all the alternatives were equally attractive. Finally, we assigned a standard normal prior distribution to the new <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mi>η</mi><mi>j</mi></msub></math> </ephtml> parameters.</p> <p>Finally, we fixed the metric of the latent factors with a normal prior distribution with location zero and standard deviation one. All the factors were assumed to be independently distributed. Overall, the full model configuration was</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mtable><mtr><mtd columnalign="right"><mrow><mrow><mo maxsize="1.2em" minsize="1.2em" stretchy="true">(</mo></mrow><msub><mi>μ</mi><msub><mi>α</mi><mi>g</mi></msub></msub><mo>,</mo><msub><mi>μ</mi><msub><mi>α</mi><mi>s</mi></msub></msub><mo>,</mo><msub><mi>σ</mi><mi>α</mi></msub><mo>,</mo><msub><mi>σ</mi><mi>d</mi></msub><mrow><mo maxsize="1.2em" minsize="1.2em" stretchy="true">)</mo></mrow></mrow></mtd><mtd columnalign="left"><mrow><mspace width="-0.166667em" /><mo>∼</mo><mspace width="-0.166667em" /><mtext>Normal</mtext><mrow><mo stretchy="false">(</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo stretchy="false">)</mo></mrow><mo>,</mo><mspace width="0.166667em" /><mrow><mo stretchy="false">(</mo><msub><mi>μ</mi><msub><mi>α</mi><mi>g</mi></msub></msub><mo>,</mo><msub><mi>μ</mi><msub><mi>α</mi><mi>s</mi></msub></msub><mo>,</mo><msub><mi>σ</mi><mi>α</mi></msub><mo>,</mo><msub><mi>σ</mi><mi>d</mi></msub><mo stretchy="false">)</mo></mrow><mspace width="-0.166667em" /><mo>≥</mo><mspace width="-0.166667em" /><mn>0</mn><mo>,</mo></mrow></mtd></mtr><mtr><mtd columnalign="right"><mrow><mrow /><msub><mi>μ</mi><msub><mi>d</mi><mi>s</mi></msub></msub></mrow></mtd><mtd columnalign="left"><mrow><mo>∼</mo><mtext>Normal</mtext><mo stretchy="false">(</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo stretchy="false">)</mo><mo>,</mo></mrow></mtd></mtr><mtr><mtd columnalign="right"><mrow><mrow /><msub><mi>α</mi><mi>j</mi></msub></mrow></mtd><mtd columnalign="left"><mrow><mo>∼</mo><mtext>Normal</mtext><mrow><mo stretchy="false">(</mo><msub><mi>μ</mi><msub><mi>α</mi><mi>s</mi></msub></msub><mo>,</mo><msub><mi>σ</mi><mi>α</mi></msub><mo stretchy="false">)</mo></mrow><mo>,</mo><mspace width="0.166667em" /><msub><mi>α</mi><mi>j</mi></msub><mo>≥</mo><mn>0</mn><mo>,</mo></mrow></mtd></mtr><mtr><mtd columnalign="right"><mrow><mrow /><msub><mi>α</mi><mrow><mi mathvariant="italic">gj</mi></mrow></msub></mrow></mtd><mtd columnalign="left"><mrow><mo>∼</mo><mtext>Normal</mtext><mrow><mo stretchy="false">(</mo><msub><mi>μ</mi><msub><mi>α</mi><mi>g</mi></msub></msub><mo>,</mo><msub><mi>σ</mi><mi>α</mi></msub><mo stretchy="false">)</mo></mrow><mo>,</mo><mspace width="0.166667em" /><msub><mi>α</mi><mrow><mi mathvariant="italic">gj</mi></mrow></msub><mo>≥</mo><mn>0</mn><mo>,</mo></mrow></mtd></mtr><mtr><mtd columnalign="right"><mrow><mrow /><msub><mi>d</mi><mi>j</mi></msub></mrow></mtd><mtd columnalign="left"><mrow><mo>∼</mo><mtext>Normal</mtext><mo stretchy="false">(</mo><msub><mi>μ</mi><msub><mi>d</mi><mi>s</mi></msub></msub><mo>,</mo><msub><mi>σ</mi><mi>d</mi></msub><mo stretchy="false">)</mo><mo>,</mo></mrow></mtd></mtr><mtr><mtd columnalign="right"><mrow><mrow /><msub><mi>η</mi><mi>j</mi></msub></mrow></mtd><mtd columnalign="left"><mrow><mo>∼</mo><mtext>Normal</mtext><mo stretchy="false">(</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo stretchy="false">)</mo><mo>,</mo></mrow></mtd></mtr><mtr><mtd columnalign="right"><mrow><mrow /><msub><mi>c</mi><mi>j</mi></msub></mrow></mtd><mtd columnalign="left"><mrow><mo>∼</mo><mtext>Beta</mtext><mo stretchy="false">(</mo><mn>1</mn><mo>,</mo><mn>3</mn><mo stretchy="false">)</mo><mo>,</mo></mrow></mtd></mtr><mtr><mtd columnalign="right"><mrow><mrow /><mo stretchy="false">(</mo><msub><mi>θ</mi><mi>g</mi></msub><mo>,</mo><msub><mi>θ</mi><mi>s</mi></msub><mo stretchy="false">)</mo></mrow></mtd><mtd columnalign="left"><mrow><mo>∼</mo><mtext>Normal</mtext><mo stretchy="false">(</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo stretchy="false">)</mo><mo>.</mo></mrow></mtd></mtr></mtable></mrow></math> </ephtml> </p> <p>Graph</p> <p>For estimating the bayesian model (<reflink idref="bib1" id="ref126">1</reflink>), we run five chains with a Hamiltoniam Monte Carlo algorithm and extracted 22,000 samples from the posterior distribution in each chain, discarding the first 2000 iterations as warmup. In total, 100,000 samples were extracted from the joint posterior distribution. The model setup, posterior convergence, and results were validated following the statistical workflow in Gelman et al. ([<reflink idref="bib19" id="ref127">19</reflink>]). For instance, we conducted predictive prior checks to ensure that the prior distributions produced meaningful data. More concretely, we generated a prior predictive distribution of the total score. To account for the missing responses, we inserted zeros in the same positions of the simulated datasets than in the real dataset. Figure S1 illustrates this prior predictive check. The mean of the distribution was 10.80, with an standard deviation of 3.58 and interquartile range of [8.70 <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mo>-</mo></math> </ephtml> 13.27]. We deemed these statistics to be a reasonable representation of realistic data.</p> <p>We also inspected the sampling diagnostics of the posterior distributions of the parameters. For assessing the convergence of the samples to the intended posterior distribution, we computed the <emph>Rhat</emph> statistic, the bulk effective sample size ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mtext>ESS</mtext><mrow><mi mathvariant="italic">bulk</mi></mrow></msub></math> </ephtml> ), and the tail effective sample size ( <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mtext>ESS</mtext><mrow><mi mathvariant="italic">tail</mi></mrow></msub></math> </ephtml> ). <emph>Rhat</emph> is a measure of how well the Markov chains mixed. Typically, <emph>Rhat</emph> values less than 1.05 are considered to be good indicators that the Markov chains converged to the same posterior distribution. On the other hand, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mtext>ESS</mtext><mrow><mi mathvariant="italic">bulk</mi></mrow></msub></math> </ephtml> is a measure of sampling efficiency for the EAP estimate of the posterior distribution, whereas <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mtext>ESS</mtext><mrow><mi mathvariant="italic">tail</mi></mrow></msub></math> </ephtml> is the minimum effective sample size for 5% and 95% of the posterior quantiles. Both <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mtext>ESS</mtext><mrow><mi mathvariant="italic">bulk</mi></mrow></msub></math> </ephtml> and <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><msub><mtext>ESS</mtext><mrow><mi mathvariant="italic">tail</mi></mrow></msub></math> </ephtml> should be at least 100 per Markov chain to have reliable estimates of the posterior quantiles.</p> <hd id="AN0180550747-23">Supplementary Information</hd> <p>Below is the link to the electronic supplementary material.</p> <p>Graph: Supplementary file 1 (pdf 8294 KB)</p> <hd id="AN0180550747-24">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0180550747-25"> <title> References </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Aho AV. Computation and computational thinking. The Computer Journal. 2012; 55; 7: 832-835. 10.1093/comjnl/bxs074</bibtext> </blist> <blist> <bibl id="bib2" idref="ref40" type="bt">2</bibl> <bibtext> Andrews G, Halford G, Boyce J. Conditional discrimination in young children: The roles of associative and relational processing. Journal of Experimental Child Psychology. 2012; 112: 84-101. 10.1016/j.jecp.2011.12.004</bibtext> </blist> <blist> <bibl id="bib3" idref="ref32" type="bt">3</bibl> <bibtext> Angeli, C, & Valanides, N. (2020). Developing young children's computational thinking with educational robotics: An interaction effect between gender and scaffolding strategy. Computers in Human Behavior,105, 105954. https://doi.org/10.1016/j.chb.2019.03.018</bibtext> </blist> <blist> <bibl id="bib4" idref="ref43" type="bt">4</bibl> <bibtext> Bakala E, Gerosa A, Hourcade J, Tejera G. Preschool children, robots, and computational thinking: A systematic review. International Journal of Child-Computer Interaction. 2021; 29: 100337. 10.1016/j.ijcci.2021.100337</bibtext> </blist> <blist> <bibl id="bib5" idref="ref82" type="bt">5</bibl> <bibtext> Bentler PM. Comparative fit indexes in structural models. Psychological Bulletin. 1990; 107: 238-246. 10.1037/0033-2909.107.2.238</bibtext> </blist> <blist> <bibl id="bib6" idref="ref33" type="bt">6</bibl> <bibtext> Bers M, Flannery L, Kazakoff ER, Sullivan A. Computational thinking and tinkering: Exploration of an early childhood robotics curriculum. Computers & Education. 2014; 72: 145-157. 10.1016/j.compedu.2013.10.020</bibtext> </blist> <blist> <bibl id="bib7" idref="ref15" type="bt">7</bibl> <bibtext> Bocconi, S, Chioccariello, A, Dettori, G, Ferrari, A, & Engelhardt, K. (2016). Developing computational thinking in compulsory education - Implications for policy and practice. In JRC Publications Repository. https://doi.org/10.2791/792158</bibtext> </blist> <blist> <bibl id="bib8" idref="ref95" type="bt">8</bibl> <bibtext> Brennan, K, & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref81" type="bt">9</bibl> <bibtext> Bürkner, P. -C. (2021). Bayesian item response modeling in R with brms and Stan. Journal of Statistical Software, 100. https://doi.org/10.18637/jss.v100.i05</bibtext> </blist> <blist> <bibtext> Caballero-Gonzalez, Y, & García-Valcárcel, A. (2018). A robotics-based approach to foster programming skills and computational thinking: Pilot experience in the classroom of early childhood education. 41–45. https://doi.org/10.1145/3284179.3284188</bibtext> </blist> <blist> <bibtext> Cai L, Yang JS, Hansen M. Generalized full-information item bifactor analysis. Psychological Methods. 2011; 16: 221-248. 10.1037/a0023350</bibtext> </blist> <blist> <bibtext> Diamond A, Carlson S, Beck D. Preschool children's performance in task switching on the dimensional change card sort task: Separating the dimensions aids the ability to switch. Developmental Neuropsychology. 2005; 28: 689-729. 10.1207/s15326942dn2802_7</bibtext> </blist> <blist> <bibtext> El-Hamamsy, L, Zapata-Cáceres, M, Marcelino, P, Bruno, B, Dehler Zufferey, J, Martín-Barroso, E, & Román-González, M. (2022a). Comparing the psychometric properties of two primary school Computational Thinking (CT) assessments for grades 3 and 4: The Beginners' CT test (BCTt) and the competent CT test (cCTt). Frontiers in Psychology, 13. https://<ulink href="http://www.frontiersin.org/articles/10.3389/fpsyg.2022.1082659">www.frontiersin.org/articles/10.3389/fpsyg.2022.1082659</ulink></bibtext> </blist> <blist> <bibtext> El-Hamamsy L, Zapata-Cáceres M, Barroso EM, Mondada F, Zufferey JD, Bruno B. The competent computational thinking test: Development and validation of an unplugged computational thinking test for upper primary school. Journal of Educational Computing Research. 2022; 60; 7: 1818-1866. 10.1177/07356331221081753</bibtext> </blist> <blist> <bibtext> Elkin M, Sullivan A, Bers M. Programming with the KIBO robotics kit in preschool classrooms. Computers in the Schools. 2016; 33: 169-186. 10.1080/07380569.2016.1216251</bibtext> </blist> <blist> <bibtext> Friedman J, Hastie T, Tibshirani R. Sparse inverse covariance estimation with the graphical lasso. Biostatistics (Oxford, England). 2008; 9; 3: 432-441. 10.1093/biostatistics/kxm045</bibtext> </blist> <blist> <bibtext> Gabry, J, & Češnovar, R. (2021). Cmdstanr: R Interface to 'CmdStan'.</bibtext> </blist> <blist> <bibtext> Garcia-Garzon E, Abad FJ, Garrido LE. Improving bi-factor exploratory modeling. Methodology. 2019; 15; 2: 45-55. 10.1027/1614-2241/a000163</bibtext> </blist> <blist> <bibtext> Gelman, A, Vehtari, A, Simpson, D, Margossian, C, Carpenter, B, Yao, Y, Kennedy, L, Gabry, J, Bürkner, P.-C, & Modrák, M. (2020). Bayesian workflow.</bibtext> </blist> <blist> <bibtext> Georgiou, K, & Angeli, C. (2019). Developing preschool children's Computational Thinking with educational robotics: The role of cognitive differences and scaffolding. 101–108. https://doi.org/10.33965/celda2019_201911L013</bibtext> </blist> <blist> <bibtext> Golino, H, & Christensen, A. P. (2022). EGAnet: Exploratory graph analysis – A framework for estimating the number of dimensions in multivariate data using network psychometrics.</bibtext> </blist> <blist> <bibtext> Golino HF, Demetriou A. Estimating the dimensionality of intelligence like data using Exploratory Graph Analysis. Intelligence. 2017; 62: 54-70. 10.1016/j.intell.2017.02.007</bibtext> </blist> <blist> <bibtext> Golino H, Epskamp S. Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PLOS ONE. 2017; 12; 6: e0174035. 10.1371/journal.pone.0174035</bibtext> </blist> <blist> <bibtext> Guggemos J, Seufert S, Román-González M. Computational thinking assessment - Towards more vivid interpretations. Technology, Knowledge and Learning. 2022. 10.1007/s10758-021-09587-2</bibtext> </blist> <blist> <bibtext> Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999; 6; 1: 1-55. 10.1080/10705519909540118</bibtext> </blist> <blist> <bibtext> Jiménez, M, Abad, F. J, Garcia-Garzon, E, Golino, H, Christensen, A. P, & Garrido, L. E. (2023). Dimensionality assessment in bifactor structures with multiple general factors: A network psychometrics approach. Psychological methods, 10.1037/met0000590. Advance online publication. https://doi.org/10.1037/met0000590</bibtext> </blist> <blist> <bibtext> Kong S-C. A framework of curriculum design for computational thinking development in K-12 education. Journal of Computers in Education. 2016; 3; 4: 377-394. 10.1007/s40692-016-0076-z</bibtext> </blist> <blist> <bibtext> Ling, D. S, Wong, C. D, & Diamond, A. (2021). Children only 3 years old can succeed at conditional "If, Then" reasoning, much earlier than anyone had thought possible. Frontiers in Psychology, 11. https://<ulink href="http://www.frontiersin.org/articles/10.3389/fpsyg.2020.571891">www.frontiersin.org/articles/10.3389/fpsyg.2020.571891</ulink></bibtext> </blist> <blist> <bibtext> Li Y, Xu S, Liu J. Development and validation of computational thinking assessment of Chinese elementary school students. Journal of Pacific Rim Psychology. 2021; 15: 18344909211010240. 10.1177/18344909211010240</bibtext> </blist> <blist> <bibtext> MacCallum RC. Working with imperfect models. Multivariate Behavioral Research. 2003; 38; 1: 113-139. 10.1207/S15327906MBR3801_5</bibtext> </blist> <blist> <bibtext> Manches, A, & Plowman, L. (2015). Computing education in children's early years: A call for debate. British Journal of Educational Technology,48, n/a–n/a. https://doi.org/10.1111/bjet.12355</bibtext> </blist> <blist> <bibtext> Maydeu-Olivares A. Assessing the size of model misfit in structural equation models. Psychometrika. 2017; 82; 3: 533-558. 10.1007/s11336-016-9552-7</bibtext> </blist> <blist> <bibtext> McCormick K, Hall J. Computational thinking learning experiences, outcomes, and research in preschool settings: A scoping review of literature. Education and Information Technologies. 2022; 27: 1-36. 10.1007/s10639-021-10765-z</bibtext> </blist> <blist> <bibtext> McDonald R, Ho M-H. Principles and practice in reporting structural equation analyses. Psychological Methods. 2002; 7: 64-82. 10.1037/1082-989X.7.1.64</bibtext> </blist> <blist> <bibtext> Nishida T, Kanemune S, Idosaka Y, Namiki M, Bell T, Kuno Y. A CS unplugged design pattern. ACM SIGCSE Bulletin. 2009; 41; 1: 231-235. 10.1145/1539024.1508951</bibtext> </blist> <blist> <bibtext> Reise SP. The rediscovery of bifactor measurement models. Multivariate Behavioral Research. 2012; 47; 5: 667-696. 10.1080/00273171.2012.715555</bibtext> </blist> <blist> <bibtext> Relkin, E, & Bers, M. (2021). TechCheck-K: A Measure of computational thinking for kindergarten children. 2021 IEEE Global Engineering Education Conference (EDUCON), 1696–1702. https://doi.org/10.1109/EDUCON46332.2021.9453926</bibtext> </blist> <blist> <bibtext> Relkin, E, Johnson, S. K, & Bers, M. U. (2023). A normative analysis of the TechCheck computational thinking assessment. Educational Technology & Society, 26(2), 118–130. https://<ulink href="http://www.jstor.org/stable/48721000">www.jstor.org/stable/48721000</ulink></bibtext> </blist> <blist> <bibtext> Relkin E, de Ruiter L, Bers MU. TechCheck: Development and validation of an unplugged assessment of computational thinking in early childhood education. Journal of Science Education and Technology. 2020; 29; 4: 482-498. 10.1007/s10956-020-09831-x</bibtext> </blist> <blist> <bibtext> Rodriguez, A, Reise, S, & Haviland, M. (2015). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods,21,. https://doi.org/10.1037/met0000045</bibtext> </blist> <blist> <bibtext> Román-González, M, Moreno-León, J, & Robles, G. (2019). Combining assessment tools for a comprehensive evaluation of Computational Thinking interventions. In S.-C. Kong & H. Abelson (Eds.), Computational thinking education (pp. 79–98). Springer. https://doi.org/10.1007/978-981-13-6528-7_6</bibtext> </blist> <blist> <bibtext> Román-González, M, Pérez-González, J.-C, & Jiménez-Fernández, C. (2015). Test de Pensamiento Computacional: Diseño y psicometría general [Computational Thinking Test: Design & general psychometry]. https://doi.org/10.13140/RG.2.1.3056.5521</bibtext> </blist> <blist> <bibtext> Román-González M, Pérez-González J-C, Jiménez-Fernández C. Which cognitive abilities underlie computational thinking? Criterion validity of the computational thinking test. Computers in Human Behavior. 2017; 72: 678-691. 10.1016/j.chb.2016.08.047</bibtext> </blist> <blist> <bibtext> Román-González M, Pérez-González J-C, Moreno-León J, Robles G. Can computational talent be detected? Predictive validity of the Computational Thinking Test. International Journal of Child-Computer Interaction. 2018; 18: 47-58. 10.1016/j.ijcci.2018.06.004</bibtext> </blist> <blist> <bibtext> Rosseel Y. Lavaan: An R package for structural equation modeling. Journal of Statistical Software. 2012; 48; 2: 1-36. 10.18637/jss.v048.i02</bibtext> </blist> <blist> <bibtext> Saxena A, Lo CK, Hew K, Wong G. Designing unplugged and plugged activities to cultivate computational thinking: An exploratory study in early childhood education. The Asia-Pacific Education Researcher. 2020; 29: 55-66. 10.1007/s40299-019-00478-w</bibtext> </blist> <blist> <bibtext> Shute VJ, Sun C, Asbell-Clarke J. Demystifying computational thinking. Educational Research Review. 2017; 22: 142-158. 10.1016/j.edurev.2017.09.003</bibtext> </blist> <blist> <bibtext> Stamatios, P. (2021). The impact of coding apps to support young children in computational chinking and computational fluency. A literature review. Frontiers in Education, 6. https://<ulink href="http://www.frontiersin.org/articles/10.3389/feduc.2021.657895">www.frontiersin.org/articles/10.3389/feduc.2021.657895</ulink></bibtext> </blist> <blist> <bibtext> Stamatios, P. (2022). Can preschoolers learn computational thinking and coding skills with ScratchJr? A systematic literature review. International Journal of Educational Reform,10567879221076076,. https://doi.org/10.1177/10567879221076077</bibtext> </blist> <blist> <bibtext> Steiger JH. Point estimation, hypothesis testing, and interval estimation using the RMSEA: Some comments and a reply to Hayduk and Glaser. Structural Equation Modeling: A Multidisciplinary Journal. 2000; 7; 2: 149-162. 10.1207/S15328007SEM0702_1</bibtext> </blist> <blist> <bibtext> Tang X, Yin Y, Lin Q, Hadad R, Zhai X. Assessing computational thinking: A systematic review of empirical studies. Computers & Education. 2020; 148: 103798. 10.1016/j.compedu.2019.103798</bibtext> </blist> <blist> <bibtext> Taylor M. Computer programming with PreK-1st grade students with intellectual disabilities. The Journal of Special Education. 2018; 52: 002246691876112. 10.1177/0022466918761120</bibtext> </blist> <blist> <bibtext> Touretzky, D, Martin, F, Seehorn, D, Breazeal, C, & Posner, T. (2019). Special session: AI for K-12 guidelines initiative. Proceedings of the 50th ACM Technical Symposium on Computer Science Education, 492–493. https://doi.org/10.1145/3287324.3287525</bibtext> </blist> <blist> <bibtext> Tsarava K, Moeller K, Román-González M, Golle J, Leifheit L, Butz MV, Ninaus M. A cognitive definition of computational thinking in primary education. Computers & Education. 2022; 179: 104425. 10.1016/j.compedu.2021.104425</bibtext> </blist> <blist> <bibtext> Vourletsis, I, & Politis, P. (2024). Greek translation, cultural adaptation, and psychometric validation of beginners computational thinking test (BCTt). Education and Information Technologies. https://doi.org/10.1007/s10639-024-12887-6</bibtext> </blist> <blist> <bibtext> Wang XC, Choi Y, Benson K, Eggleston C, Weber D. Teacher's role in fostering preschoolers' computational thinking: An exploratory case study. Early Education and Development. 2020; 32: 1-23. 10.1080/10409289.2020.1759012</bibtext> </blist> <blist> <bibtext> Weintrop, D, Wise Rutstein, D, Bienkowski, M, & McGee, S. (2021). Assessing computational thinking: An overview of the field. Computer Science Education,31(2), 113–116. https://doi.org/10.1080/08993408.2021.1918380</bibtext> </blist> <blist> <bibtext> Wing JM. Computational thinking. Communications of the ACM. 2006; 49; 3: 33-35. 10.1145/1118178.1118215</bibtext> </blist> <blist> <bibtext> Zapata-Cáceres, Maria, & Fanchamps, N. (2021). Using the Beginners Computational Thinking Test to measure development on computational concepts among preschoolers: 5th APSCE international conference on computational thinking and stem education 2021. Proceedings of the 5th APSCE International Computational Thinking and STEM in Education Conference 2021, 32–37.</bibtext> </blist> <blist> <bibtext> Zapata-Cáceres, M, Martín-Barroso, E, & Román-González, M. (2020). Computational thinking test for beginners: Design and content validation. 2020 IEEE Global Engineering Education Conference (EDUCON), 1905–1914. https://doi.org/10.1109/EDUCON45650.2020.9125368</bibtext> </blist> <blist> <bibtext> Zapata-Cáceres, M, Martín-Barroso, E, & Román-González, M. (2021). BCTt: Beginners Computational Thinking Test. In Understanding computing education (Vol. 1, pp. 46–56). Raspberry Pi Foundation.</bibtext> </blist> <blist> <bibtext> Zhang S, Wong GKW. Development and validation of a computational thinking test for lower primary school students. Educational Technology Research and Development. 2023. 10.1007/s11423-023-10231-2</bibtext> </blist> </ref> <ref id="AN0180550747-26"> <title> Footnotes </title> <blist> <bibtext> https://intef.es/</bibtext> </blist> <blist> <bibtext> <ulink href="http://www.codeweek.it/cody-roby-en/">http://www.codeweek.it/cody-roby-en/</ulink> </bibtext> </blist> <blist> <bibtext> https://ai4k12.org/</bibtext> </blist> <blist> <bibtext> https://vimeo.com/544462776/34e6be254a</bibtext> </blist> <blist> <bibtext> The diagnostic results for the 20-item selection of the BCTt and the BCTt-SF can be found in Table S1 and Table S2 from the supplemental material, respectively.</bibtext> </blist> <blist> <bibtext> Note that hierEGA is an exploratory tool concerned with the estimation of the number of factors in the data, whereas CFA is confirmatory and provides the measurement model with the regression weights of the factors on the items. As such, they do not provide redundant information and have been recommended to be used in tandem (Golino & Demetriou, [22]).</bibtext> </blist> <blist> <bibtext> We also used exploratory factor analysis (EFA) with the unweighted least squares estimator and target rotation to check the agreement of the loading pattern with the clustering structure of hierEGA. However, many negative cross-loadings were estimated across the structure, wrongly implying that a higher ability level in some traits is detrimental to others. Therefore, we decided to interpret only the results of the CFA models.</bibtext> </blist> </ref> <aug> <p>By Marcos Jiménez; María Zapata-Cáceres; Marcos Román-González; Gregorio Robles; Jesús Moreno-León and Estefanía Martín-Barroso</p> <p>Reported by Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib57" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib47" firstref="ref3"></nolink> <nolink nlid="nl3" bibid="bib58" firstref="ref4"></nolink> <nolink nlid="nl4" bibid="bib14" firstref="ref5"></nolink> <nolink nlid="nl5" bibid="bib37" firstref="ref6"></nolink> <nolink nlid="nl6" bibid="bib48" firstref="ref7"></nolink> <nolink nlid="nl7" bibid="bib20" firstref="ref8"></nolink> <nolink nlid="nl8" bibid="bib49" firstref="ref9"></nolink> <nolink nlid="nl9" bibid="bib27" firstref="ref10"></nolink> <nolink nlid="nl10" bibid="bib31" firstref="ref11"></nolink> <nolink nlid="nl11" bibid="bib46" firstref="ref12"></nolink> <nolink nlid="nl12" bibid="bib56" firstref="ref13"></nolink> <nolink nlid="nl13" bibid="bib44" firstref="ref16"></nolink> <nolink nlid="nl14" bibid="bib52" firstref="ref17"></nolink> <nolink nlid="nl15" bibid="bib38" firstref="ref20"></nolink> <nolink nlid="nl16" bibid="bib60" firstref="ref22"></nolink> <nolink nlid="nl17" bibid="bib61" firstref="ref23"></nolink> <nolink nlid="nl18" bibid="bib51" firstref="ref24"></nolink> <nolink nlid="nl19" bibid="bib33" firstref="ref30"></nolink> <nolink nlid="nl20" bibid="bib15" firstref="ref35"></nolink> <nolink nlid="nl21" bibid="bib12" firstref="ref41"></nolink> <nolink nlid="nl22" bibid="bib28" firstref="ref42"></nolink> <nolink nlid="nl23" bibid="bib29" firstref="ref48"></nolink> <nolink nlid="nl24" bibid="bib39" firstref="ref50"></nolink> <nolink nlid="nl25" bibid="bib42" firstref="ref52"></nolink> <nolink nlid="nl26" bibid="bib62" firstref="ref54"></nolink> <nolink nlid="nl27" bibid="bib13" firstref="ref60"></nolink> <nolink nlid="nl28" bibid="bib53" firstref="ref64"></nolink> <nolink nlid="nl29" bibid="bib35" firstref="ref66"></nolink> <nolink nlid="nl30" bibid="bib59" firstref="ref68"></nolink> <nolink nlid="nl31" bibid="bib30" firstref="ref71"></nolink> <nolink nlid="nl32" bibid="bib23" firstref="ref72"></nolink> <nolink nlid="nl33" bibid="bib16" firstref="ref73"></nolink> <nolink nlid="nl34" bibid="bib26" firstref="ref75"></nolink> <nolink nlid="nl35" bibid="bib11" firstref="ref76"></nolink> <nolink nlid="nl36" bibid="bib21" firstref="ref77"></nolink> <nolink nlid="nl37" bibid="bib45" firstref="ref78"></nolink> <nolink nlid="nl38" bibid="bib17" firstref="ref79"></nolink> <nolink nlid="nl39" bibid="bib43" firstref="ref92"></nolink> <nolink nlid="nl40" bibid="bib54" firstref="ref94"></nolink> <nolink nlid="nl41" bibid="bib41" firstref="ref97"></nolink> <nolink nlid="nl42" bibid="bib55" firstref="ref102"></nolink> <nolink nlid="nl43" bibid="bib36" firstref="ref115"></nolink> <nolink nlid="nl44" bibid="bib18" firstref="ref116"></nolink> <nolink nlid="nl45" bibid="bib40" firstref="ref117"></nolink> <nolink nlid="nl46" bibid="bib32" firstref="ref118"></nolink> <nolink nlid="nl47" bibid="bib50" firstref="ref119"></nolink> <nolink nlid="nl48" bibid="bib34" firstref="ref122"></nolink> <nolink nlid="nl49" bibid="bib25" firstref="ref123"></nolink> <nolink nlid="nl50" bibid="bib19" firstref="ref127"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1446117
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Computational Concepts and Their Assessment in Preschool Students: An Empirical Study
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Marcos+Jiménez%22">Marcos Jiménez</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-4029-6144">0000-0003-4029-6144</externalLink>)<br /><searchLink fieldCode="AR" term="%22María+Zapata-Cáceres%22">María Zapata-Cáceres</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-8817-5889">0000-0002-8817-5889</externalLink>)<br /><searchLink fieldCode="AR" term="%22Marcos+Román-González%22">Marcos Román-González</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-8506-1715">0000-0001-8506-1715</externalLink>)<br /><searchLink fieldCode="AR" term="%22Gregorio+Robles%22">Gregorio Robles</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-1442-6761">0000-0002-1442-6761</externalLink>)<br /><searchLink fieldCode="AR" term="%22Jesús+Moreno-León%22">Jesús Moreno-León</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-3821-5707">0000-0002-3821-5707</externalLink>)<br /><searchLink fieldCode="AR" term="%22Estefanía+Martín-Barroso%22">Estefanía Martín-Barroso</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5652-5592">0000-0001-5652-5592</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Science+Education+and+Technology%22"><i>Journal of Science Education and Technology</i></searchLink>. 2024 33(6):998-1020.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 23
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Computation%22">Computation</searchLink><br /><searchLink fieldCode="DE" term="%22Thinking+Skills%22">Thinking Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Evaluation%22">Student Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Preschool+Children%22">Preschool Children</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Solving%22">Problem Solving</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Tests%22">Cognitive Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Psychometrics%22">Psychometrics</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring%22">Scoring</searchLink><br /><searchLink fieldCode="DE" term="%22Skill+Development%22">Skill Development</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s10956-024-10142-8
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1059-0145<br />1573-1839
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Computational thinking (CT) is a multidimensional term that encompasses a wide variety of problem-solving skills related to the field of computer science. Unfortunately, standardized, valid, and reliable methods to assess CT skills in preschool children are lacking, compromising the reliability of the results reported in CT interventions. To surpass this limitation, we validated in a sample of 700 preschool students (5-6 years old) the Beginners Computational Thinking test Short-Form (BCTt-SF), an unplugged 12-item instrument that measures three of the most common computational concepts assessed in preschool research: sequences, loops, and conditionals. The theoretical model underpinning the BCTt-SF was supported by dimensionality assessment, which suggested that preschool students can be distinguished in terms of four specific abilities (i.e., sequences, simple loops, nested loops, and conditionals) and that all of these abilities were related by a general factor. We modeled this hierarchical structure with a bi-factor model that presented excellent psychometric properties, from good statistical fit indices to adequate reliability of the general ability. To take full advantage of this model, we created an online application in the Shiny platform (https://computationalthinkingtests.shinyapps.io/SF-BCTt/) for the seamless scoring of examinees by any teacher or researcher who uses the BCTt-SF to assess CT skills in preschool children. Finally, we demonstrated how the BCTt-SF can be used to test the impact of educational interventions for improving CT skills in preschoolers.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Note
  Label: Notes
  Group: Note
  Data: https://osf.io/3xn7a/?view_only=7190e194469f44119aa9174b768a406d
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1446117
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1446117
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10956-024-10142-8
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 23
        StartPage: 998
    Subjects:
      – SubjectFull: Computation
        Type: general
      – SubjectFull: Thinking Skills
        Type: general
      – SubjectFull: Student Evaluation
        Type: general
      – SubjectFull: Preschool Children
        Type: general
      – SubjectFull: Problem Solving
        Type: general
      – SubjectFull: Cognitive Tests
        Type: general
      – SubjectFull: Psychometrics
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Scoring
        Type: general
      – SubjectFull: Skill Development
        Type: general
    Titles:
      – TitleFull: Computational Concepts and Their Assessment in Preschool Students: An Empirical Study
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Marcos Jiménez
      – PersonEntity:
          Name:
            NameFull: María Zapata-Cáceres
      – PersonEntity:
          Name:
            NameFull: Marcos Román-González
      – PersonEntity:
          Name:
            NameFull: Gregorio Robles
      – PersonEntity:
          Name:
            NameFull: Jesús Moreno-León
      – PersonEntity:
          Name:
            NameFull: Estefanía Martín-Barroso
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1059-0145
            – Type: issn-electronic
              Value: 1573-1839
          Numbering:
            – Type: volume
              Value: 33
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: Journal of Science Education and Technology
              Type: main
ResultId 1