View in EDS HTML Full Text PDF Full Text

Can Failure Be Made Productive Also in Bayesian Reasoning? A Conceptual Replication Study

Saved in:
Bibliographic Details
Title: Can Failure Be Made Productive Also in Bayesian Reasoning? A Conceptual Replication Study
Language: English
Authors: Katharina Loibl (ORCID 0000-0002-1773-1913), Timo Leuders (ORCID 0000-0002-7621-7826)
Source: Instructional Science: An International Journal of the Learning Sciences. 2025 53(6):1739-1758.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 20
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Instructional Design, Problem Solving, Direct Instruction, Failure, Instructional Effectiveness, Bayesian Statistics, Statistics Education, Undergraduate Students
DOI: 10.1007/s11251-024-09670-y
ISSN: 0020-4277
1573-1952
Abstract: The composite instructional design PS-I combines an initial problem-solving phase (PS) with a subsequent explicit instruction phase (I). PS-I has proven effective for conceptual learning in comparison to instructional designs with the reverse order (I-PS), especially when the explicit instruction phase productively builds on students' erroneous or incomplete (i.e., failed) solution attempts. Building on student solutions during explicit instruction may support students to integrate their intermediate knowledge (acquired during problem solving) with the newly introduced knowledge components. While these effects have been shown for learning the concept of variance in multiple studies, it remains unclear whether these effects generalize to other situations. We conducted a conceptual replication study of Loibl and Rummel (Loibl and Rummel, "Learning and Instruction" 34:74-85, 2014a) choosing Bayesian reasoning as target knowledge. 75 students were assigned to four conditions in a 2 x 2 design (factor 1: PS-I vs. I-PS; factor 2: instruction phase with vs. without typical student solutions). In contrast to Loibl and Rummel (2014a), we did neither find a main effect for PS-I vs. I-PS, nor for building on typical student solutions. The missing effect of PS-I can be explained by the fact that students merely activated their prior knowledge on probabilities without exploring the problem-solving space and without becoming aware of their knowledge gaps. The missing effect of building on typical student solutions can be explained by a mismatch of the solutions generated and the ones included in the explicit instruction. Therefore, building on typical student solutions did not foster an integration of students' intermediate knowledge and the introduced knowledge components.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1493444
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHcagiOSnF7rEuaQkdxFe8EAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDFvWahfRVIDZvsUOdAIBEICBm4bIt-ioEVd-Uzm11fjawYst27xmuXTrYh_blca9P5VBQmhwRV2w46Q6LNmyX1EFie9sjRkXLuRKCwZcAPjoxLev9jES8o5U6P1HuaVpaN7mwvNvCW5vgI6aTRreNgJEXL7txvPFw48bRs5095p5HllTNUPOaceKxQxQDqAYWYfNx73oKJxyrIVkfscoLjl1KQvgTX41DI_Ufusw
Text:
  Availability: 1
  Value: <anid>AN0189211899;isl01dec.25;2025Nov12.05:05;v2.2.500</anid> <title id="AN0189211899-1">Can failure be made productive also in Bayesian reasoning? A conceptual replication study </title> <p>The composite instructional design PS-I combines an initial problem-solving phase (PS) with a subsequent explicit instruction phase (I). PS-I has proven effective for conceptual learning in comparison to instructional designs with the reverse order (I-PS), especially when the explicit instruction phase productively builds on students' erroneous or incomplete (i.e., failed) solution attempts. Building on student solutions during explicit instruction may support students to integrate their intermediate knowledge (acquired during problem solving) with the newly introduced knowledge components. While these effects have been shown for learning the concept of variance in multiple studies, it remains unclear whether these effects generalize to other situations. We conducted a conceptual replication study of Loibl and Rummel (Loibl and Rummel, Learning and Instruction 34:74–85, 2014a) choosing Bayesian reasoning as target knowledge. 75 students were assigned to four conditions in a 2 × 2 design (factor 1: PS-I vs. I-PS; factor 2: instruction phase with vs. without typical student solutions). In contrast to Loibl and Rummel (2014a), we did neither find a main effect for PS-I vs. I-PS, nor for building on typical student solutions. The missing effect of PS-I can be explained by the fact that students merely activated their prior knowledge on probabilities without exploring the problem-solving space and without becoming aware of their knowledge gaps. The missing effect of building on typical student solutions can be explained by a mismatch of the solutions generated and the ones included in the explicit instruction. Therefore, building on typical student solutions did not foster an integration of students' intermediate knowledge and the introduced knowledge components.</p> <p>Keywords: Problem-solving prior to instruction; Conceptual replication; Bayesian reasoning; Composite instructional design; Education Curriculum and Pedagogy Specialist Studies In Education</p> <hd id="AN0189211899-2">Introduction</hd> <p>Which kind of learning is more efficient—developing and trying out your own ideas or applying what is instructed? Of course, there is no either-or answer to this question, but one should seek to find good combinations of both (de Jong et al., [<reflink idref="bib6" id="ref1">6</reflink>]). One of these combinations is problem-solving prior to instruction (PS-I). PS-I is a composite instructional design that combines an initial problem-solving phase (PS) targeting a yet unknown concept with a subsequent explicit instruction phase (I). It has been argued that the initial problem-solving phase prompts students to activate their prior knowledge, makes them generate intuitive ideas about the domain in question (Kapur & Bielaczyc, [<reflink idref="bib11" id="ref2">11</reflink>]), and fosters students' awareness of knowledge gaps as they usually fail to generate a complete and correct solution. Subsequent explicit instruction can then build on students' solutions, also on the erroneous ones (Loibl & Rummel, [<reflink idref="bib18" id="ref3">18</reflink>]), and thereby focus students' attention on the relevant components of the introduced concept (Loibl et al., [<reflink idref="bib17" id="ref4">17</reflink>]) and support students in integrating the newly introduced knowledge components into their existing knowledge base. Since in the problem-solving phase the students typically fail to deliver a complete solution, and since the instruction phase can productively build on these attempts, one also uses the term "productive failure" (Kapur & Bielaczyc, [<reflink idref="bib11" id="ref5">11</reflink>]).</p> <p>Multiple studies have shown beneficial effects of PS-I on the acquisition of conceptual knowledge when compared to the reversed order (explicit instruction prior to problem solving, I-PS) (for a meta-analysis see Sinha & Kapur, [<reflink idref="bib27" id="ref6">27</reflink>]). These effects occur especially when the instructional design supports students in detecting their knowledge gaps and in integrating the new knowledge components into their knowledge base (as revealed by the systematic review in Loibl et al., [<reflink idref="bib17" id="ref7">17</reflink>]).</p> <p>In order to investigate the various postulated mechanisms which are assumed to underly PS-I, one has to conduct studies that specify these mechanisms with respect to the concrete instruction in the two phases, the assumed learning processes, and how they modify the learners' knowledge. We will exemplify this for a study design (Loibl & Rummel, [<reflink idref="bib18" id="ref8">18</reflink>]), which can be considered paradigmatic for many similar studies found in PS-I research (Loibl et al., [<reflink idref="bib17" id="ref9">17</reflink>]; Sinha & Kapur, [<reflink idref="bib27" id="ref10">27</reflink>]). A useful means to systematically describe the elements of such an instructional design and its theoretical assumptions is the CID framework (<emph>cognitive analyses of composite instructional designs</emph>, Loibl et al., [<reflink idref="bib16" id="ref11">16</reflink>]), which requires a theoretical specification on the levels of knowledge, learning, and instruction (cf. KLI framework by Koedinger et al., [<reflink idref="bib12" id="ref12">12</reflink>] for single-phase designs) in a composite instructional design (i.e., an instructional design with multiple phases, such as PS-I). In such composite instructional designs, the interaction of the three levels and the two or more phases are more complex, since the knowledge acquired during the first phase (called intermediate knowledge in the CID framework) can alter the learning processes during the second phase and thereby ultimately affect the learning outcomes.</p> <p>A theoretical description of the PS-I design is illustrated in Fig. 1, upper part: The problem-solving phase is initiated by presenting the problem situation (I<subs>1</subs>) and the problem-solving goal. For example, many PS-I studies in the context of teaching variance, use a task that presents three soccer players and data on their scores per season. The problem-solving goal is to find the most consistent soccer player out of these three players (Kapur, [<reflink idref="bib10" id="ref13">10</reflink>]). During the initial problem-solving phase students activate their prior knowledge (K<subs>0</subs>) to generate solution ideas (L<subs>1</subs>). Typical solution ideas, for instance, are comparing the means or the difference from year-to-year of the three players. As they usually fail to solve the problem correctly, they become aware of the knowledge gaps and acquire knowledge on what does not work (K<subs>1</subs>) (negative knowledge, Oser & Spychiger, [<reflink idref="bib23" id="ref14">23</reflink>]). Moreover, they encode the problem and their solution ideas as examples (Renkl et al., [<reflink idref="bib25" id="ref15">25</reflink>]) (also K<subs>1</subs>). The subsequent phase of explicit instruction then introduces the correct solution (I<subs>2</subs>) (for the problem task presented above: comparing the mean absolute deviation from the mean value or the standard deviation), possibly symbolically represented by a formula. The explicit instruction includes conceptual elements of the target concept (K<subs>2</subs>) (including all values to obtain a precise result, mean as reference point to avoid the impact of the sequence of the data, absolute values of differences from the mean in order that values above and below the mean do not cancel each other out, dividing by the sample size to account for different sample sizes) and the procedure (also K<subs>2</subs>) (calculating the absolute differences from each value to the mean, taking the sum of these differences, dividing the sum by the sample size). Being positioned after the PS phase, such an instruction can also build on student solutions. Students can link (L<subs>2</subs>) the processed explanations to their negative knowledge and encoded examples (K<subs>1</subs>), which is assumed to foster conceptual knowledge (K<subs>2</subs>).</p> <p>Graph: Fig. 1 PS-I (upper part) and I-PS (lower part) analyzed with the CID framework (adapted from Loibl et al., [<reflink idref="bib16" id="ref16">16</reflink>])</p> <p>In contrast, in the I-PS sequence, that is, when the explicit instruction phase comes first (I<subs>1</subs> in Fig. 1 lower part), students process the instructional explanation without linking it to their own knowledge (L<subs>1</subs>), even if instruction builds on typical student solutions. Moreover, during the subsequent problem-solving phase, students do not need to generate their own solutions ideas, but instead apply the learnt procedure (L<subs>2</subs>) (e.g., applying the formula on the problem data). This process in the second phase strengthens students' procedural knowledge, but does not necessarily enhance conceptual knowledge (K<subs>2</subs>) since students typically do not reflect on their knowledge during the execution of procedures without being prompted to do so (see Loibl et al., [<reflink idref="bib16" id="ref17">16</reflink>]).</p> <p>In order to investigate these assumptions on the impact of the instructional design on learning processes and the development of knowledge, Loibl and Rummel ([<reflink idref="bib18" id="ref18">18</reflink>]) systematically varied the factors <emph>order of phases</emph> (PS-I vs. I-PS) and <emph>type of explicit instruction</emph> (with and without building on typical student solutions) and found significant main effects of both factors on conceptual knowledge outcome. Moreover, they also found a significant interaction showing that PS-I is only more beneficial than I-PS if the subsequent instruction builds on typical student solutions (i.e., solutions that learners frequently make during problem-solving). Thus, prior knowledge activation (L<subs>1</subs>) during the problem-solving phase alone does not seem to explain the PS-I effect, students also need the opportunity to integrate (L<subs>2</subs>) the newly introduced knowledge components into their intermediate knowledge base (K<subs>1</subs>), which is supported by explicit instruction that builds on student solutions (I<subs>2</subs>).</p> <p>Loibl and Rummel ([<reflink idref="bib18" id="ref19">18</reflink>]) conducted their study with the learning topic "variance" from the area of descriptive statistics This topic has been used in a majority of studies on PS-I, presumably since it possesses the adequate constituents to elicit mechanisms typical for PS-I, which are the richness of the target knowledge, the accessibility of prior knowledge, and the opportunity for students to approach components of the target knowledge by individual exploration (Loibl et al., [<reflink idref="bib17" id="ref20">17</reflink>]; Sinha & Kapur, [<reflink idref="bib27" id="ref21">27</reflink>]). Another reason for the frequent use of the topic may also be found in the effort of empirical research to achieve coherence and cumulative findings. However, with this dominance of one specific learning topic, it remains unclear whether the effects generalize to other learning topics with a similar information structure. Therefore, the goal of the current study is a conceptual replication with a new learning topic, for which we chose "Bayesian reasoning". In this topic students are asked to determine the (posterior) probability of an event, given the (prior and conditional) probabilities of alternative preceding conditions. When in a concrete situation, this process is carried out correctly, i.e., in accordance with the Bayesian theorem, for example in the context of a written task (word problem), the term "Bayesian reasoning" is used (Gigerenzer & Hoffrage, [<reflink idref="bib9" id="ref22">9</reflink>]; Thevenot & Barrouillet, [<reflink idref="bib28" id="ref23">28</reflink>]).</p> <p>The topic "Bayesian reasoning" shares various characteristics of the "variance" topic as described above (and as evident for example in Brase & Barbey, [<reflink idref="bib3" id="ref24">3</reflink>]). The comparison of the two topics does not primarily relate to the domain (descriptive statistics vs probability theory) but to the structure of the problem situation, the solution process, and the complexity of the target knowledge. The target knowledge as used for the normative solution comprises an arithmetic combination of several pieces of numeric information provided in the problem situation. In both topics this requires learners to find multiple steps of appropriate arithmetic operations (e.g., add values of data points vs. add conjunctive probabilities of outcomes, weight information by dividing by number of data points vs. dividing by summed probabilities). Therefore, the construction of the solution requires to simultaneously account for multiple features of the situation, which when neglected, lead to erroneous incomplete solution methods (e.g., neglecting sample size vs. neglecting prior probabilities). In both situations, the solution is achieved by a concrete method of determining a numerical value (a measure for the variance of the data vs. the conditional probability in question). We concede that there are several ways to construct a variance measure (e.g., mean absolute deviation or standard deviation) while there is only one correct solution for the posterior probability. However, this has no relevance for the learners during the solution process, and in fact, the alternative correct solutions for the variance problem hardly ever occur empirically (Kapur, [<reflink idref="bib10" id="ref25">10</reflink>]). We summarize this comparison in Table 1.</p> <p>Table 1 Comparison of the knowledge structures of the previous study on variance (Loibl & Rummel, [<reflink idref="bib18" id="ref26">18</reflink>]) and of the present study on Bayesian reasoning</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left"><p>Topic</p></th><th align="left"><p>Prior knowledge</p></th><th align="left"><p>Solution attempts</p></th><th align="left"><p>Target knowledge</p></th></tr></thead><tbody><tr><td align="left"><p>Variance as second-order measure of data distribution in descriptive statistics</p></td><td align="left"><p>• Statistical (single-number) measure</p><p>• Concept of average</p><p>• Calculate averages</p></td><td align="left"><p>• Differences with/without reference point</p><p>• Absolute or signed differences</p><p>• Measure with/without normalization </p><p>• Etc</p></td><td align="left"><p>Conceptual:</p><p>Determine average absolute difference from mean as reference point</p><p>Procedural:</p><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mi>D</mi><mo>=</mo><mfrac><mfenced close=")" open="("><mrow><mfenced close="|" open="|"><mrow><mn>9</mn><mo>-</mo><mn>13</mn></mrow></mfenced><mo>+</mo><mfenced close="|" open="|"><mrow><mn>14</mn><mo>-</mo><mn>13</mn></mrow></mfenced><mo>+</mo><mfenced close="|" open="|"><mrow><mn>11</mn><mo>-</mo><mn>13</mn></mrow></mfenced><mo>+</mo><mo>...</mo></mrow></mfenced><mn>10</mn></mfrac></mrow></math><inline-graphic mime-subtype="GIF" href="11251_2024_9670_Article_IEq1.gif" /></p></td></tr><tr><td align="left"><p>Bayesian reasoning as two-step-random process within probability theory</p></td><td align="left"><p>• Concept of probability</p><p>• Translation between probabilities and fractions</p></td><td align="left"><p>• Compare only prior probabilities or only probabilities of data</p><p>• Combine probabilities additively</p><p>• Normalize probabilities</p><p>• Etc</p></td><td align="left"><p>Conceptual:</p><p>Combine prior probabilities with conditional probabilities of data (multiplicatively)</p><p>Procedural:</p><p><math xmlns="http://www.w3.org/1998/Math/MathML"><mrow xmlns=""><mi>P</mi><mo>=</mo><mfrac><mrow><mn>80</mn><mo>%</mo><mo>·</mo><mn>30</mn><mo>%</mo></mrow><mrow><mn>80</mn><mo>%</mo><mo>·</mo><mn>30</mn><mo>%</mo><mo>+</mo><mn>20</mn><mo>%</mo><mo>·</mo><mn>60</mn><mo>%</mo></mrow></mfrac></mrow></math><inline-graphic mime-subtype="GIF" href="11251_2024_9670_Article_IEq2.gif" /></p></td></tr></tbody></table> </ephtml> </p> <p>A conceptual replication tests the same theoretical assumptions as an existing study but varies elements of the research procedure (Crandall & Sherman, [<reflink idref="bib5" id="ref27">5</reflink>]). In our case, we chose to change the learning topic, the corresponding materials, and the age group of the participants. By such variations, conceptual replications represent a test on the generalizability of a finding and the underlying theoretical assumptions, but do not test the operationalization of the original study. Following Nosek and Errington ([<reflink idref="bib22" id="ref28">22</reflink>]), we reflected this distinction and opted for a conceptual replication before conducting the study.</p> <hd id="AN0189211899-3">Conceptual replication study</hd> <p>The present study is a conceptual replication of Loibl and Rummel ([<reflink idref="bib18" id="ref29">18</reflink>]). We substitute various elements of the previous study, in particular the target knowledge and the instructional materials, while maintaining the complexity of the information structure and the assumed learning mechanisms. For the purpose of doing this systematically, we followed the components of the CID framework, as can be seen in the synopsis in Table 2. The characteristic of the prior and the target knowledge of the two studies has already been described above (Table 1).</p> <p>Table 2 Comparison of the original study (Loibl & Rummel, [<reflink idref="bib18" id="ref30">18</reflink>]) and the replication study with regard to target knowledge, instructional design, and learning processes (cf. Figure 1)</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left"><p>Component</p></th><th align="left"><p>Original study</p></th><th align="left"><p>Conceptual replication study</p></th></tr></thead><tbody><tr><td align="left"><p>Target knowledge (K<sub>2</sub>)</p></td><td align="left"><p>Cf. Table 1, upper line</p><p>Components of conceptual knowledge:</p><p>• Consider all values (not only extreme values)</p><p>• Use distances from reference point (average value) for variability measure</p><p>• Use absolute differences (not signed differences)</p><p>• Normalize by number of cases</p></td><td align="left"><p>Cf. Table 1, lower line</p><p>Components of conceptual knowledge:</p><p>• Consider both prior and conditional probabilities (not only one of them)</p><p>• Use multiplicative combination (not additive) for two-step probability</p><p>• Compare two cases (true positive and false negative)</p><p>• Normalize by common probability of two cases</p></td></tr><tr><td align="left"><p>Instructional design in problem-solving phase (I<sub>1</sub>)</p></td><td align="left"><p>"Find a method to compare the three data-sets with respect to consistency."</p></td><td align="left"><p>"Find a method to determine the (posterior) probability for the three cases."</p></td></tr><tr><td align="left"><p>Learning processes in problem-solving phase and resulting intermediate knowledge (I<sub>1</sub>, K<sub>1</sub>)</p></td><td align="left"><p>Activating or generating several (erroneous or only partially adequate) knowledge components for measuring dispersion in given datasets</p><p>Generating negative knowledge based on incorrect solution attempts, e.g., comparing means does not work</p></td><td align="left"><p>Activating or generating several (erroneous or only partially) correct knowledge components for determining posterior probabilities from given prior and conditional probabilities</p><p>Generating negative knowledge based on incorrect solution attempts, e.g., using only conditional probabilities does not work</p></td></tr><tr><td align="left"><p>Instructional design in explicit instruction phase (I<sub>2</sub>)</p></td><td align="left"><p>Explaining error in erroneous solutions (mean instead of deviation; signed instead of absolute deviation) by building on typical student solutions</p><p>Demonstrating complete solution</p></td><td align="left"><p>Explaining error in erroneous solutions (conditional probabilities only; additive instead of multiplicative combination; non-normalized probabilities) by building on typical student solutions</p><p>Demonstrating complete solution</p></td></tr><tr><td align="left"><p>Learning processes in explicit instruction phase (L<sub>2</sub>)</p></td><td align="left"><p>Generating and integrating knowledge components into the conceptual target knowledge</p></td><td align="left"><p>Generating and integrating knowledge components into the conceptual target knowledge</p></td></tr></tbody></table> </ephtml> </p> <p>The topic "Bayesian reasoning", though also taken from the broad domain of "reasoning with data", is curricularly independent from "measuring variance". The former applies knowledge on probabilities, while the latter expands knowledge on descriptive statistics. However, both topics can be considered psychologically equivalent with respect to the knowledge structure (see analysis above, Tables 1 and 2). Moreover, the learning processes during problem solving can be induced by respective instructional designs. The task in the original study presented the dataset as a list of data points. In the case of Bayesian reasoning, one has to carefully choose the representation of the task in a way that learners need to actually construct the step of multiplying prior and conditional probabilities and that they are able to find this solution step by inspecting the given data and accompanying graphical representations (see material below). Also, the typical errors in situations of Bayesian reasoning (Table 3) have been extensively studied and documented (Gigerenzer & Hoffrage, [<reflink idref="bib9" id="ref31">9</reflink>]; Loibl & Leuders, [<reflink idref="bib14" id="ref32">14</reflink>]; Zhu & Gigerenzer, [<reflink idref="bib30" id="ref33">30</reflink>]). These errors particularly occur when students are asked to argue with probabilities (Loibl & Leuders, [<reflink idref="bib15" id="ref34">15</reflink>]; McDowell & Jacobs, [<reflink idref="bib20" id="ref35">20</reflink>]). Similar to the topic of variance, these erroneous solutions occur during a problem-solving phase, contain parts of the complete solution, and can be reflected and integrated in a subsequent explicit instruction phase.</p> <p>Table 3 Correct Bayesian reasoning and typical erroneous solutions</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left"><p>Task & solution type</p></th><th align="left"><p>Example</p></th></tr></thead><tbody><tr><td align="left"><p>Task</p></td><td align="left"><p>Given<italic>:</italic></p><p>80% chance of <italic>H</italic>, (e.g., infection)</p><p>30% chance of <italic>D</italic> given <italic>H</italic>, (e.g., positive test when infected)</p><p>60% chance of <italic>D</italic> given <italic>non-H</italic> (e.g., positive test when not infected)</p><p>Required: What is the chance of <italic>H</italic> when <italic>D</italic>? (e.g., infected when positive test)</p></td></tr><tr><td align="left"><p>Correct solution (Bayesian reasoning)</p></td><td align="left"><p>80% chance of <italic>H</italic> and then 30% chance of <italic>D</italic> = 80%·30% = 24%</p><p>20% chance of <italic>non-</italic><italic>H</italic> and then 60% chance of <italic>D</italic> = 20%·60% = 12%</p><p>Chance for <italic>D</italic>: 24% + 12% = 36%</p><p>Chance of <italic>H</italic> when <italic>D</italic>: 24% / 36% = 67%</p></td></tr><tr><td align="left"><p>Typical erroneous solutions</p></td><td align="left" /></tr><tr><td align="left"><p> a) Missing normalization</p></td><td align="left"><p>a) First steps same as in correct solution</p><p>"Chance of <italic>H</italic> when <italic>D</italic>: 24%"</p></td></tr><tr><td align="left"><p> b) Prior only</p></td><td align="left"><p>b) "Chance of <italic>H</italic> when <italic>D</italic>: 80%"</p></td></tr><tr><td align="left"><p> c) Evidence only</p></td><td align="left"><p>c) "Chance of <italic>H</italic> when <italic>D</italic>: 30%"</p><p>(Possibly with normalization: 30%/(30% + 60%))</p></td></tr><tr><td align="left"><p> d) Averaging priors and evidence</p></td><td align="left"><p>d) Averaging instead of multiplication:</p><p>"Chance of <italic>H</italic> and <italic>D</italic>: (80% + 30%)/2 = 55%", etc.</p><p>(Possibly with normalization: 55%/(55% + 40%))</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0189211899-4">Research question and hypotheses</hd> <p>In our replication study, we—similar to Loibl and Rummel ([<reflink idref="bib18" id="ref36">18</reflink>])—investigate how the learning outcome depends on the order of the instructional phases and on the use of typical student solutions in the explicit instruction phase (cf. the two sequences specified within the CID framework in Fig. 1). Therefore, we implement the same 2 × 2 design (factor 1: PS-I vs. I-PS; factor 2: instruction phase with vs. without typical student solutions) as in Loibl and Rummel ([<reflink idref="bib18" id="ref37">18</reflink>]). To investigate the generalizability of the findings and the underlying theoretical assumptions, we conduct a conceptual replication by transferring the study to a new learning topic with a similar information structure. In so far as the various elements (cf. Table 1) across the two studies can be considered equivalent with respect to the assumed learning processes (cf. L<subs>1</subs> and L<subs>2</subs> within the CID framework in Fig. 1), we intend to replicate the findings of the previous study (Loibl & Rummel, [<reflink idref="bib18" id="ref38">18</reflink>]) with regard to the following hypotheses:</p> <p>Concerning the order of the phases (PS-I vs. I-PS), we hypothesize that.</p> <p></p> <ulist> <item> students in the PS-I conditions outperform their counterparts in the I-PS conditions on conceptual knowledge outcome (hypothesis 1a) as the former activate and differentiate their prior knowledge before receiving instruction.</item> <p></p> <item> students in the I-PS conditions outperform their counterparts in the PS-I condition on procedural knowledge outcome (hypothesis 1b) as only the former have the opportunity to practice the procedure after learning the correct solution during instruction.</item> </ulist> <p>Concerning the type of instruction (explicit instruction phase with vs. without building on typical student solutions), we hypothesize that.</p> <p></p> <ulist> <item> students in the conditions that include typical student solutions in the explicit instruction phase outperform their counterparts where instruction does not build on student solutions on conceptual learning outcome as the former are supported in linking the processed explanations to their existing knowledge (hypothesis 2).</item> </ulist> <p>For conceptual knowledge, we further hypothesize an additive effect, that is, building on typical student solutions in a PS-I condition results in the highest conceptual knowledge outcome compared to all other conditions (hypothesis 3).</p> <p>In addition, we are interested in students' solutions generated during the problem-solving phase:</p> <p></p> <ulist> <item> Do students (similar to the variance task) generate multiple solution attempts during the problem-solving phase in our study?</item> <p></p> <item> Do students generate the typical erroneous solutions (cf. Table 3)?</item> </ulist> <hd id="AN0189211899-5">Methods</hd> <p></p> <hd id="AN0189211899-6">Participants</hd> <p>To determine the required sample size, we conducted a power analysis with g-power (Faul et al., [<reflink idref="bib7" id="ref39">7</reflink>]). Given the medium and large effect sizes found by Loibl and Rummel ([<reflink idref="bib18" id="ref40">18</reflink>]), we used the following parameters: F-Test ANCOVA (fixed effects, main effects, and interactions), effect size = 0.5, alpha =.05, power =.95, df = 1, groups = 4, covariates = 0. The power analysis revealed a required sample size of 55 participants. To account for possible dropouts, we chose a slightly higher sample size. Participants were 75 undergraduate teacher students that were equally distributed across four conditions (cf. Table 4). Participants were recruited within one course of their study program. Participation was voluntary. All participants provided informed consent.</p> <p>Table 4 Experimental conditions</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left" rowspan="2" /><th align="left" colspan="2"><p>Form of instruction</p></th></tr><tr><th align="left"><p>Standard instruction without typical student solutions (I)</p></th><th align="left"><p>Instruction with typical student solutions (I<sup>ts</sup>)</p></th></tr></thead><tbody><tr><td align="left" colspan="3"><p>Order of the phases</p></td></tr><tr><td align="left"><p> Problem-solving prior to instruction</p></td><td align="left"><p>PS-I</p></td><td align="left"><p>PS-I<sup>ts</sup></p></td></tr><tr><td align="left"><p> Instruction prior to problem-solving</p></td><td align="left"><p>I-PS</p></td><td align="left"><p>I<sup>ts</sup>-PS</p></td></tr></tbody></table> </ephtml> </p> <p>Table 5 displays the descriptive statistics of the sample. One participant did not fill in the questionnaire, therefore descriptive data for this participant is missing. Our analyses did not reveal significant differences across conditions with regard to age (<emph>F</emph>(<reflink idref="bib3" id="ref41">3</reflink>,<reflink idref="bib70" id="ref42">70</reflink>) = 0.83, <emph>p </emph>= 0.48), gender distribution (χ<sups>2</sups>(<reflink idref="bib3" id="ref43">3</reflink>, _I_N_i_ = 74) = 0.94, <emph>p </emph>=.82), or familiarity with the topic[<reflink idref="bib1" id="ref44">1</reflink>] (χ<sups>2</sups>(<reflink idref="bib9" id="ref45">9</reflink>, _I_N_i_ = 74) = 7.20, <emph>p </emph>=.62). Moreover, Bayesian statistics (Bayesian ANOVA for age, Bayesian contingency table tests for gender and familiarity) provided substantial to extreme evidence for the null hypotheses of equivalence (age: BF<subs>01</subs> = 5.78, gender: BF<subs>01</subs> = 36.30, familiarity: BF<subs>01</subs> = 2542.80).</p> <p>Table 5 Descriptive statistics of the sample. One student in the PS-I condition did not fill out the questionnaire, but was included in the further analyses, increasing the sample by one (marked as + 1)</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left" /><th align="left"><p>Sample</p></th><th align="left"><p>PS-I</p></th><th align="left"><p>PS-I<sup>ts</sup></p></th><th align="left"><p>I-PS</p></th><th align="left"><p>I<sup>ts</sup>-PS</p></th></tr></thead><tbody><tr><td align="left"><p><italic>N</italic></p></td><td align="left"><p>74 (+ 1)</p></td><td align="left"><p>18 (+ 1)</p></td><td align="left"><p>19</p></td><td align="left"><p>19</p></td><td align="left"><p>18</p></td></tr><tr><td align="left" colspan="5"><p>Age</p></td><td align="left" /></tr><tr><td align="left"><p> Mean (SD)</p></td><td align="left"><p>20.81 (2.82)</p></td><td align="left"><p>20.44 (1.20)</p></td><td align="left"><p>20.95 (2.09)</p></td><td align="left"><p>20.26 (1.37)</p></td><td align="left"><p>21.61 (5.00)</p></td></tr><tr><td align="left" colspan="4"><p>Gender</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p> Female</p></td><td align="left"><p>62</p></td><td align="left"><p>14</p></td><td align="left"><p>16</p></td><td align="left"><p>17</p></td><td align="left"><p>15</p></td></tr><tr><td align="left"><p> Male</p></td><td align="left"><p>12</p></td><td align="left"><p>4</p></td><td align="left"><p>3</p></td><td align="left"><p>2</p></td><td align="left"><p>3</p></td></tr><tr><td align="left"><p> Diverse</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td><td align="left"><p>0</p></td></tr><tr><td align="left" colspan="6"><p>Familiarity with topic</p></td></tr><tr><td align="left"><p> Not at all familiar</p></td><td align="left"><p>41</p></td><td align="left"><p>9</p></td><td align="left"><p>10</p></td><td align="left"><p>12</p></td><td align="left"><p>10</p></td></tr><tr><td align="left"><p> Rather not familiar</p></td><td align="left"><p>28</p></td><td align="left"><p>9</p></td><td align="left"><p>7</p></td><td align="left"><p>5</p></td><td align="left"><p>7</p></td></tr><tr><td align="left"><p> Somewhat familiar</p></td><td align="left"><p>3</p></td><td align="left"><p>0</p></td><td align="left"><p>1</p></td><td align="left"><p>2</p></td><td align="left"><p>0</p></td></tr><tr><td align="left"><p> Very familiar</p></td><td align="left"><p>2</p></td><td align="left"><p>0</p></td><td align="left"><p>1</p></td><td align="left"><p>0</p></td><td align="left"><p>1</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0189211899-7">Experimental conditions</hd> <p>As mentioned above, we implemented four conditions (cf. Table 4) that differed in the order of the phases (PS-I, PS-I<sups>ts</sups> vs. I-PS, I<sups>ts</sups>-PS) and the type of explicit instruction (with or without typical student solutions: I vs. I<sups>ts</sups>). In the problem-solving prior to instruction conditions (PS-I and PS-I<sups>ts</sups>), students first engaged in problem solving. For details on the problem-solving task see Sect. "Learning materials". Afterwards they received explicit instruction. In the instruction prior to problem-solving conditions (I-PS and I<sups>ts</sups>-PS), students first received explicit instruction. Afterwards they solved practice problems that were isomorphic to the ones discussed during the explicit instruction phase.</p> <p>In the conditions without typical student solutions (PS-I and I-PS), the explicit instruction focused on the correct solution only. As in Loibl and Rummel ([<reflink idref="bib18" id="ref46">18</reflink>]), the explicit instruction included conceptual elements of the target concept and the correct procedure, that is, applying Bayesian reasoning to several cases. In the conditions with typical student solutions (PS-I<sups>ts</sups> and I<sups>ts</sups>-PS), the explicit instruction first presented and refuted typical student solutions before introducing the correct solution. For more details regarding the explicit instruction see Sect. "Learning materials".</p> <hd id="AN0189211899-8">Learning materials</hd> <p>The learning topic was Bayesian reasoning. As participants were teacher students, the problems were embedded in a cover story on diagnosing pupils' erroneous strategies for comparing decimals. The exact wording was:<emph>The pupil in front of you has an incorrect strategy for comparing decimals—either SL ("shorter is larger", i.e., the decimal with fewer decimal places is taken as the larger decimal) or WN ("whole numbers", i.e., the digits after the decimal point are interpreted as a whole number). The probability that this pupil (from a specific class) has the SL strategy is high (80%). Thus, the probability that this pupil has the WN strategy is low.</emph><emph>The student is asked to solve a task (for example 4.8 > 4.63). The specific task is not relevant now, only the probabilities for an error matter. The probability that a pupil with the SL strategy makes a mistake here is rather low (30%). The probability that a pupil with the WN strategy makes a mistake here is rather high (60%).</emph><emph>The concrete pupil in front of you solves the task incorrectly.</emph></p> <p></p> <ulist> <item> <emph>What is the probability that this pupil has the erroneous strategy SL?</emph> </item> <p></p> <item> <emph>What is the probability that this pupil has the erroneous strategy WN?</emph> </item> </ulist> <p> <emph>Write down and record everything you think about to determine these probabilities.</emph> </p> <p>This text was accompanied by three bar diagrams with equal length which represented the probability values.</p> <p>In the <emph>problem-solving phase</emph> students received three cases that differed in the prior probabilities and/or the likelihoods and were asked to find the posterior probability (see the example solution in Table 3). Students were also prompted to compare case 1 and 2 as well as case 1 and 3 (see Table 6 for the exact values of the cases). The cases were selected in a way that the comparisons highlight the limitations of typical student solutions. Case 1 and case 2 share the same likelihoods (conditional probabilities), but different prior probabilities. The comparison of these cases highlights the limits of evidence-only strategies and, thus, show the need to include the prior probabilities. Case 3 shares the prior probabilities with case 1 but has uninformative likelihoods. Thus, students have to find a method to combine the prior probabilities and the likelihoods in a way that the posterior probabilities of case 1 differ from the prior probabilities, but for case 3 do not differ. This comparison should highlight the limit of an averaging-strategy which would also alter the probabilities for case 3. In the conditions that started with explicit instruction (I-PS, I<sups>ts</sups>-PS), the values of the cases were slightly modified to avoid the possibility that students would answer the tasks by providing the memorized values of the posterior probabilities instead of applying the learnt solution procedure. However, the structure was the same (i.e., same likelihoods and different prior probabilities for case 1 and 2, uninformative likelihoods for case 3).</p> <p>Table 6 Values of the three cases used in the problem-solving phase. If values were different for I-PS and I<sups>ts</sups>-PS, these values are added in parentheses</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left" /><th align="left"><p>Case 1</p></th><th align="left"><p>Case 2</p></th><th align="left"><p>Case 3</p></th></tr></thead><tbody><tr><td align="left" colspan="4"><p>Prior probabilities</p></td></tr><tr><td align="left"><p> p(SL) = </p></td><td align="left"><p>80%</p></td><td align="left"><p>35%</p></td><td align="left"><p>80%</p></td></tr><tr><td align="left"><p> p(WN) = </p></td><td align="left"><p>20%</p></td><td align="left"><p>65%</p></td><td align="left"><p>20%</p></td></tr><tr><td align="left" colspan="4"><p>Likelihoods for error</p></td></tr><tr><td align="left"><p> p(error | SL) = </p></td><td align="left"><p>30% (15%)</p></td><td align="left"><p>30% (15%)</p></td><td align="left"><p>70% (85%)</p></td></tr><tr><td align="left"><p> p(error | WN) = </p></td><td align="left"><p>60%</p></td><td align="left"><p>60%</p></td><td align="left"><p>70% (85%)</p></td></tr><tr><td align="left" colspan="4"><p>Posterior probabilities (not displayed)</p></td></tr><tr><td align="left"><p> p(SL | error) = </p></td><td align="left"><p>67% (50%)</p></td><td align="left"><p>21% (12%)</p></td><td align="left"><p>80%</p></td></tr><tr><td align="left"><p> p(WN | error) = </p></td><td align="left"><p>33% (50%)</p></td><td align="left"><p>79% (88%)</p></td><td align="left"><p>20%</p></td></tr></tbody></table> </ephtml> </p> <p>We decided not to include further cases to refute further erroneous solution strategies, such as "missing normalization" or "including prior only", since these solution strategies did occur only rarely in a previous study. This study (Loibl & Leuders, [<reflink idref="bib15" id="ref47">15</reflink>]) used analogous tasks and the same context, but—due to its research focus—avoided numerical representations and relied on graphical representations.</p> <p>The <emph>explicit instruction</emph> was given as a video lecture to keep instruction identical across conditions. The videos had a duration of 10 min. The videos introduced the task and explained correct Bayesian reasoning including the correct calculation procedure. In one version of the video (used for PS-I<sups>ts</sups> and I<sups>ts</sups>-PS) two erroneous solutions (evidence only, averaging priors and likelihoods) were presented and refuted before introducing the correct solution. Refutation was done by comparing the different cases and showing that the incorrect solution strategies lead to unreasonable results. In the other version of the video (used for PS-I and I-PS), the correct solution was applied to all three cases.</p> <hd id="AN0189211899-9">Experimental procedure</hd> <p>The experimental procedure included two learning phases—problem solving and explicit instruction—in different order and a written post-test (cf. Figure 2).</p> <p>Graph: Fig. 2 Experimental procedure</p> <p>In the <emph>problem-solving phase</emph>, students individually worked on three cases. For the exact structure and wording of the cases see Sect. "Learning materials". In each condition, students had 30 min to work on all three cases.</p> <p>The <emph>explicit instruction phase</emph> consisted of the presentation of a video with a length of 10 min. For the differences between the used videos in the conditions with and without typical student solutions see Sect. "Learning materials". All students within one condition watched the video together on a large screen. They were not allowed to speak or ask questions.</p> <p>After completing the explicit instruction phase and the problem-solving phase (in either order), students filled in a <emph>written post-test</emph>. Students had 30 min to answer the post-test items. All students finished the post-test on time.</p> <hd id="AN0189211899-10">Learning outcome measure</hd> <p>A post-test assessed the learning outcomes after the second learning phase. The structure of the post-test measures reflected the structure chosen in the original study:</p> <p>The post-test included two items which tested for procedural skills. These items were isomorphic to the ones used in the learning phases (i.e., same wording, different values for the prior probabilities and likelihoods). Thus, students had to apply the learnt solution procedure. Students received one point per correct solution. Cronbach's alpha was (despite the low number of items) very high (0.96). An independent rater coded about 20% of the data (four randomly selected students per condition) with perfect interrater agreement.</p> <p>The post-test included four items testing for conceptual knowledge. Three of these items presented incorrect solutions (evidence only, averaging priors and evidence, missing normalization, cf. Table 3) and asked students to reason mathematically why the solution is correct or incorrect (so called "debugging items"). Students received one point per correct reasoning. One additional item asked students to reason non-numerically; this task provided only verbal descriptions of prior probabilities and likelihoods (very low, rather low, medium, rather high, very high) to impede procedural calculations. Students received one point for correct reasoning. Cronbach's alpha was (given the low number of items) acceptable (0.59). An independent rater coded about 20% of the data (four randomly selected students per condition), interrater reliability was very high (ICC(<reflink idref="bib2" id="ref48">2</reflink>,<reflink idref="bib1" id="ref49">1</reflink>) = 0.96).</p> <hd id="AN0189211899-11">Statistical analyses</hd> <p>To test the requirements, we first compared the four conditions regarding their age, the gender distribution, and the distribution of the familiarity with the topic using classical statistics (ANOVA for age, χ<sups>2</sups> for gender and familiarity). Since we assumed equivalence across conditions in these measures, we conducted Bayesian statistics (Bayesian ANOVA for age, Bayesian contingency table tests for gender and familiarity) to test the probability of the null hypotheses.</p> <p>With regard to our dependent variables—procedural and conceptual knowledge outcomes—we conducted Levene tests to check the homogeneity of variance and Shapiro–Wilk tests to check for normal distribution.</p> <p>To test our hypotheses, we conducted two separate ANOVAs (or rank transformation tests where the criteria of normal distribution were not met) with the two factors <emph>order of phases</emph> and <emph>type of instruction</emph>. One ANOVA with the dependent variable conceptual knowledge outcome tested hypothesis 1a (main effect of order of phases), hypothesis 2 (main effect of type of instruction), and hypothesis 3 (interaction). One ANOVA with the dependent variable procedural knowledge outcome tested hypotheses 1b (main effect of order of phases).</p> <p>All classical statistics were conducted in SPPS (Version 29.0.0.0), the Bayesian analyses with JASP (Version 0.16.4).</p> <hd id="AN0189211899-12">Results</hd> <p></p> <hd id="AN0189211899-13">Learning outcomes</hd> <p>Table 7 displays the descriptive values for the learning outcomes. On average students received 2.4 (of 4 possible points) in the conceptual knowledge items. With regard to the order of phases (hypothesis 1a), students of the two conditions that started with problem solving (PS-I and PS-I<sups>ts</sups>) received 2.4 points (SD = 1.1) and students of the two conditions that started with explicit instruction (I-PS and I<sups>ts</sups>-PS) received 2.3 points (SD = 1.3). With regard to the type of instruction (hypothesis 2), the conditions with typical student solutions in the instruction phase (PS-I<sups>ts</sups> and I<sups>ts</sups>-PS) received 2.5 points (SD = 1.2), the conditions without these solutions (PS-I and I-PS) received 1.3 points (SD = 1.2). As expected in hypothesis 3, with 2.63 points (SD = 1.07) the PS-I<sups>ts</sups> condition performed best. On average students received 1.5 (from 2 possible points) in the procedural knowledge items. With regard to the order of phases (hypothesis 1a), students of the two conditions that started with explicit instruction (I-PS and I<sups>ts</sups>-PS) received 1.5 points (SD = 0.9) and students of the two conditions that started with problem solving (PS-I and PS-I<sups>ts</sups>) received 1.6 points (SD = 0.8).</p> <p>Table 7 Mean values (standard deviation) for learning outcomes</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left" rowspan="2" /><th align="left"><p>Sample</p></th><th align="left"><p>PS-I</p></th><th align="left"><p>PS-I<sup>ts</sup></p></th><th align="left"><p>I-PS</p></th><th align="left"><p>I<sup>ts</sup>-PS</p></th></tr><tr><th align="left"><p>(<italic>N</italic> = 75)</p></th><th align="left"><p>(<italic>N</italic> = 19)</p></th><th align="left"><p>(<italic>N</italic> = 19)</p></th><th align="left"><p>(<italic>N</italic> = 19)</p></th><th align="left"><p>(<italic>N</italic> = 18)</p></th></tr></thead><tbody><tr><td align="left"><p>Conceptual knowledge (max. 4 points)</p></td><td align="left"><p>2.37 (1.19)</p></td><td align="left"><p>2.21 (1.13)</p></td><td align="left"><p>2.63 (1.07)</p></td><td align="left"><p>2.37 (1.26)</p></td><td align="left"><p>2.28 (1.36)</p></td></tr><tr><td align="left"><p>Procedural knowledge (max. 2 points)</p></td><td align="left"><p>1.55 (0.83)</p></td><td align="left"><p>1.63 (0.76)</p></td><td align="left"><p>1.53 (0.84)</p></td><td align="left"><p>1.58 (0.84)</p></td><td align="left"><p>1.44 (0.92)</p></td></tr></tbody></table> </ephtml> </p> <p>The Levene test was not significant for procedural knowledge (<emph>p </emph>=0.53) and conceptual knowledge (<emph>p </emph>= 0.56) indicating homogeneity of variance. However, the Shapiro–Wilk test revealed that the distribution of the values for procedural knowledge significantly deviated from normal distribution (<emph>p </emph><.001 for all conditions). Therefore, following Conover and Iman ([<reflink idref="bib4" id="ref50">4</reflink>]), we conducted a rank transformation test for procedural knowledge with the factors order of phases (PS-I, PS-I<sups>ts</sups> vs. I-PS, I<sups>ts</sups>-PS) and type of instruction (with vs. without typical student solutions: I vs. I<sups>ts</sups>). For conceptual knowledge, the Shapiro–Wilk test suggested normal distribution for all conditions, except for PS-I<sups>ts</sups> (<emph>p </emph>=.01 for PS-I<sups>ts</sups>, <emph>p </emph>>.05 for all other conditions). Given the robustness of the ANOVA, for conceptual knowledge we calculated an ANOVA with the factors order of phases and type of instruction. In addition, we also provide the values with the rank transformation test. The distributions of the procedural and the conceptual knowledge outcome are provided in the appendix.</p> <p>For conceptual knowledge, the ANOVA revealed no significant differences regarding the factor order of phases (hypothesis 1a: <emph>F</emph>(<reflink idref="bib1" id="ref51">1</reflink>,<reflink idref="bib71" id="ref52">71</reflink>) = 0.12, <emph>p </emph>=.73, η<subs>p</subs><sups>2</sups> = 0.002), the factor type of instruction (hypothesis 2: <emph>F</emph>(<reflink idref="bib1" id="ref53">1</reflink>,<reflink idref="bib71" id="ref54">71</reflink>) = 0.35, <emph>p </emph>=.56, η<subs>p</subs><sups>2</sups> = 0.005), or the interaction (hypothesis 3: <emph>F</emph>(<reflink idref="bib1" id="ref55">1</reflink>,<reflink idref="bib71" id="ref56">71</reflink>) = 0.84, <emph>p </emph>=.36, η<subs>p</subs><sups>2</sups> < 0.012). Equally, the rank transformation test revealed no significant differences regarding the factor order of phases (<emph>F</emph>(<reflink idref="bib1" id="ref57">1</reflink>,<reflink idref="bib71" id="ref58">71</reflink>) = 0.06, <emph>p </emph>=.81, η<subs>p</subs><sups>2</sups> = 0.001), the factor type of instruction (<emph>F</emph>(<reflink idref="bib1" id="ref59">1</reflink>,<reflink idref="bib71" id="ref60">71</reflink>) = 0.40, <emph>p </emph>=.53, η<subs>p</subs><sups>2</sups> = 0.006), or the interaction (<emph>F</emph>(<reflink idref="bib1" id="ref61">1</reflink>,<reflink idref="bib71" id="ref62">71</reflink>) = 0.77, <emph>p </emph>=.38, η<subs>p</subs><sups>2</sups> < 0.011). Thus, these results do not provide support for hypothesis 1a that problem-solving prior to instruction would better support the acquisition of conceptual knowledge than the reversed order, nor for hypothesis 2 that including typical student solution in the instruction would better support the acquisition of conceptual knowledge than instruction without typical student solutions, nor for hypothesis 3 than PS-I<sups>ts</sups> would outperform all other conditions.</p> <p>For procedural knowledge, the rank transformation test revealed no significant differences regarding the factor order of phases (hypothesis 1b: <emph>F</emph>(<reflink idref="bib1" id="ref63">1</reflink>,<reflink idref="bib71" id="ref64">71</reflink>) = 0.04,<emph> p </emph>=.84, η<subs>p</subs><sups>2</sups> = 0.001), the factor type of instruction (<emph>F</emph>(<reflink idref="bib1" id="ref65">1</reflink>,<reflink idref="bib71" id="ref66">71</reflink>) = 0.37, <emph>p </emph>=.55, η<subs>p</subs><sups>2</sups> = 0.005), or the interaction (<emph>F(</emph>1,71) = 0.01, <emph>p </emph>=.94, η<subs>p</subs><sups>2</sups> < 0.001). Thus, these results do not provide support for hypothesis 1b that practicing problem solving after instruction would better support the acquisition of procedural knowledge than the reversed order.</p> <hd id="AN0189211899-14">Exploratory analysis of the learning process</hd> <p>In order to gain more insight into the possible reasons for our results, we analysed the learning process. We counted the number of solution attempts in the problem-solving phase, which are regarded as indicator for an effective problem-solving phase (Loibl et al., [<reflink idref="bib17" id="ref67">17</reflink>]), in the two conditions that started with problem solving (PS-I and PS-I<sups>ts</sups>). Out of the 38 students, 31 students generated only one solution attempt, 6 students generated two solution attempts, and 1 student generated three solution attempts.</p> <p>We further investigated the correspondence of students' solution attempts to the ones included in the explicit instruction, as a measure for the alignment of the explicit instruction phase and the problem-solving phase. We therefore checked how many students generated each typical solution and coded whether or not the solution was included in the instruction phase. The frequencies are displayed in Table 8. We will reflect on these exploratory results in the following discussion.</p> <p>Table 8 Erroneous solutions in problem-solving and instruction phase</p> <p> <ephtml> <table rules="groups"><thead><tr><th align="left"><p>Typical student solutions</p></th><th align="left"><p>Number of students</p></th><th align="left"><p>Included in instruction phase</p></th></tr></thead><tbody><tr><td align="left"><p>Missing normalization</p></td><td char="." align="char"><p>33</p></td><td align="left"><p>No</p></td></tr><tr><td align="left"><p>Prior only</p></td><td char="." align="char"><p>7</p></td><td align="left"><p>No</p></td></tr><tr><td align="left"><p>Evidence only</p></td><td char="." align="char"><p>1</p></td><td align="left"><p>Yes</p></td></tr><tr><td align="left"><p>Averaging priors and evidence</p></td><td char="." align="char"><p>1</p></td><td align="left"><p>Yes</p></td></tr><tr><td align="left"><p>Others</p></td><td char="." align="char"><p>4</p></td><td align="left"><p>No</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0189211899-15">Discussion</hd> <p>The present study aimed at a conceptual replication of Loibl and Rummel ([<reflink idref="bib18" id="ref68">18</reflink>]) with a different topic "Bayesian reasoning". The study is no direct replication, since we changed the learning topic, the corresponding materials, and the age group of the participants (see Crandall & Sherman, [<reflink idref="bib5" id="ref69">5</reflink>] for a comparison of direct and conceptual replication studies). As these changes may influence the outcome, they need to be considered when interpreting the results. As in Loibl and Rummel ([<reflink idref="bib18" id="ref70">18</reflink>]), we investigated how the learning outcome depends on the order of the phases (PS-I, PS-I<sups>ts</sups> vs. I-PS, I<sups>ts</sups>-PS) and on the use of typical student solutions in the explicit instruction phase (with vs. without typical student solutions: I vs. I<sups>ts</sups>). In contrast to Loibl and Rummel ([<reflink idref="bib18" id="ref71">18</reflink>]), and in spite of the well-designed parallelism in the instructional design, we did not find a main effect, neither for problem-solving prior to instruction (in comparison to the reversed order), nor for building on typical student solutions.</p> <hd id="AN0189211899-16">Why did the replication fail?</hd> <p>Since in other contexts and similar designs the expected effects could be found (Sinha & Kapur, [<reflink idref="bib27" id="ref72">27</reflink>]), our findings may have several interpretations: For instance, the generality of the PS-I vs. I-PS effect could be questioned and partially attributed to the specific contexts in which it is found. Possibly the power of our study, though adequate for the effect sizes expected based on the original study, was not sufficient for potentially lower effect sizes of our replication study. However, the exploratory analysis above, revealed that the reasons should rather be looked for on the level of the learning processes initiated by the instructional design.</p> <p>To get more insights on why the replication failed despite the structural similarities of the learning topics, we analyzed the problem-solving process as well as the fit between students' solution attempts and the explicit instruction phase. With regard to the problem-solving process, our data reveals that in the initial problem-solving phase (in PS-I and PS-I<sups>ts</sups>) 31 of 38 students only generated one solution attempt, which they applied to all three cases. Thus, students merely activated their prior knowledge on probabilities (probabilities have to be multiplied) but did not extensively explore the problem space nor generated multiple knowledge components. In contrast, in the original study students generated about four different solution attempts (reported in Loibl & Rummel, [<reflink idref="bib19" id="ref73">19</reflink>]: mean quantity of solution attempts = 4.13, SD = 1.35). This difference in the amount of students' exploration (L<subs>1</subs>) may have impacted the findings. One reason for the students' limited problem-solving processes may be that students did not become aware of their knowledge gaps. The three datasets (scores of soccer player) in the original study (Loibl & Rummel, [<reflink idref="bib18" id="ref74">18</reflink>]) are designed in a way that by comparing the results of the three player it becomes apparent that merely applying students' prior knowledge (e.g., mean, range) does not allow to distinguish between the three players, which fosters negative knowledge (i.e., knowledge on how not to solve the problem, Oser & Spychiger, [<reflink idref="bib23" id="ref75">23</reflink>]). This design automatically prompts students to go beyond their prior knowledge. We intended a similar effect with the design of our three cases. The cases were designed in a way that it becomes apparent that taking the evidence only or averaging priors and evidence does not work. Based on prior research (Loibl & Leuders, [<reflink idref="bib15" id="ref76">15</reflink>]), we expected students to start with these erroneous solution attempts. Unfortunately, they did not. The difference between the solutions generated in this study in comparison to the solutions from our prior research might have been caused by the variation in the representation, specifically by the combination of graphical and numerical presentation of the data in the current study. Instead of the expected errors, students' solution attempts in this study lacked the normalization step (cf. Table 1: division by the total probability). The design of the cases did not provide hints that solutions without normalization are incorrect and, thus, did not support the acquisition of negative knowledge. Therefore, students likely were satisfied with their initial solution attempt, which stopped them from further exploration. As students did not acquire negative knowledge during the initial problem-solving phase (in PS-I and PS-I<sups>ts</sups>), they were not better prepared for instruction than their counterparts in the instruction prior to problem-solving conditions (I-PS, I<sups>ts</sups>-PS). PS-I research further demonstrated that providing contrasting cases during the problem-solving phase (i.e., datasets highlighting one feature at a time) that provide implicit feedback (Nathan, [<reflink idref="bib21" id="ref77">21</reflink>]; Roll et al., [<reflink idref="bib26" id="ref78">26</reflink>]) on students' solution attempts support students in generating more elaborated solution attempts (Loibl & Rummel, [<reflink idref="bib19" id="ref79">19</reflink>]) which, in turn, may foster learning (Loibl et al., [<reflink idref="bib17" id="ref80">17</reflink>]).</p> <p>The unexpected student solutions also may have caused the failed replication regarding the second factor: building on student solutions in the explicit instruction phase. As apparent in Table 8 only two (out of 38) students in the problem-solving prior to instruction conditions (PS-I, PS-I<sups>ts</sups>) generated solutions that corresponded to the erroneous solutions included in the explicit instruction in the PS-I<sups>ts</sups> and I<sups>ts</sups>-PS conditions. In other words, while the included solutions are typical student solutions in general research on Bayesian reasoning, they apparently are not typical for the present sample. Due to the mismatch between the student solutions included in the explicit instruction phase and what students actually generated, the explicit instruction did not refute students' actual misconceptions (Tippett, [<reflink idref="bib29" id="ref81">29</reflink>]). Asterhan and Dotan ([<reflink idref="bib1" id="ref82">1</reflink>]) state that a fit between instruction and learners' individual understanding (e.g., by comparing own and correct solutions) can increase learning success as it allows integrating the new knowledge components by linkage and adaptation of the existing knowledge base (cf. Gadgil et al., [<reflink idref="bib8" id="ref83">8</reflink>]). Thus, the missing fit between students' generated solution attempts and the ones included in the explicit instruction phase may explain why we did not find a main effect for including student solutions in the explicit instruction phase. This interpretation is supported by findings by Loibl and Leuders ([<reflink idref="bib13" id="ref84">13</reflink>]) showing that the highest effect was found for students who actually generated the solution types during problem solving that were later included in the explicit instruction phase.</p> <hd id="AN0189211899-17">Lessons learned</hd> <p>The conceptual replication, despite the described failures, can still provide insights on important design features that need to be considered in practice and research. First, the problem-solving phase should include implicit (Nathan, [<reflink idref="bib21" id="ref85">21</reflink>]; Roll et al., [<reflink idref="bib26" id="ref86">26</reflink>]) or explicit feedback to facilitate the acquisition of negative knowledge and to prompt the generation of further solution attempts. Acquiring negative knowledge (Oser & Spychiger, [<reflink idref="bib23" id="ref87">23</reflink>]) and generating multiple solution attempts (Kapur & Bielaczyc, [<reflink idref="bib11" id="ref88">11</reflink>]) seems to be a crucial element to render PS-I productive as these processes prepare students for subsequent instruction.</p> <p>Second, the explicit instruction phase should include student solutions that really are typical. Only an alignment between the processes and products of the problem-solving phase and the subsequent explicit instruction phase allows to refute misconceptions and to integrate the newly introduced knowledge components to the ones activated or generated during problem solving.</p> <p>While these two design recommendations are relevant for practice and research likewise, from a research perspective another aspect to consider is to better structure the problem-solving phase in order to reduce variation of procedures and, thus, achieve higher control, for instance with regard to which solution attempts students generate. Structuring the problem-solving phase, however, poses the challenge to balance the structure in a way that it still allows valid and complex problem-solving processes (Boomgaarden et al., [<reflink idref="bib2" id="ref89">2</reflink>]).</p> <hd id="AN0189211899-18">Conclusion</hd> <p>While we failed to replicate the findings of Loibl and Rummel ([<reflink idref="bib18" id="ref90">18</reflink>]) in our conceptual replication study, our results do not contradict the theoretical assumptions made by Loibl and Rummel. Given the learning processes that actually took place in our study (which diverged from the expected processes), the theoretical assumptions would actually predict the null effects found in our study. In line with the knowledge-learning-instruction (KLI) framework (Koedinger et al., [<reflink idref="bib12" id="ref91">12</reflink>]), our results, thus, highlight the importance of not only testing effects of the instructional design, but also consider indicators of the learning processes (Reinhold et al., [<reflink idref="bib24" id="ref92">24</reflink>]). Only if the learning processes indeed match the assumed processes, experimental variations of the instructional design can test the theoretical assumptions. The KLI framework (Koedinger et al., [<reflink idref="bib12" id="ref93">12</reflink>]) for single-phase designs and the CID framework (Loibl et al., [<reflink idref="bib16" id="ref94">16</reflink>]) for multi-phase designs provide a heuristic for such an endeavor.</p> <hd id="AN0189211899-19">Author contributions</hd> <p>Katharina Loibl: Conceptualization, Methodology, Project administration, Funding acquisition, Investigation, Formal analysis, Interpretation of the data, Writing—original draft, Writing—review & editing. Timo Leuders: Conceptualization, Methodology, Investigation, Interpretation of the data, Writing—original draft, Writing—review & editing.</p> <hd id="AN0189211899-20">Funding</hd> <p>Open Access funding enabled and organized by Projekt DEAL. The study was funded by the Deutsche Telekom Stiftung. We thank our student research assistants Natalja Sablowski and Ann-Christin Hermann for their help in coding the data.</p> <hd id="AN0189211899-21">Data availability</hd> <p>The dataset is available from the corresponding author on reasonable request.</p> <hd id="AN0189211899-22">Declarations</hd> <p></p> <hd id="AN0189211899-23">Competing interest</hd> <p>The authors have no competing interests.</p> <hd id="AN0189211899-24">Appendix</hd> <p>The distributions of the knowledge outcomes are depicted in Fig. </p> <p>Graph: Fig. 3 Distribution plots of the knowledge outcomes</p> <p>3.</p> <hd id="AN0189211899-25">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0189211899-26"> <title> References </title> <blist> <bibl id="bib1" idref="ref44" type="bt">1</bibl> <bibtext> Asterhan CS, Dotan A. Feedback that corrects and contrasts students' erroneous solutions with expert ones improves expository instruction for conceptual change. Instructional Science. 2018; 46: 337-355. 10.1007/s11251-017-9441-1</bibtext> </blist> <blist> <bibl id="bib2" idref="ref48" type="bt">2</bibl> <bibtext> Boomgaarden, A, Loibl, K, & Leuders, T. (2022). The trade-off between complexity and accuracy: Preparing for computer-based adaptive instruction on fractions. Interactive Learning Environments, 1–16.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref24" type="bt">3</bibl> <bibtext> Brase GL, Barbey AK. Mental representations of statistical information. Advances in Psychology Research. 2006; 41: 91-113</bibtext> </blist> <blist> <bibl id="bib4" idref="ref50" type="bt">4</bibl> <bibtext> Conover WJ, Iman RL. Rank transformations as a bridge between parametric and nonparametric statistics. The American Statistician. 1981; 35; 3: 124-129. 10.1080/00031305.1981.10479327</bibtext> </blist> <blist> <bibl id="bib5" idref="ref27" type="bt">5</bibl> <bibtext> Crandall CS, Sherman JW. On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology. 2016; 66: 93-99. 10.1016/j.jesp.2015.10.002</bibtext> </blist> <blist> <bibl id="bib6" idref="ref1" type="bt">6</bibl> <bibtext> de Jong T, Lazonder AW, Chinn CA, Fischer F, Gobert J, Hmelo-Silver CE, Koedinger KR, Krajcik JS, Kyza EA, Linn MC, Pedaste M, Scheiter K, Zacharia ZC. Let's talk evidence–The case for combining inquiry-based and direct instruction. Educational Research Review. 2023; 39: 100536. 10.1016/j.edurev.2023.100536</bibtext> </blist> <blist> <bibl id="bib7" idref="ref39" type="bt">7</bibl> <bibtext> Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods. 2007; 39: 175-191. 10.3758/BF03193146</bibtext> </blist> <blist> <bibl id="bib8" idref="ref83" type="bt">8</bibl> <bibtext> Gadgil S, Nokes-Malach TJ, Chi MT. Effectiveness of holistic mental model confrontation in driving conceptual change. Learning and Instruction. 2012; 22; 1: 47-61. 10.1016/j.learninstruc.2011.06.002</bibtext> </blist> <blist> <bibl id="bib9" idref="ref22" type="bt">9</bibl> <bibtext> Gigerenzer G, Hoffrage U. How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review. 1995; 102; 4: 684. 10.1037/0033-295X.102.4.684</bibtext> </blist> <blist> <bibtext> Kapur M. Productive failure in learning the concept of variance. Instructional Science. 2012; 40: 651-672. 10.1007/s11251-012-9209-6</bibtext> </blist> <blist> <bibtext> Kapur M, Bielaczyc K. Designing for productive failure. The Journal of the Learning Sciences. 2012; 21; 1: 45-83. 10.1080/10508406.2011.591717</bibtext> </blist> <blist> <bibtext> Koedinger KR, Corbett AT, Perfetti C. The Knowledge-Learning-Instruction framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science. 2012; 36; 5: 757-798. 10.1111/j.1551-6709.2012.01245.x</bibtext> </blist> <blist> <bibtext> Loibl K, Leuders T. How to make failure productive: Fostering learning from errors through elaboration prompts. Learning and Instruction. 2019; 62: 1-10. 10.1016/j.learninstruc.2019.03.002</bibtext> </blist> <blist> <bibtext> Loibl K, Leuders TDenison S, Mack M, Xu Y, Armstrong BC. "Take the middle"—Averaging prior and evidence as effective heuristic in Bayesian reasoning. Proceedings of the 42nd Annual conference of the Cognitive Science Society. 2020; Cognitive Science Society: 1764-1770</bibtext> </blist> <blist> <bibtext> Loibl K, Leuders T. Thinking in proportions rather than probabilities facilitates Bayesian reasoning. Proceedings of the 45th Annual Conference of the Cognitive Science Society. 2024; Cognitive Science Society: 1-12</bibtext> </blist> <blist> <bibtext> Loibl K, Leuders T, Glogger-Frey I, Rummel N. CID: A framework for the cognitive analysis of composite instructional designs. Instructional Science. 2024. 10.1007/s11251-024-09665-9</bibtext> </blist> <blist> <bibtext> Loibl K, Roll I, Rummel N. Towards a theory of when and how problem solving followed by instruction supports learning. Educational Psychology Review. 2017; 29: 693-715. 10.1007/s10648-016-9379-x</bibtext> </blist> <blist> <bibtext> Loibl K, Rummel N. Knowing what you don't know makes failure productive. Learning and Instruction. 2014; 34: 74-85. 10.1016/j.learninstruc.2014.08.004</bibtext> </blist> <blist> <bibtext> Loibl K, Rummel N. The impact of guidance during problem-solving prior to instruction on students' inventions and learning outcomes. Instructional Science. 2014; 42: 305-326. 10.1007/s11251-013-9282-5</bibtext> </blist> <blist> <bibtext> McDowell M, Jacobs P. Meta-analysis of the effect of natural frequencies on Bayesian reasoning. Psychological Bulletin. 2017; 143; 12: 1273. 10.1037/bul0000126</bibtext> </blist> <blist> <bibtext> Nathan MJ. Knowledge and situational feedback in a learning environment for algebra story problem solving. Interactive Learning Environments. 1998; 5; 1: 135-159. 10.1080/1049482980050110</bibtext> </blist> <blist> <bibtext> Nosek BA, Errington TM. The best time to argue about what a replication means?. Before You Do It. Nature. 2020; 583; 7817: 518-520</bibtext> </blist> <blist> <bibtext> Oser, F, & Spychiger, M. (2005). Lernen ist schmerzhaft: Zur Theorie des negativen Wissens und zur Praxis der Fehlerkultur [Learning is painful: On the theory of negative knowledge and the practice of error culture]. Beltz.</bibtext> </blist> <blist> <bibtext> Reinhold F, Leuders T, Loibl K, Nückles M, Beege M, Boelmann JM. Learning mechanisms explaining learning with digital tools in educational settings: A cognitive process framework. Educational Psychology Review. 2024; 36; 1: 14. 10.1007/s10648-024-09845-6</bibtext> </blist> <blist> <bibtext> Renkl ATierney R, Rizvi F, Ercikan K. Exemplars. International encyclopedia of education. 20234; Elsevier: 612-622. 10.1016/B978-0-12-818630-5.14067-9</bibtext> </blist> <blist> <bibtext> Roll I, Wiese E, Long Y, Aleven V, Koedinger KRSottilare R, Graesser A, Hu X, Goldberg B. Tutoring self- and co-regulation with intelligent tutoring systems to help students acquire better learning skills. Design recommendations for adaptive intelligent tutoring systems: Volume 2—Adaptive instructional strategies. 2014; US Army Research Laboratory: 169-182</bibtext> </blist> <blist> <bibtext> Sinha T, Kapur M. When problem solving followed by instruction works: Evidence for productive failure. Review of Educational Research. 2021; 91; 5: 761-798. 10.3102/00346543211019105</bibtext> </blist> <blist> <bibtext> Thevenot C, Barrouillet PKadosh RC, Dowker A. Arithmetic word problem solving and mental representations. The Oxford handbook of numerical cognition. 2014; Oxford Library of Psychology</bibtext> </blist> <blist> <bibtext> Tippett CD. Refutation text in science education: A review of two decades of research. International Journal of Science and Mathematics Education. 2010; 8: 951-970. 10.1007/s10763-010-9203-x</bibtext> </blist> <blist> <bibtext> Zhu L, Gigerenzer G. Children can solve Bayesian problems: The role of representation in mental computation. Cognition. 2006; 98: 287-308. 10.1016/j.cognition.2004.12.003</bibtext> </blist> </ref> <ref id="AN0189211899-27"> <title> Footnotes </title> <blist> <bibtext> The familiarity with the topic was a retrospective self-assessment ("How familiar were you with the topic Bayesian reasoning prior to this session?") on a four-point scale from 0 = not at all familiar to 3 = very familiar.</bibtext> </blist> </ref> <aug> <p>By Katharina Loibl and Timo Leuders</p> <p>Reported by Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib11" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib18" firstref="ref3"></nolink> <nolink nlid="nl3" bibid="bib17" firstref="ref4"></nolink> <nolink nlid="nl4" bibid="bib27" firstref="ref6"></nolink> <nolink nlid="nl5" bibid="bib16" firstref="ref11"></nolink> <nolink nlid="nl6" bibid="bib12" firstref="ref12"></nolink> <nolink nlid="nl7" bibid="bib10" firstref="ref13"></nolink> <nolink nlid="nl8" bibid="bib23" firstref="ref14"></nolink> <nolink nlid="nl9" bibid="bib25" firstref="ref15"></nolink> <nolink nlid="nl10" bibid="bib28" firstref="ref23"></nolink> <nolink nlid="nl11" bibid="bib22" firstref="ref28"></nolink> <nolink nlid="nl12" bibid="bib14" firstref="ref32"></nolink> <nolink nlid="nl13" bibid="bib30" firstref="ref33"></nolink> <nolink nlid="nl14" bibid="bib15" firstref="ref34"></nolink> <nolink nlid="nl15" bibid="bib20" firstref="ref35"></nolink> <nolink nlid="nl16" bibid="bib70" firstref="ref42"></nolink> <nolink nlid="nl17" bibid="bib71" firstref="ref52"></nolink> <nolink nlid="nl18" bibid="bib19" firstref="ref73"></nolink> <nolink nlid="nl19" bibid="bib21" firstref="ref77"></nolink> <nolink nlid="nl20" bibid="bib26" firstref="ref78"></nolink> <nolink nlid="nl21" bibid="bib29" firstref="ref81"></nolink> <nolink nlid="nl22" bibid="bib13" firstref="ref84"></nolink> <nolink nlid="nl23" bibid="bib24" firstref="ref92"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1493444
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Can Failure Be Made Productive Also in Bayesian Reasoning? A Conceptual Replication Study
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Katharina+Loibl%22">Katharina Loibl</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-1773-1913">0000-0002-1773-1913</externalLink>)<br /><searchLink fieldCode="AR" term="%22Timo+Leuders%22">Timo Leuders</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-7621-7826">0000-0002-7621-7826</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Instructional+Science%3A+An+International+Journal+of+the+Learning+Sciences%22"><i>Instructional Science: An International Journal of the Learning Sciences</i></searchLink>. 2025 53(6):1739-1758.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 20
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Instructional+Design%22">Instructional Design</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Solving%22">Problem Solving</searchLink><br /><searchLink fieldCode="DE" term="%22Direct+Instruction%22">Direct Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Failure%22">Failure</searchLink><br /><searchLink fieldCode="DE" term="%22Instructional+Effectiveness%22">Instructional Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Bayesian+Statistics%22">Bayesian Statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Statistics+Education%22">Statistics Education</searchLink><br /><searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s11251-024-09670-y
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0020-4277<br />1573-1952
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The composite instructional design PS-I combines an initial problem-solving phase (PS) with a subsequent explicit instruction phase (I). PS-I has proven effective for conceptual learning in comparison to instructional designs with the reverse order (I-PS), especially when the explicit instruction phase productively builds on students' erroneous or incomplete (i.e., failed) solution attempts. Building on student solutions during explicit instruction may support students to integrate their intermediate knowledge (acquired during problem solving) with the newly introduced knowledge components. While these effects have been shown for learning the concept of variance in multiple studies, it remains unclear whether these effects generalize to other situations. We conducted a conceptual replication study of Loibl and Rummel (Loibl and Rummel, "Learning and Instruction" 34:74-85, 2014a) choosing Bayesian reasoning as target knowledge. 75 students were assigned to four conditions in a 2 x 2 design (factor 1: PS-I vs. I-PS; factor 2: instruction phase with vs. without typical student solutions). In contrast to Loibl and Rummel (2014a), we did neither find a main effect for PS-I vs. I-PS, nor for building on typical student solutions. The missing effect of PS-I can be explained by the fact that students merely activated their prior knowledge on probabilities without exploring the problem-solving space and without becoming aware of their knowledge gaps. The missing effect of building on typical student solutions can be explained by a mismatch of the solutions generated and the ones included in the explicit instruction. Therefore, building on typical student solutions did not foster an integration of students' intermediate knowledge and the introduced knowledge components.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1493444
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1493444
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s11251-024-09670-y
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 20
        StartPage: 1739
    Subjects:
      – SubjectFull: Instructional Design
        Type: general
      – SubjectFull: Problem Solving
        Type: general
      – SubjectFull: Direct Instruction
        Type: general
      – SubjectFull: Failure
        Type: general
      – SubjectFull: Instructional Effectiveness
        Type: general
      – SubjectFull: Bayesian Statistics
        Type: general
      – SubjectFull: Statistics Education
        Type: general
      – SubjectFull: Undergraduate Students
        Type: general
    Titles:
      – TitleFull: Can Failure Be Made Productive Also in Bayesian Reasoning? A Conceptual Replication Study
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Katharina Loibl
      – PersonEntity:
          Name:
            NameFull: Timo Leuders
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0020-4277
            – Type: issn-electronic
              Value: 1573-1952
          Numbering:
            – Type: volume
              Value: 53
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: Instructional Science: An International Journal of the Learning Sciences
              Type: main
ResultId 1