View in EDS HTML Full Text PDF Full Text

Productive Failure as an Instructional Approach to Promote Future Learning

Saved in:
Bibliographic Details
Title: Productive Failure as an Instructional Approach to Promote Future Learning
Language: English
Authors: Steenhof, Naomi (ORCID 0000-0001-7931-9134), Woods, Nicole N., Van Gerven, Pascal W. M., Mylopoulos, Maria
Source: Advances in Health Sciences Education. Oct 2019 24(4):739-749.
Availability: Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 11
Publication Date: 2019
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Failure, Teaching Methods, Problem Solving, Instructional Effectiveness, Doctoral Students, Pharmaceutical Education, Direct Instruction, Foreign Countries
Geographic Terms: Canada (Toronto)
DOI: 10.1007/s10459-019-09895-4
ISSN: 1382-4996
Abstract: Productive failure is an instructional approach that requires learners to struggle as they attempt to generate solutions to problems before, rather than after, receiving direct instruction on a targeted concept. Studies demonstrate that productive failure prepares students for later learning of new, related knowledge. Our study explored the effectiveness of productive failure as an instructional intervention in health professions education with respect to (a) acquisition and application of a novel concept, and (b) learners' preparation for future learning of new, related content. Forty year-one students enrolled in the Doctor of Pharmacy program at the University of Toronto were randomly assigned to a productive failure (i.e. attempt to generate solutions before receiving instruction) or direct instruction only learning condition. After a practice phase, participants completed a series of tests designed to measure knowledge acquisition, knowledge application, and preparation for future learning (new learning is required for successful problem solving). As expected, no difference in performance was seen between participants on the acquisition and application tests. However, participants in the productive failure condition outperformed those in the direct instruction condition on the preparation for future learning test. These results emphasize the role of struggle in learning and support the theory that engaging students in solving problems that are beyond their abilities can be a productive exercise in failure. The results suggest that productive failure assists learners in acquiring the conceptual knowledge needed to facilitate learning in the future.
Abstractor: As Provided
Entry Date: 2019
Accession Number: EJ1230253
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGgUq5ZPINtXMpnNqPLhNycAAAA4TCB3gYJKoZIhvcNAQcGoIHQMIHNAgEAMIHHBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDDskHbpZ4rFszz8P_gIBEICBmW9oxibsx3EVKBLe-FwVILTOxbRN_frqmuiSQxUtYH--RNwTUXmDuWJVeXOdVq_BsmtdH8A9e23ka3h8JUQhjSO6Szn6ogRHtJK_sDmz5XUqoRu2cT2emqmBiFh-_BS8uKH9zlI9ncWTUKk9RdSbJjOVc3AKwA5YQimCpMeZsY2QsU9fLqUr8NTUvghdIPelQVoIE9IQfmKIsA==
Text:
  Availability: 1
  Value: <anid>AN0138912231;oak01oct.19;2019Oct04.04:40;v2.2.500</anid> <title id="AN0138912231-1">Productive failure as an instructional approach to promote future learning </title> <p>Productive failure is an instructional approach that requires learners to struggle as they attempt to generate solutions to problems before, rather than after, receiving direct instruction on a targeted concept. Studies demonstrate that productive failure prepares students for later learning of new, related knowledge. Our study explored the effectiveness of productive failure as an instructional intervention in health professions education with respect to (a) acquisition and application of a novel concept, and (b) learners' preparation for future learning of new, related content. Forty year-one students enrolled in the Doctor of Pharmacy program at the University of Toronto were randomly assigned to a productive failure (i.e. attempt to generate solutions before receiving instruction) or direct instruction only learning condition. After a practice phase, participants completed a series of tests designed to measure knowledge acquisition, knowledge application, and preparation for future learning (new learning is required for successful problem solving). As expected, no difference in performance was seen between participants on the acquisition and application tests. However, participants in the productive failure condition outperformed those in the direct instruction condition on the preparation for future learning test. These results emphasize the role of struggle in learning and support the theory that engaging students in solving problems that are beyond their abilities can be a productive exercise in failure. The results suggest that productive failure assists learners in acquiring the conceptual knowledge needed to facilitate learning in the future.</p> <p>Keywords: Adaptive expertise; Clinical reasoning; Innovation; Pharmacy education; Preparation for future learning; Productive failure</p> <hd id="AN0138912231-2">Introduction</hd> <p>When clinicians enter practice, they encounter an inevitable combination of problems: ones that they have studied in school, and ones they have not seen before. It follows, therefore, that alongside exposing trainees to known, routine problems, education should ideally also prepare trainees to generate solutions to new and expected problems in practice (Mylopoulos et al. [<reflink idref="bib13" id="ref1">13</reflink>]). This capacity to learn new information, to use resources effectively and innovatively, and to invent new strategies for learning while solving problems is defined in the literature as 'Preparation for Future Learning (PFL)' and is understood to be essential for adaptive expertise (Mylopoulos and Woods [<reflink idref="bib17" id="ref2">17</reflink>]). In practice, experts who have not developed PFL will still be able to solve routine problems, but will likely underperform in situations of ambiguity, novelty, or complexity (Mylopoulos and Scardamalia [<reflink idref="bib15" id="ref3">15</reflink>]). Unfortunately, many practicing clinicians incorrectly apply known solutions to new problems (Mamede et al. [<reflink idref="bib12" id="ref4">12</reflink>]; Saposnik et al. [<reflink idref="bib18" id="ref5">18</reflink>]). This is unsurprising since current health professions education frequently focuses on instruction that optimizes replication of previous performance or application of known solutions rather than assessing the ability to generate new solutions or learn in the future (Mylopoulos et al. [<reflink idref="bib13" id="ref6">13</reflink>]). Given this discrepancy between professional expectations and what is currently taught, it is imperative for educators to design innovative systems of instruction and assessment that develop PFL (Mylopoulos et al. [<reflink idref="bib13" id="ref7">13</reflink>]).</p> <p>Recent research in human cognition and learning suggests that educators can design learning experiences that allow students to efficiently master a core selection of knowledge and skills, while also supporting development of PFL (Bransford and Schwartz [<reflink idref="bib2" id="ref8">2</reflink>]; Kapur [<reflink idref="bib7" id="ref9">7</reflink>]; Mylopoulos and Woods [<reflink idref="bib16" id="ref10">16</reflink>]). Approaches to education that support development of PFL aim to provide trainees with cycles of instruction and assessment that offer opportunities for both acquiring and applying knowledge as well as using existing knowledge to learn new concepts or solve novel problems (Woods and Mylopoulos [<reflink idref="bib24" id="ref11">24</reflink>]). For example, studies of classroom instruction have found that, given a combination of discovery learning experiences (e.g., contrasting cases, invention activities) and subsequent instruction, students demonstrate improved learning of related yet new material when next placed in knowledge-rich environments (Bransford and Schwartz [<reflink idref="bib2" id="ref12">2</reflink>]; Mylopoulos et al. [<reflink idref="bib13" id="ref13">13</reflink>]). These types of instructional interventions have been termed 'guided discovery' (Bruner [<reflink idref="bib3" id="ref14">3</reflink>]). In one study, students who explored and charted data sets on memory experiments were better able to predict the results of a novel experiment, but only if they heard a lecture on the topic afterwards (Bransford and Schwartz [<reflink idref="bib2" id="ref15">2</reflink>]). Struggling through the problem first prepares learners to grasp the significance of the expert solution (Schwartz and Bransford [<reflink idref="bib20" id="ref16">20</reflink>]). Notably, the design of these interventions are different than hybrid problem-based learning, flipped classroom, or case-based learning designs because students are placed in a knowledge-rich environment with a new problem only after they have struggled, been shown an expert solution, and learned the concept. Assessment then focuses on whether the student is able to take action to acquire the knowledge they need to solve a new problem.</p> <p>Critically, instruction that encourages guided discovery may initially appear unproductive because student performance on standard assessments, which emphasize replication and application of knowledge, does not improve (Schwartz and Martin [<reflink idref="bib23" id="ref17">23</reflink>]). Students engaged in guided discovery may generate atypical or incorrect solutions and, not surprisingly, several studies have demonstrated that students' performance on standard assessments is poor (Kapur and Rummel [<reflink idref="bib10" id="ref18">10</reflink>]; Schwartz et al. [<reflink idref="bib22" id="ref19">22</reflink>]). Therefore, assessing the efficacy of guided discovery instructional design strategies requires that testing go beyond measuring acquisition and application (because there may be no impact), and also assess students on their ability to learn in the future (Woods and Mylopoulos [<reflink idref="bib24" id="ref20">24</reflink>]).</p> <p>'Productive Failure' is an example of a guided discovery instructional design strategy that maximizes future learning by first engaging students in problem solving, and then teaching the central concept and procedures associated with the problem (Bransford and Schwartz [<reflink idref="bib2" id="ref21">2</reflink>]; Kapur [<reflink idref="bib7" id="ref22">7</reflink>]). Learners typically fail to generate the established solution in the initial problem-solving stage, but during the instruction phase the learners can (a) think about what they were doing, (b) recognize the limitations of their generated solutions, and (c) consolidate their knowledge into the established solution (Kapur [<reflink idref="bib8" id="ref23">8</reflink>]). Researchers hypothesize that generating solutions may activate and differentiate relevant prior knowledge during the problem-solving phase, and may help learners notice the limits of their prior knowledge (Kapur [<reflink idref="bib7" id="ref24">7</reflink>]; Kapur and Rummel [<reflink idref="bib10" id="ref25">10</reflink>]). For example, early work on productive failure showed that engaging students in complex, ill-structured problems requiring them to apply concepts in Newtonian kinematics without providing support structures could be productive (Kapur [<reflink idref="bib6" id="ref26">6</reflink>]). In a later study, students who received productive failure instruction—as opposed to direct instruction—outperformed students on conceptual understanding and transfer of learning around the concept of variance (Kapur [<reflink idref="bib7" id="ref27">7</reflink>]). Based on work to date, Kapur suggests several key design features for the benefits of productive failure to be appreciated: (a) the initial problem-solving task should be challenging enough to engage the learner in the exploration, but not so challenging that the learner gives up, (b) the problem-solving task must admit multiple solutions, strategies, and representations, that is, afford sufficient problem and solution spaces for exploration, (c) problem solving must activate learner's prior knowledge—formal as well as intuitive—to solve the problem, and (d) an expert should build upon the student-generated solutions by comparing and contrasting them with the correct solution (Kapur [<reflink idref="bib8" id="ref28">8</reflink>]).</p> <p>The success of the work of Kapur and others suggests that productive failure might be a promising strategy for enhancing clinical education for health professions. However, productive failure has largely been studied in elementary education, where the problems and concepts are relatively simple. To date, productive failure has not been explored in the context of health professions education, where students are expected to draw on multiple disciplines (e.g., math, biology, physiology, chemistry) and apply these principles to complex problems.</p> <p>To begin to explore using productive failure in health professions education, the objectives of the current study were (a) to compare the effectiveness of productive failure relative to direct instruction on acquisition and application of a novel concept, and (b) to compare the effectiveness of productive failure relative to direct instruction on an assessment which tests students' preparation for future learning. We hypothesized that on the acquisition and application tests there would be no performance difference between participants who learn using direct instruction or productive failure materials. However, we expected that the participants in the productive failure condition would outperform those in the direct instruction condition on the PFL assessment.</p> <hd id="AN0138912231-3">Methods</hd> <p></p> <hd id="AN0138912231-4">Participants</hd> <p>This study took place during the autumn of 2017 at the Leslie Dan Faculty of Pharmacy at the University of Toronto (U of T) (Canada). The participants were year-one students enrolled in the Doctor of Pharmacy (PharmD) program. The researchers chose the year-one PharmD population deliberately to ensure that participants would have minimal prior exposure to the new concepts, but still be able to understand the basic terminology presented in the materials. This was intentional as the study was designed to go beyond the knowledge, skills, and abilities of the learners to test learning of a novel concept.</p> <p>The researchers contacted 235 year-one pharmacy students through the class LISTSERV and in-class announcements. Forty-three (18%) of the contacted students volunteered to participate. After enrollment, the participants were randomly assigned to a learning condition: either direct instruction (22 participants) or productive failure (21 participants). Data for three participants (one from the direct instruction condition and two from the productive failure condition) were excluded from the analysis as the participants did not complete the entire experiment.</p> <p>An honorarium of thirty Canadian dollars in the form of a gift card was offered to students who participated in the study. The Research Ethics Board at the U of T granted human research ethics approval (Protocol Reference #34479).</p> <hd id="AN0138912231-5">Material development</hd> <p></p> <hd id="AN0138912231-6">Learning and assessment materials</hd> <p>Both the direct instruction and productive failure learning materials were developed by the authors. When deciding what content area to teach and assess in this study, guidance was taken from the design principles of Kapur's productive failure studies (Kapur [<reflink idref="bib8" id="ref29">8</reflink>]). Understanding creatinine clearance is a challenging problem, but not so challenging that participants are not able to engage. There are multiple solutions and approaches that are accepted by experts. Participants have prior knowledge in maths, biology, physiology, and chemistry, which could be activated to help them solve the problem, but is insufficient for the appropriate application of the concept.</p> <p>The Cockcroft–Gault equation, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mi>r</mi><mi>C</mi><mi>l</mi><mo>=</mo><mfrac><mrow><mfenced close=")" open="("><mrow><mn>140</mn><mo>-</mo><mi>a</mi><mi>g</mi><mi>e</mi></mrow></mfenced><mspace width="0.166667em" /><mo>×</mo><mspace width="0.166667em" /><mi>w</mi><mi>e</mi><mi>i</mi><mi>g</mi><mi>h</mi><mi>t</mi><mspace width="0.166667em" /><mfenced close=")" open="("><mtext>kg</mtext></mfenced><mspace width="0.166667em" /><mo>×</mo><mspace width="0.166667em" /><mn>1.23</mn></mrow><mrow><mi>s</mi><mi>e</mi><mi>r</mi><mi>u</mi><mi>m</mi><mspace width="0.166667em" /><mi>c</mi><mi>r</mi><mi>e</mi><mi>a</mi><mi>t</mi><mi>i</mi><mi>n</mi><mi>i</mi><mi>n</mi><mi>e</mi><mspace width="0.166667em" /><mfenced close=")" open="("><mrow><mi mathvariant="normal">μ</mi><mtext>mol</mtext><mo stretchy="false">/</mo><mtext>L</mtext></mrow></mfenced></mrow></mfrac></mrow></math> </ephtml> , traditionally has been used to estimate creatinine clearance (CrCl) when determining medication doses (Cockcroft and Gault [<reflink idref="bib4" id="ref30">4</reflink>]). The problem of creatinine clearance is an important problem to study because novices (whether pharmacists, physicians, or nurses) tend to use the Cockcroft–Gault equation quite differently than experts. In a hospital setting, pharmacists are frequently consulted regarding renal dosing of medication. Novices tend to plug-and-play: collect the data for the variables in the equation (age, weight, sex, serum creatinine), enter the values into the equation, calculate a number, and then use that number to choose the right dose from a dosing table or medication dosing resource. However, this output can be misleading if novices do not understand the limitations and applicability of the equation. These nuances made the problem an interesting one to study using the productive failure methodology.</p> <p>The materials were pilot tested with four novices (health professions education students) and two experts (8 + years of clinical experience) and were revised as necessary based on pilot testing. The participants learned independently using paper-based learning and assessment materials. The assessment questions were in a multiple-choice format.</p> <hd id="AN0138912231-7">Design</hd> <p></p> <hd id="AN0138912231-8">Learning phase</hd> <p>All phases of the study occurred within a single experimental session lasting approximately 2 h. All participants first completed a 40-min learning phase in which they were asked to study the same concepts of estimating creatinine clearance based on kidney function (Fig. 1). The materials in this phase were different depending on which learning condition the participant was randomly assigned to during enrollment. The participants in the direct instruction learning condition were told about the problem of estimating creatinine clearance based on serum concentrations of creatinine, and then they were given the Cockcroft–Gault equation, <ephtml> <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>C</mi><mi>r</mi><mi>C</mi><mi>l</mi><mo>=</mo><mfrac><mrow><mfenced close=")" open="("><mrow><mn>140</mn><mo>-</mo><mi>a</mi><mi>g</mi><mi>e</mi></mrow></mfenced><mspace width="0.166667em" /><mo>×</mo><mspace width="0.166667em" /><mi>w</mi><mi>e</mi><mi>i</mi><mi>g</mi><mi>h</mi><mi>t</mi><mspace width="0.166667em" /><mfenced close=")" open="("><mtext>kg</mtext></mfenced><mspace width="0.166667em" /><mo>×</mo><mspace width="0.166667em" /><mn>1.23</mn></mrow><mrow><mi>s</mi><mi>e</mi><mi>r</mi><mi>u</mi><mi>m</mi><mspace width="0.166667em" /><mi>c</mi><mi>r</mi><mi>e</mi><mi>a</mi><mi>t</mi><mi>i</mi><mi>n</mi><mi>i</mi><mi>n</mi><mi>e</mi><mspace width="0.166667em" /><mfenced close=")" open="("><mrow><mi mathvariant="normal">μ</mi><mtext>mol</mtext><mo stretchy="false">/</mo><mtext>L</mtext></mrow></mfenced></mrow></mfrac></mrow></math> </ephtml> (Cockcroft and Gault [<reflink idref="bib4" id="ref31">4</reflink>]), as a potential solution to this problem. The participants in the productive failure learning condition were told about the same problem of estimating creatinine clearance, but instead of being given the Cockcroft–Gault equation, they were given raw data from the original Cockcroft and Gault study, and asked to invent a formula that would best approximate creatinine clearance. The participants were supplied with graph paper to assist them in evaluating the association between some of the variables (e.g., age and creatinine concentration in the urine). The learners were instructed to make as many attempts as they could for at least 15 min (or until the learning phase was complete) and all materials were collected at the end of this phase.</p> <p>Graph: Fig. 1Study design</p> <hd id="AN0138912231-9">Instruction and practice phase</hd> <p>All participants completed the same 10-min instruction and practice phase which assessed content that was common to both learning conditions. The participants were given the Cockcroft–Gault formula and asked to answer ten multiple-choice questions that assessed recall of content from the learning phase (Cockcroft and Gault [<reflink idref="bib4" id="ref32">4</reflink>]). The participants were also given the opportunity to practice calculating creatinine clearance using the Cockcroft–Gault formula (3 multiple-choice questions). For the participants in the productive failure condition, this was the first time that they had seen the Cockcroft–Gault formula.</p> <hd id="AN0138912231-10">Assessment phase</hd> <p>All participants completed a 40-min series of sixteen multiple-choice questions designed to assess knowledge acquisition, knowledge application, and PFL. An example of acquisition is the knowledge the learners exercise in performing the task of determining creatinine clearance when given the appropriate variables. An example of application, is direct use of the equation in a case-based example. The PFL assessment items contained new content written in the stem of each multiple-choice question (example: antibiotic dosing in a patient with acute kidney injury) that students would have to learn in order to successfully answer the question (see Table 1 for sample questions). At the end of the experiment, the participants were asked whether they had heard of the Cockcroft–Gault equation before, and, if yes, whether they had used it.</p> <p>Sample assessment questions</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" /><th align="left" colspan="3"><p>Example</p></th></tr></thead><tbody><tr><td align="left"><p>Acquisition</p></td><td align="left" colspan="3"><p>YR is a 43-year-old man with a serum creatinine of 90 µmol/L. He weighs 42 kg. What is his creatinine clearance?</p><p> a. 90 mL/min</p><p> d. 19 mL/min</p><p> c. 56 mL/min</p><p> d. 65 mL/min</p></td></tr><tr><td align="left"><p>Application</p></td><td align="left" colspan="3"><p>Irving is a 23-year-old male with a serum creatinine of 220 µmol/L. He weighs 50 kg. Ricky is a 65-year-old male with a serum creatinine 160 µmol/L. He weighs 55 kg.</p><p>Who has a higher creatinine clearance?</p><p> a. Irving</p><p> b. Ricky</p><p>If Ricky gained 20 kg, by what percentage would his creatinine clearance improve?</p><p> a. 10%</p><p> b. 20%</p><p> c. 34%</p></td></tr><tr><td align="left" rowspan="9"><p>Preparation for future learning</p></td><td align="left" colspan="3"><p>Maggie is a 54-year-old woman who is admitted to the ICU with a severe lung infection. Today she weighs 56 kg. Her serum creatinine 2 days ago was 68 µmol/L. During rounds the ICU nurse comments that the patient has not produced any urine in 2 days. The ICU doctor wants to know whether you want to change the dose of her antibiotic. Currently Maggie is receiving meropenem 1 g intravenous (IV) every 8 h</p></td></tr><tr><td align="left"><p>Creatinine clearance (mL/min)</p></td><td align="left"><p>IV dose</p></td><td align="left"><p>Frequency</p></td></tr><tr><td align="left"><p>> 50</p></td><td align="left"><p>1 g</p></td><td align="left"><p>Every 8 h</p></td></tr><tr><td align="left"><p>25–50</p></td><td align="left"><p>1 g</p></td><td align="left"><p>Every 12 h</p></td></tr><tr><td align="left"><p>10–25</p></td><td align="left"><p>500 mg</p></td><td align="left"><p>Every 12 h</p></td></tr><tr><td align="left"><p>< 10</p></td><td align="left"><p>500 mg</p></td><td align="left"><p>Every 24 h</p></td></tr><tr><td align="left" colspan="3"><p> a. No, leave the meropenem dose at 1 g IV every 8 h</p></td></tr><tr><td align="left" colspan="3"><p> b. Yes, change the meropenem dose to 500 mg IV every 12 h</p></td></tr><tr><td align="left" colspan="3"><p> c. You want to change the dose, but need more information</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0138912231-11">Analysis</hd> <p>The data collected were the scores of the recall quiz (administered in the instruction and practice phase), and the scores from the assessment phase—acquisition, application, and PFL. For each participant, the proportion of correct responses on the recall quiz, acquisition, application, and PFL assessment were calculated as outcome measures (1 point for each correct response). In addition, the data from the recall quiz were submitted to an independent-samples <emph>t</emph> test, comparing the direct instruction and productive failure conditions. This analysis was secondary, intended only to ensure that the both groups were able to comprehend the content and had a basic understanding of the formula.</p> <p>The acquisition and application assessments were analysed separately using the Mann–Whitney U. Based on the literature, we did not expect a significant difference in either the acquisition or the application assessment. The PFL assessment was analysed using an analysis of covariance (ANCOVA) with the learning condition (direct instruction vs. productive failure) as the between subject variable and practice test performance as the covariate. Differences between the groups were determined to be statistically significant if <emph>p </emph><.05.</p> <hd id="AN0138912231-12">Results</hd> <p>Correct responses on each assessment were recorded for each participant. Means and standard deviations of these proportions are provided in Table 2.</p> <p>Mean proportion of correct results in the instruction and practice phase and PFL (preparation for future learning) phases</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2" /><th align="left" colspan="2"><p>Direct instruction</p></th><th align="left" colspan="2"><p>Productive failure</p></th></tr><tr><th align="left"><p>M</p></th><th align="left"><p>SD</p></th><th align="left"><p>M</p></th><th align="left"><p>SD</p></th></tr></thead><tbody><tr><td align="left"><p>Instruction and practice</p></td><td char="." align="char"><p>.87</p></td><td char="." align="char"><p>.07</p></td><td char="." align="char"><p>.81</p></td><td char="." align="char"><p>.07</p></td></tr><tr><td align="left"><p>PFL</p></td><td char="." align="char"><p>.67</p></td><td char="." align="char"><p>.12</p></td><td char="." align="char"><p>.75</p></td><td char="." align="char"><p>.11</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0138912231-13">Instruction and practice phase</hd> <p>On the recall quiz, which took place during the instruction and practice phase, the participants in the direct instruction group outperformed those in the productive failure group. The mean score on the recall quiz was.87 for learners in the direct instruction condition and.81 for learners in the productive failure condition. Although this was a secondary outcome, an independent-samples <emph>t</emph>-test was conducted to compare the two learning conditions on a practice assessment and was found to be significant, <emph>t</emph>(<reflink idref="bib1" id="ref33">1</reflink>, 38) = 2.24, <emph>p</emph> =.031.</p> <hd id="AN0138912231-14">Assessment phase</hd> <p> <emph>Acquisition:</emph> The mean score on the acquisition phase was 1 for learners in the direct instruction condition and.98 for learners in the productive failure condition (range =.67–1). Due to the non-normal distribution, an independent samples Mann–Whitney U was conducted to compare the two learning conditions and we found no significant effect of learning condition, <emph>p</emph> =.810.</p> <p> <emph>Application:</emph> The mean score on the application phase was.91 for learners in the direct instruction condition and.98 for learners in the productive failure condition. (range =.67–1). An independent-samples <emph>t</emph>-test was conducted to compare the two learning conditions and the difference was not significant, <emph>t</emph>(<reflink idref="bib1" id="ref34">1</reflink>,<reflink idref="bib38" id="ref35">38</reflink>) = 1.9, <emph>p</emph> =.06. Due to the non-normal distribution, an independent samples Mann–Whitney U was conducted to compare the two learning conditions and we found no significant effect of learning condition, <emph>p</emph> =.247.</p> <p> <emph>Preparation for future learning:</emph> Participants in the productive failure condition outperformed those in the direct instruction condition on the PFL assessment. Participants in the direct instruction condition obtained a mean score of.67 and the productive failure condition a mean score of.75 (range =.3–.9). A one-way ANCOVA was conducted to compare the two learning conditions on the PFL assessment. Performance in the instruction and practice phase was used as a covariate (<emph>p</emph> =.137). A significant effect of learning condition was found, <emph>F</emph>(<reflink idref="bib1" id="ref36">1</reflink>, 38) = 6.53, <emph>p</emph> =.04. The effect size of the difference is in the moderate range: Cohen's <emph>d</emph> = 0.69.</p> <hd id="AN0138912231-15">Discussion</hd> <p>This study investigated the effectiveness of productive failure as an instructional approach relative to direct instruction in health professions education. The specific objectives were to compare the effectiveness of productive failure with direct instruction on knowledge acquisition, knowledge application, and PFL assessments. Participants in the productive failure condition significantly outperformed those in the direct instruction condition on the PFL assessment without compromising performance on the knowledge acquisition and knowledge application assessments.</p> <p>Interestingly, participants in the direct instruction condition outperformed participants in the productive failure learning condition on the recall quiz during the instruction and practice phase. If the productive failure materials were simply superior materials, then the participants in the productive failure condition would have performed the same (or better) than those in the direct instruction condition on the recall quiz during the instruction and practice phase. However, the resulting crossover suggests that this was not the case. During the learning phase, none of the participants in the productive failure learning condition were able to generate a formula that accurately estimated creatinine clearance. So, their superior performance on the PFL assessment could be due to prior knowledge activation and differentiation during the learning phase, which may have created the opportunity to develop conceptual understanding in the instruction and practice phase. Critically, these gains in conceptual understanding occurred despite productive failure participants failing to generate an established solution during the learning phase.</p> <p>Our results support Kapur's hypothesis that placing a well-designed problem-solving phase prior to instruction may help learners to better notice and attend to the critical features of the concept (DeCaro and Rittle-Johnson [<reflink idref="bib5" id="ref37">5</reflink>]; Kapur [<reflink idref="bib7" id="ref38">7</reflink>]; Schwartz et al. [<reflink idref="bib21" id="ref39">21</reflink>]). It has been suggested that when learners struggle to discover solutions, they begin to (a) understand the problems that led to the expert theory or model, and (b) comprehend the limitations of the established solution once it is presented to them (Bjork and Bjork [<reflink idref="bib1" id="ref40">1</reflink>]; Schwartz et al. [<reflink idref="bib22" id="ref41">22</reflink>]). The struggle and invention that occurs in the problem-solving phase may be crucial if the goal is to develop deeper conceptual understanding of the constraints and limitations of estimating creatinine clearance using the Cockcroft–Gault formula. The instruction and practice phase consolidated their knowledge into the established solution and prepared learners for future problem solving in a knowledge-rich environment (Schwartz and Bransford [<reflink idref="bib20" id="ref42">20</reflink>]).</p> <p>The similar patterns of performance on the acquisition and application assessments is consistent with previous studies that used traditional assessment and found no difference in outcomes when comparing guided discovery activities with direct instruction. Structured generation activities followed by instruction (such as productive failure) can appear unproductive if standard assessments, which emphasize replication and application, are used (Schwartz and Martin [<reflink idref="bib23" id="ref43">23</reflink>]). Our results emphasize the importance of aligning instruction and assessment methods (Mylopoulos and Woods [<reflink idref="bib16" id="ref44">16</reflink>]). In this study, the PFL measure allowed us to emphasize the preparation of learners to use their knowledge for later learning, as opposed to demanding expert-like performance during learning (Woods and Mylopoulos [<reflink idref="bib24" id="ref45">24</reflink>]).</p> <p>There are a few aspects of the study that may limit transferability to clinical learning outside of the laboratory. First, the researchers tightly controlled the learning experience and the time on task was artificially limited. In real world use of these strategies, direct instruction would likely take considerably less time. Consequently, the participants in this study may have experienced boredom, which could have confounded post-test performance (Leppink [<reflink idref="bib11" id="ref46">11</reflink>]). Second, the authors developed the learning and assessment materials and tested them with a small group of novices and experts solely for use in this experimental setting. Thus, the psychometric properties of the materials generated are not available and they could not be reliably exported into a real word classroom.</p> <p>The results of this study reveal several implications for educators. First, teaching complex topics earlier in the curriculum using methods like productive failure may support three types of performance—acquisition, application, and PFL. Second, productive failure—along with other instructional approaches, such as instruction that integrates basic science mechanisms with clinical manifestations or contrasting cases—appears to foster the acquisition of knowledge that supports new learning (Woods and Mylopoulos [<reflink idref="bib24" id="ref47">24</reflink>]). Given that health professionals must regularly develop new knowledge to ensure that they adapt and advance their practice, these models of teaching and assessment could be a valuable addition to any curriculum (Mylopoulos et al. [<reflink idref="bib14" id="ref48">14</reflink>]).</p> <p>The results of this study emphasize the value of struggle during learning and support the theory that problem solving prior to instruction may be more effective than direct instruction when preparing novices to learn new knowledge in a related domain. Teaching strategies that maximize performance in the short term may not necessarily be the ones that maximize learning in the longer term (Kapur and Bielaczyc [<reflink idref="bib9" id="ref49">9</reflink>]; Schmidt and Bjork [<reflink idref="bib19" id="ref50">19</reflink>]). This study supports the idea that engaging students in solving problems that are beyond their abilities can be a productive exercise in failure.</p> <hd id="AN0138912231-16">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0138912231-17"> <title> References </title> <blist> <bibl id="bib1" idref="ref33" type="bt">1</bibl> <bibtext> Bjork E, Bjork R Gernsbacher M, Pomerantz J. Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. Psychology and the real world: Essays illustrating fundamental contributions to society. 2014: New York; Worth: 56-64</bibtext> </blist> <blist> <bibl id="bib2" idref="ref8" type="bt">2</bibl> <bibtext> Bransford JD, Schwartz DL. Rethinking transfer: A simple proposal with multiple implications. Review of Research in Education. 1999; 24: 61-100. 10.2307/1167267</bibtext> </blist> <blist> <bibl id="bib3" idref="ref14" type="bt">3</bibl> <bibtext> Bruner J. The process of education. 1960: Cambridge, Massachusetts; Harward University Press</bibtext> </blist> <blist> <bibl id="bib4" idref="ref30" type="bt">4</bibl> <bibtext> Cockcroft DW, Gault MH. Prediction of creatinine clearance from serum creatinine. Nephron. 1976; 16: 31-41. 10.1159/000180580</bibtext> </blist> <blist> <bibl id="bib5" idref="ref37" type="bt">5</bibl> <bibtext> DeCaro MS, Rittle-Johnson B. Exploring mathematics problems prepares children to learn from instruction. Journal of Experimental Child Psychology. 2012; 113; 4: 552-568. 10.1016/j.jecp.2012.06.009</bibtext> </blist> <blist> <bibl id="bib6" idref="ref26" type="bt">6</bibl> <bibtext> Kapur M. Productive failure. Cognition and Instruction. 2008; 26: 379-424. 10.1080/07370000802212669</bibtext> </blist> <blist> <bibl id="bib7" idref="ref9" type="bt">7</bibl> <bibtext> Kapur M. Productive failure in learning math. Cognitive Science. 2014; 38: 1008-1022. 10.1111/cogs.12107</bibtext> </blist> <blist> <bibl id="bib8" idref="ref23" type="bt">8</bibl> <bibtext> Kapur M. Examining productive failure, productive success, unproductive failure, and unproductive success in Learning. Educational Psychologist. 2016; 51: 289-299. 10.1080/00461520.2016.1155457</bibtext> </blist> <blist> <bibl id="bib9" idref="ref49" type="bt">9</bibl> <bibtext> Kapur M, Bielaczyc K. Designing for productive failure. Journal of the Learning Sciences. 2012; 21: 45-83. 10.1080/10508406.2011.591717</bibtext> </blist> <blist> <bibtext> Kapur M, Rummel N. Productive failure in learning from generation and invention activities. Instructional Science. 2012; 40: 645-650. 10.1007/s11251-012-9235-4</bibtext> </blist> <blist> <bibtext> Leppink J. When I say ... time on task. Medical Education. 2017; 51: 1101-1102. 10.1111/medu.13298</bibtext> </blist> <blist> <bibtext> Mamede S, van Gog T, van den Berge K, Rikers RMJP, van Saase JLCM, van Guldener C, Schmidt HG. Effect of availability bias and reflective reasoning on diagnostic accuracy among internal medicine residents. JAMA. 2010; 304; 11: 1198-1203. 10.1001/jama.2010.1276</bibtext> </blist> <blist> <bibtext> Mylopoulos M, Brydges R, Woods NN, Manzone J, Schwartz DL. Preparation for future learning: A missing competency in health professions education?. Medical Education. 2016; 50: 115-123. 10.1111/medu.12893</bibtext> </blist> <blist> <bibtext> Mylopoulos M, Kulasegaram K, Woods NN. Developing the experts we need: Fostering adaptive expertise through education. Journal of Evaluation in Clinical Practice. 2018; 24; 3: 674-677. 10.1111/jep.12905</bibtext> </blist> <blist> <bibtext> Mylopoulos M, Scardamalia M. Doctors' perspectives on their innovations in daily practice: Implications for knowledge building in health care. Medical Education. 2008; 42: 975-981. 10.1111/j.1365-2923.2008.03153.x</bibtext> </blist> <blist> <bibtext> Mylopoulos M, Woods NN. Preparing medical students for future learning using basic science instruction. Medical Education. 2014; 48: 667-673. 10.1111/medu.12426</bibtext> </blist> <blist> <bibtext> Mylopoulos M, Woods NN. When I say ... adaptive expertise. Medical Education. 2017; 51; 7: 685-686. 10.1111/medu.13247</bibtext> </blist> <blist> <bibtext> Saposnik G, Redelmeier D, Ruff CC, Tobler PN. Cognitive biases associated with medical decisions: A systematic review. BMC Medical Informatics and Decision Making. 2016; 16; 1: 138. 10.1186/s12911-016-0377-1</bibtext> </blist> <blist> <bibtext> Schmidt RA, Bjork RA. New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science. 1992; 3: 207-217. 10.1111/j.1467-9280.1992.tb00029.x</bibtext> </blist> <blist> <bibtext> Schwartz D, Bransford JD. A time for telling. Cognition and Instruction. 1998; 16: 475-5223. 10.1207/s1532690xci1604_4</bibtext> </blist> <blist> <bibtext> Schwartz D, Chase C, Oppezzo M, Chin D. Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. Journal of Educational Psychology. 2011; 103: 759-775. 10.1037/a0025140</bibtext> </blist> <blist> <bibtext> Schwartz D, Lindgren R, Lewis S Duffy TM, Tobias S. Constructivism in an age of non-constructivist assessments. Constructivist instruction: Success or failure?. 2009: London; Routledge</bibtext> </blist> <blist> <bibtext> Schwartz D, Martin T. Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction. 2004; 22: 129-184. 10.1207/s1532690xci2202_1</bibtext> </blist> <blist> <bibtext> Woods NN, Mylopoulos M. How to improve the teaching of clinical reasoning: From processing to preparation. Medical Education. 2015; 49: 952-953. 10.1111/medu.12823</bibtext> </blist> </ref> <aug> <p>By Naomi Steenhof; Nicole N. Woods; Pascal W. M. Van Gerven and Maria Mylopoulos</p> <p>Reported by Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib13" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib17" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib15" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib12" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib18" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib16" firstref="ref10"></nolink> <nolink nlid="nl7" bibid="bib24" firstref="ref11"></nolink> <nolink nlid="nl8" bibid="bib20" firstref="ref16"></nolink> <nolink nlid="nl9" bibid="bib23" firstref="ref17"></nolink> <nolink nlid="nl10" bibid="bib10" firstref="ref18"></nolink> <nolink nlid="nl11" bibid="bib22" firstref="ref19"></nolink> <nolink nlid="nl12" bibid="bib38" firstref="ref35"></nolink> <nolink nlid="nl13" bibid="bib21" firstref="ref39"></nolink> <nolink nlid="nl14" bibid="bib11" firstref="ref46"></nolink> <nolink nlid="nl15" bibid="bib14" firstref="ref48"></nolink> <nolink nlid="nl16" bibid="bib19" firstref="ref50"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1230253
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Productive Failure as an Instructional Approach to Promote Future Learning
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Steenhof%2C+Naomi%22">Steenhof, Naomi</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0001-7931-9134">0000-0001-7931-9134</externalLink>)<br /><searchLink fieldCode="AR" term="%22Woods%2C+Nicole+N%2E%22">Woods, Nicole N.</searchLink><br /><searchLink fieldCode="AR" term="%22Van+Gerven%2C+Pascal+W%2E+M%2E%22">Van Gerven, Pascal W. M.</searchLink><br /><searchLink fieldCode="AR" term="%22Mylopoulos%2C+Maria%22">Mylopoulos, Maria</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Advances+in+Health+Sciences+Education%22"><i>Advances in Health Sciences Education</i></searchLink>. Oct 2019 24(4):739-749.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 11
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2019
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Failure%22">Failure</searchLink><br /><searchLink fieldCode="DE" term="%22Teaching+Methods%22">Teaching Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Solving%22">Problem Solving</searchLink><br /><searchLink fieldCode="DE" term="%22Instructional+Effectiveness%22">Instructional Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Doctoral+Students%22">Doctoral Students</searchLink><br /><searchLink fieldCode="DE" term="%22Pharmaceutical+Education%22">Pharmaceutical Education</searchLink><br /><searchLink fieldCode="DE" term="%22Direct+Instruction%22">Direct Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Canada+%28Toronto%29%22">Canada (Toronto)</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s10459-019-09895-4
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1382-4996
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Productive failure is an instructional approach that requires learners to struggle as they attempt to generate solutions to problems before, rather than after, receiving direct instruction on a targeted concept. Studies demonstrate that productive failure prepares students for later learning of new, related knowledge. Our study explored the effectiveness of productive failure as an instructional intervention in health professions education with respect to (a) acquisition and application of a novel concept, and (b) learners' preparation for future learning of new, related content. Forty year-one students enrolled in the Doctor of Pharmacy program at the University of Toronto were randomly assigned to a productive failure (i.e. attempt to generate solutions before receiving instruction) or direct instruction only learning condition. After a practice phase, participants completed a series of tests designed to measure knowledge acquisition, knowledge application, and preparation for future learning (new learning is required for successful problem solving). As expected, no difference in performance was seen between participants on the acquisition and application tests. However, participants in the productive failure condition outperformed those in the direct instruction condition on the preparation for future learning test. These results emphasize the role of struggle in learning and support the theory that engaging students in solving problems that are beyond their abilities can be a productive exercise in failure. The results suggest that productive failure assists learners in acquiring the conceptual knowledge needed to facilitate learning in the future.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2019
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1230253
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1230253
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10459-019-09895-4
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
        StartPage: 739
    Subjects:
      – SubjectFull: Failure
        Type: general
      – SubjectFull: Teaching Methods
        Type: general
      – SubjectFull: Problem Solving
        Type: general
      – SubjectFull: Instructional Effectiveness
        Type: general
      – SubjectFull: Doctoral Students
        Type: general
      – SubjectFull: Pharmaceutical Education
        Type: general
      – SubjectFull: Direct Instruction
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Canada (Toronto)
        Type: general
    Titles:
      – TitleFull: Productive Failure as an Instructional Approach to Promote Future Learning
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Steenhof, Naomi
      – PersonEntity:
          Name:
            NameFull: Woods, Nicole N.
      – PersonEntity:
          Name:
            NameFull: Van Gerven, Pascal W. M.
      – PersonEntity:
          Name:
            NameFull: Mylopoulos, Maria
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Type: published
              Y: 2019
          Identifiers:
            – Type: issn-print
              Value: 1382-4996
          Numbering:
            – Type: volume
              Value: 24
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Advances in Health Sciences Education
              Type: main
ResultId 1