View in EDS HTML Full Text PDF Full Text

School Students' Confidence When Answering Diagnostic Questions Online

Saved in:
Bibliographic Details
Title: School Students' Confidence When Answering Diagnostic Questions Online
Language: English
Authors: Foster, Colin (ORCID 0000-0003-1648-7485), Woodhead, Simon (ORCID 0000-0002-2192-9797), Barton, Craig (ORCID 0000-0003-3566-2795), Clark-Wilson, Alison (ORCID 0000-0002-6479-4677)
Source: Educational Studies in Mathematics. Mar 2022 109(3):491-521.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 31
Publication Date: 2022
Document Type: Journal Articles
Reports - Research
Education Level: Elementary Education
Secondary Education
Elementary Secondary Education
Descriptors: Foreign Countries, Elementary School Students, Secondary School Students, Multiple Choice Tests, Diagnostic Tests, Confidence Testing, Computer Assisted Testing, Gender Differences, Socioeconomic Status, Age Differences, Mathematics Tests
Geographic Terms: United Kingdom
DOI: 10.1007/s10649-021-10084-7
ISSN: 0013-1954
Abstract: In this paper, we analyse a large, opportunistic dataset of responses (N = 219,826) to online, diagnostic multiple-choice mathematics questions, provided by 6-16-year-old UK school mathematics students (N = 7302). For each response, students were invited to indicate on a 5-point Likert-type scale how confident they were that their response was correct. Using demographic data available from the online platform, we examine the relationships between confidence and facility (the proportion of questions correct), as well as gender, age and socioeconomic disadvantage. We found a positive correlation between student confidence and mean facility, higher confidence for boys than for girls and lower confidence for students classified as socioeconomically disadvantaged, even after accounting for facility. We found that confidence was lower for older students, and this was particularly marked across the primary to secondary school transition. An important feature of the online platform used is that, when students answer a question incorrectly, they are presented with an analogous question about 3 weeks later. We exploited this feature to obtain the first evidence in an authentic school mathematics context for the "hypercorrection effect" (Butterfield & Metcalfe J EXP PSYCHOL 27:1491-1494, 2001), which is the observation that errors made with higher confidence are more likely to be corrected. These findings have implications for classroom practices that have the potential to support more effective and efficient learning of mathematics.
Abstractor: As Provided
Entry Date: 2022
Accession Number: EJ1326555
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwF7_wN5RSnxAtdI_hKdUC9IAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDP07jZ7ojLOrnamkmAIBEICBmw4lmaywnQB_Iq5Ny2yxeioWUwv7DqzfrwFxg9ecXplimovvWu1VUVqkme8nqIbeSzT5GO6-_KbTYPHa4thMt3mc3_G4MHitG-o5oTaTMxWNBRy-Eiw5Zim2OJH6rFm7416vaqk3R6J8Vh5VmZAFDZg3cwvLPzpFPYKPnJ5NrsdeolK14Yh-3VvxVp8LaAcjH60W0ZGjjOUm5Fy4
Text:
  Availability: 1
  Value: <anid>AN0155342115;esm01mar.22;2022Feb22.04:27;v2.2.500</anid> <title id="AN0155342115-1">School students' confidence when answering diagnostic questions online </title> <p>In this paper, we analyse a large, opportunistic dataset of responses (N = 219,826) to online, diagnostic multiple-choice mathematics questions, provided by 6–16-year-old UK school mathematics students (N = 7302). For each response, students were invited to indicate on a 5-point Likert-type scale how confident they were that their response was correct. Using demographic data available from the online platform, we examine the relationships between confidence and facility (the proportion of questions correct), as well as gender, age and socioeconomic disadvantage. We found a positive correlation between student confidence and mean facility, higher confidence for boys than for girls and lower confidence for students classified as socioeconomically disadvantaged, even after accounting for facility. We found that confidence was lower for older students, and this was particularly marked across the primary to secondary school transition. An important feature of the online platform used is that, when students answer a question incorrectly, they are presented with an analogous question about 3 weeks later. We exploited this feature to obtain the first evidence in an authentic school mathematics context for the hypercorrection effect (Butterfield & Metcalfe J EXP PSYCHOL 27:1491–1494, 2001), which is the observation that errors made with higher confidence are more likely to be corrected. These findings have implications for classroom practices that have the potential to support more effective and efficient learning of mathematics.</p> <p>Keywords: Assessment; Confidence; Diagnostic questions; Hypercorrection effect; Misconceptions</p> <hd id="AN0155342115-2">Introduction</hd> <p>The importance of school students' confidence in their learning of mathematics has been increasingly recognised in recent years (e.g., Foster, [<reflink idref="bib21" id="ref1">21</reflink>], [<reflink idref="bib22" id="ref2">22</reflink>]; Hannula, [<reflink idref="bib32" id="ref3">32</reflink>]). The mathematics education literature conceptualises students' mathematics confidence within the affective domain (e.g., Code et al., [<reflink idref="bib13" id="ref4">13</reflink>]; Di Martino & Zan, [<reflink idref="bib16" id="ref5">16</reflink>]; Hannula et al., [<reflink idref="bib33" id="ref6">33</reflink>]; Kyriacou, [<reflink idref="bib35" id="ref7">35</reflink>]; Pepin & Roesken-Winter, [<reflink idref="bib51" id="ref8">51</reflink>]), where there is an abundance of similar and sometimes overlapping constructs relating to confidence (see Clarkson et al., [<reflink idref="bib11" id="ref9">11</reflink>]; Marsh et al., [<reflink idref="bib38" id="ref10">38</reflink>]). The focus for this study is on a fine-grained notion of confidence, corresponding to a student's post hoc judgment on the probability that the answer that they have just given is correct (see Bandura, [<reflink idref="bib2" id="ref11">2</reflink>]; Foster, [<reflink idref="bib21" id="ref12">21</reflink>], [<reflink idref="bib22" id="ref13">22</reflink>]; Stankov et al., [<reflink idref="bib61" id="ref14">61</reflink>]). However, we see this understanding of confidence as being situated within broader notions of "mathematics confidence" (Burton, [<reflink idref="bib6" id="ref15">6</reflink>]; Galbraith & Haines, [<reflink idref="bib25" id="ref16">25</reflink>]; Pierce & Stacey, [<reflink idref="bib52" id="ref17">52</reflink>]) that relate to positive feelings about mathematics in general and studying mathematics in particular (see Fennema & Sherman, [<reflink idref="bib19" id="ref18">19</reflink>]; Lim & Chapman, [<reflink idref="bib36" id="ref19">36</reflink>]).</p> <p>Student confidence in mathematics is associated with higher attainment (Mullis et al., [<reflink idref="bib46" id="ref20">46</reflink>]), and, in an early meta-analysis of 113 primary studies, Ma and Kishor ([<reflink idref="bib37" id="ref21">37</reflink>]) found evidence for a <emph>causal</emph> effect of attitude to mathematics on mathematics attainment. Since then, increasingly strong evidence has accumulated for a reciprocal, bidirectional relation between confidence and attainment (e.g., Ganley & Lubienski, [<reflink idref="bib27" id="ref22">27</reflink>]; Pinxten et al., [<reflink idref="bib53" id="ref23">53</reflink>]). Whether students feel confident about their learning of mathematics or not affects whether they find learning mathematics in school a happy or an unhappy experience (Foster, [<reflink idref="bib21" id="ref24">21</reflink>]; Hannula, [<reflink idref="bib32" id="ref25">32</reflink>]; Morsanyi et al., [<reflink idref="bib45" id="ref26">45</reflink>]), influences their success in this critical gatekeeping subject and predicts their future career choices (Franz-Odendaal et al., [<reflink idref="bib23" id="ref27">23</reflink>]; Paulsen & Betz, [<reflink idref="bib50" id="ref28">50</reflink>]).</p> <p>In addition to the relationship with attainment, research has also found associations between students' mathematics confidence and a variety of demographic variables, including gender, age and socioeconomic disadvantage. Students' confidence is lower for girls, for older students and for students characterised as socioeconomically disadvantaged (e.g., Ganley & Lubienski, [<reflink idref="bib27" id="ref29">27</reflink>]; Mullis et al., [<reflink idref="bib46" id="ref30">46</reflink>]; OECD, [<reflink idref="bib47" id="ref31">47</reflink>], [<reflink idref="bib48" id="ref32">48</reflink>]). However, the detailed relationships among these variables are not known, such as whether the relative under-confidence of girls varies with age or socioeconomic disadvantage, whether the decrease in student confidence with age can be fully accounted for by a decrease in attainment or whether the lower confidence of socioeconomically disadvantaged students can be fully accounted for by lower attainment. Understanding these relationships could give schools important insights into ways to support students' learning of mathematics more effectively and equitably (see Foster, [<reflink idref="bib21" id="ref33">21</reflink>], [<reflink idref="bib22" id="ref34">22</reflink>]).</p> <p>The opportunistic dataset that forms the basis for this study derives from the <emph>Eedi</emph> online platform (eedi.co.uk), created by the second and third authors, which provides teachers with free access to over 50,000 multiple-choice diagnostic mathematics questions (e.g., see Fig. 1). As can be seen at the bottom of Fig. 1, a distinctive feature of the <emph>Eedi</emph> platform is its incorporation of a <emph>confidence assessment</emph> for every diagnostic question, with students invited to indicate after each response which of five confidence emojis (, , , or ) best reflects their confidence level (Barton, [<reflink idref="bib5" id="ref35">5</reflink>]). Although confidence assessment instruments have been used frequently within the research literature, this feature is innovative from the point of view of assessment tools currently widely available to, and popular with, classroom teachers of mathematics. This study constitutes a response to calls for "the mathematics education research community to support companies to adopt more evidence-led approaches both in the design and evaluation of their products" (Clark-Wilson et al., [<reflink idref="bib12" id="ref36">12</reflink>], p. 1237).</p> <p>Graph: Fig. 1 A typical question presentation</p> <p>The platform is intended for use by teachers as a formative assessment tool (Barton, [<reflink idref="bib3" id="ref37">3</reflink>], [<reflink idref="bib4" id="ref38">4</reflink>], [<reflink idref="bib5" id="ref39">5</reflink>]) and is currently used by students in over 5800 schools worldwide (over 4500 of which are in the UK). The <emph>Eedi</emph> platform is free to all teachers across the world and contains diagnostic questions designed for all ages from 5 to 16. While the site does include questions specifically tailored to the English national curriculum, the <emph>Eedi Ultimate Scheme of Work</emph>[<reflink idref="bib1" id="ref40">1</reflink>] is designed to be suitable for use with any curriculum. There are more than 200 topics units, comprising more than 3000 questions that teachers can map to their curricula.</p> <p>An <emph>Eedi</emph> Diagnostic Question is a multiple-choice question with one correct answer and three incorrect distractors. As usual with diagnostic questions, each distractor is carefully designed to provoke or reveal a different common error or misconception. A student responds by selecting one of four options (A, B, C or D), and an optional free-text box is available for the student's explanation. For the example question shown in Fig. 1, the percentages of students responding to each of the options were as follows: A (17%), B (correct, 58%), C (19%) and D (6%). The diagnostic feature of this is that the two most common incorrect responses (A and C) may be taken to indicate very different kinds of student difficulties. Students responding with A might be experiencing confusion with negative numbers, whereas students responding with C might have confused "descending" with "ascending". It is clear in this case that an appropriate teacher response could be quite different for these two groups of students: focusing teaching on directed numbers may be of no help at all to students who responded with C. The <emph>Eedi</emph> platform provides easy and cost-free access for teachers to a very large collection of such questions, which they can use formatively to enhance their classroom teaching. The dataset generated is large and suitable for research analysis.</p> <p>Diagnostic Questions may be used to assess facts, procedures or concepts (see Fig. 2 for examples of each). Diagnostic Questions are particularly well suited to assessing recall of facts (Fig. 2a) or simple procedures (Fig. 2b). However, if carefully designed, they can also be used to assess performance on <emph>multi-step</emph> procedures, by isolating one stage in the procedure (Fig. 2c). This provides more helpful information for a teacher, as it reveals exactly where in a process a student has difficulty. Diagnostic Questions are designed with the intention that students spend no longer than 1 min considering their answer. If a fact- or procedure-based question takes longer to answer than this, then it is likely that the student is having to consider several sub-steps, which makes it difficult to capture the specific nature of their misunderstanding in the three distractors. Finally, Diagnostic Questions can also be used to assess conceptual understanding, as in the example shown in Fig. 2d. Here, students might spend longer considering their answer, potentially thinking about various examples and non-examples in order to reach their final decision.</p> <p>Graph: Fig. 2 Examples of diagnostic questions assessing (a) a fact (link to question: https://diagnosticquestions.com/Questions/Go#/93384); (b) a simple procedure (link to question: https://diagnosticquestions.com/Questions/Go#/107849); (c) a multi-step procedure (link to question: https://diagnosticquestions.com/Questions/Go#/106633); (d) a concept (link to question: https://diagnosticquestions.com/Questions/Go#/99575)</p> <p>A further feature of the <emph>Eedi</emph> platform allows us to explore evidence for the <emph>hypercorrection effect</emph>, which is the observation that errors made with high confidence are more easily corrected than those made with low confidence (Butterfield & Metcalfe, [<reflink idref="bib8" id="ref41">8</reflink>]; see Barton, [<reflink idref="bib5" id="ref42">5</reflink>]). On the <emph>Eedi</emph> platform, teachers assign "quizzes", each consisting of 5–20 Diagnostic Questions, and students complete these quizzes on the website or via the mobile app, either during school time or as part of home learning. Teachers may also assign their class a "scheme of work", comprising a sequence of topic units, each of which includes <emph>two</emph> quizzes covering the same content, with the same level of difficulty. One is intended for use immediately after the topic has been taught (quiz A) and the other as a review quiz administered about 3 weeks later (quiz B). The questions in these two quizzes differ only in the numbers used and the order in which the responses are presented and are intended to test exactly the same constructs and reveal exactly the same misconceptions. Quiz B is intended to take advantage of spaced retrieval practice (Rohrer & Taylor, [<reflink idref="bib55" id="ref43">55</reflink>]) and to distinguish "learning" from "performance" (Soderstrom & Bjork, [<reflink idref="bib59" id="ref44">59</reflink>]) by revealing what knowledge has been retained and whether errors or misconceptions have remained or changed. For our purposes, the quiz A–quiz B structure also enables us to explore evidence for the hypercorrection effect in students' responses on quiz B, administered 3 weeks after quiz A. We are not aware of any previous study examining the hypercorrection effect in school mathematics in authentic learning settings such as this.</p> <p>Consequently, our research questions for this study, using the data obtained from the <emph>Eedi</emph> platform, are:</p> <p></p> <ulist> <item> How are students' confidence ratings related to their mean facility, age, gender and socioeconomic disadvantage, and how do these variables interact?</item> <p></p> <item> Is there evidence for the hypercorrection effect in students' responses to a second set of questions (quiz B) administered 3 weeks after the first (quiz A)?</item> </ulist> <hd id="AN0155342115-3">Confidence and learning mathematics</hd> <p>The affective domain in mathematics has been studied for at least 30 years, and the mathematics education literature conceives of students' mathematical confidence as a component of the affective domain (see Code et al., [<reflink idref="bib13" id="ref45">13</reflink>]; Di Martino & Zan, [<reflink idref="bib16" id="ref46">16</reflink>]; Hannula et al., [<reflink idref="bib33" id="ref47">33</reflink>]; Kyriacou, [<reflink idref="bib35" id="ref48">35</reflink>]; Pepin & Roesken-Winter, [<reflink idref="bib51" id="ref49">51</reflink>]). Within this literature, there are multiple similar and overlapping confidence-related constructs, ranging in scope from broad to narrow (see Clarkson et al., [<reflink idref="bib11" id="ref50">11</reflink>]; Marsh et al., [<reflink idref="bib38" id="ref51">38</reflink>]). At the broad end, there are subject-level notions of "mathematics confidence" (e.g., Burton, [<reflink idref="bib6" id="ref52">6</reflink>]; Pierce & Stacey, [<reflink idref="bib52" id="ref53">52</reflink>]), such as Galbraith & Haines's ([<reflink idref="bib25" id="ref54">25</reflink>], p. 278) definition that mathematically confident students "believe they obtain value for effort, do not worry about learning hard topics, expect to get good results, and feel good about mathematics as a subject". Such constructs have often been measured using variations on the <emph>Confidence in Learning Mathematics Scale</emph> within the <emph>Fennema-Sherman Mathematics Attitudes Scales</emph> (Fennema & Sherman, [<reflink idref="bib19" id="ref55">19</reflink>]; Lim & Chapman, [<reflink idref="bib36" id="ref56">36</reflink>]). At the finer-grained level, more focused on specific tasks or items, are constructs such as mathematical self-efficacy (Bandura, [<reflink idref="bib2" id="ref57">2</reflink>]), which is a student's belief about their chances of successfully performing a <emph>future</emph> mathematical task, and <emph>confidence of response</emph>, a student's post hoc judgment on the probability that the answer that they have just given is correct (Foster, [<reflink idref="bib21" id="ref58">21</reflink>], [<reflink idref="bib22" id="ref59">22</reflink>]; Stankov et al., [<reflink idref="bib61" id="ref60">61</reflink>]).</p> <p>Previous research has identified associations between confidence and a variety of important demographic characteristics, which we attempt to replicate and build on in this study. In particular, the <emph>hypercorrection effect</emph> (Butterfield & Metcalfe, [<reflink idref="bib8" id="ref61">8</reflink>]), which is the observation that errors made with high confidence are more likely to be corrected than those made with low confidence, has been repeatedly demonstrated in laboratory settings (e.g., Butler et al., [<reflink idref="bib7" id="ref62">7</reflink>]; Metcalfe & Finn, [<reflink idref="bib42" id="ref63">42</reflink>], [<reflink idref="bib43" id="ref64">43</reflink>]), but this study will be the first attempt to replicate it in an authentic mathematics learning situation.</p> <hd id="AN0155342115-4">Associations between confidence and facility, gender, socioeconomic disadvantage and age</hd> <p>The 2019 Trends in Maths and Science Study (TIMSS) found that for students in both years 5 and 9 (ages 9–10 and 13–14 respectively), there was a strong positive association between their confidence in their mathematical ability and their average attainment (Mullis et al., [<reflink idref="bib46" id="ref65">46</reflink>]). Quantitative studies have revealed the most likely model to be a reciprocal, bidirectional relation between confidence and attainment (Ganley & Lubienski, [<reflink idref="bib27" id="ref66">27</reflink>]; Ma & Kishor, [<reflink idref="bib37" id="ref67">37</reflink>]; Pinxten et al., [<reflink idref="bib53" id="ref68">53</reflink>]). Fischhoff et al. ([<reflink idref="bib20" id="ref69">20</reflink>], p. 552) defined a student as being <emph>well calibrated</emph> "if, over the long run, for all propositions assigned a given probability, the proportion that is true is equal to the probability assigned"; in other words, students accurately estimate the probability that their responses will be correct. Students have often been found to be well calibrated, and Foster ([<reflink idref="bib21" id="ref70">21</reflink>]) previously reported a correlation of <emph>r</emph> =.546 between facility and mean confidence for 345 students aged 11–14 in the topic of directed numbers. However, children also often overestimate their performance, believing that this will lead others to view them more favourably (see Panaoura et al., [<reflink idref="bib49" id="ref71">49</reflink>]). Several additional factors are also known to be related to students' confidence in mathematics, and principal among these are gender, socioeconomic disadvantage and age.</p> <p>Research has repeatedly found that girls show lower confidence in mathematics than boys do (e.g., Frost et al., [<reflink idref="bib24" id="ref72">24</reflink>]; Ganley & Lubienski, [<reflink idref="bib27" id="ref73">27</reflink>]; Mullis et al., [<reflink idref="bib46" id="ref74">46</reflink>]), even when girls obtain higher mathematics grades than boys (Pomerantz et al., [<reflink idref="bib54" id="ref75">54</reflink>]). The 2019 TIMSS found that in England more 13–14-year-old boys than girls (18% compared to 10%) expressed being "very confident", and more girls than boys (44% compared to 30%) expressed being "not confident" (Mullis et al., [<reflink idref="bib46" id="ref76">46</reflink>]). Similarly, the 2012 Programme for International Student Assessment (PISA) international comparative study found that, among 15-year-old students, more girls (35%) than boys (25%) reported feeling helpless when doing mathematics problems (OECD, [<reflink idref="bib47" id="ref77">47</reflink>]), and it seems likely that societal stereotypes about gendered roles contribute to girls' lower confidence in mathematics (Mendick & Francis, [<reflink idref="bib40" id="ref78">40</reflink>]). More recent PISA studies have not asked students to judge their capabilities in particular subjects, such as mathematics, and instead have asked students to report on "their general sense of efficacy, or competence, particularly in the face of adversity" (OECD, [<reflink idref="bib48" id="ref79">48</reflink>], p. 190). PISA 2018 found that, in almost every country, girls expressed greater fear of failure than boys did, and this gender gap was much wider among higher-achieving students. Despite a considerable body of research, it is not known to what extent the relative under-confidence of girls relates to other variables, such as age and socioeconomic disadvantage, and these are questions we will address in this study.</p> <p>The same 2012 PISA study (OECD, [<reflink idref="bib47" id="ref80">47</reflink>]) also found that more socioeconomically disadvantaged students (35%) than advantaged students (24%) reported feeling helpless when doing mathematics problems, and the most recent 2018 PISA study found that, in almost every country, socioeconomically disadvantaged students reported lower general self-confidence than their more advantaged peers (OECD, [<reflink idref="bib48" id="ref81">48</reflink>]). Whether this lower confidence among socioeconomically disadvantaged students is simply related to lower attainment, or is more than would be predicted purely on the basis of attainment, is not known, and one of the aims of this study is to explore this.</p> <p>Finally, students' confidence in mathematics is known to reduce as they get older and progress through schooling (Galton et al., [<reflink idref="bib26" id="ref82">26</reflink>]; Greany et al., [<reflink idref="bib31" id="ref83">31</reflink>]; Zanobini & Usai, [<reflink idref="bib63" id="ref84">63</reflink>]). This may be a consequence of the mathematics that they are learning becoming perceived as increasingly abstract and difficult, leading to greater disaffection and a reduction in confidence. It may also be related to students' developing metacognition and self-representation, as they become more conscious of any difficulties and limitations. The 2019 TIMSS study (Mullis et al., [<reflink idref="bib46" id="ref85">46</reflink>]) found that student confidence declined with age from primary to secondary school, with the proportion of students reporting themselves to be "very confident" or "somewhat confident" in mathematics decreasing from 76% in year 5 (age 9–10) to 63% in year 9 (age 13–14) (Mullis et al., [<reflink idref="bib46" id="ref86">46</reflink>]). Once again, it is not known the extent to which this decrease in confidence can be accounted for purely as a result of a decrease in success with mathematics, or whether additional factors are involved, and our analysis will also address that question.</p> <hd id="AN0155342115-5">The hypercorrection effect</hd> <p>A further focus of the present study is the <emph>hypercorrection effect</emph>, which is the observation that errors made with high confidence are more likely to be corrected than those made with low confidence (Butterfield & Metcalfe, [<reflink idref="bib8" id="ref87">8</reflink>]). The effect has been repeatedly demonstrated in laboratory settings, usually involving college students being tested on general-knowledge questions, but has also been found in young children (Metcalfe & Finn, [<reflink idref="bib43" id="ref88">43</reflink>]). The hypercorrection effect has been reported in studies using educationally relevant material in science (van Loon et al., [<reflink idref="bib62" id="ref89">62</reflink>]) and in an authentic classroom context with college students studying horticulture (Carpenter et al., [<reflink idref="bib10" id="ref90">10</reflink>]). However, we are not aware of any studies examining the hypercorrection effect in mathematics, or in authentic school settings as part of educationally relevant assessment, and exploring this, with a large dataset, is one of the main aims of this study.</p> <p>The hypercorrection effect is surprising on theoretical grounds: it seems plausible that errors made with high confidence should be <emph>more difficult</emph> to correct, since we would expect them to be more deeply entrenched (Butterfield & Metcalfe, [<reflink idref="bib9" id="ref91">9</reflink>]). One possible mechanism for hypercorrection is that the surprise experienced when a person finds that their high confidence response is wrong heightens their attentional resources and so enhances memory (Butterfield & Metcalfe, [<reflink idref="bib9" id="ref92">9</reflink>]). Alternatively, it may be that high confidence errors might act as mediators to the correct response, by associating the correct response with the error (the <emph>semantic mediation hypothesis</emph>, see Metcalfe & Huelser, [<reflink idref="bib44" id="ref93">44</reflink>]). Finally, it may be that high confidence over an error suggests that it lies in a richly developed semantic landscape, where familiarity with the domain makes it more likely that the correct information is stored in semantic memory, even if not strongly enough to be given as the initial response (Butterfield & Metcalfe, [<reflink idref="bib9" id="ref94">9</reflink>]). Study participants are more likely to claim that they "knew it all along" when they are corrected after making an error with high confidence than when making a less confident error (Metcalfe & Finn, [<reflink idref="bib42" id="ref95">42</reflink>]). More recently, it has been suggested that error correction is mainly related to prior knowledge, and that confidence is primarily a proxy for prior knowledge (Sitzman et al., [<reflink idref="bib58" id="ref96">58</reflink>]).</p> <p>Hypercorrection effects have been found to be persistent over time (Butler et al., [<reflink idref="bib7" id="ref97">7</reflink>]) and appear to offer the potential to improve learning in the classroom (Barton, [<reflink idref="bib5" id="ref98">5</reflink>]; de Bruin & van Gog, [<reflink idref="bib15" id="ref99">15</reflink>]; Efklides, [<reflink idref="bib18" id="ref100">18</reflink>]). Trying to avoid students making errors may be a counterproductive goal for the teacher, and it may be preferable to allow and even encourage students to make mistakes in low-stakes assessments. This may be especially valuable if, when these errors are made with high confidence, they are likely to be hypercorrected (Metcalfe, [<reflink idref="bib41" id="ref101">41</reflink>]). Consequently, the possibility of attempting to harness the hypercorrection effect for practical use in the mathematics classroom has recently been proposed (Barton, [<reflink idref="bib3" id="ref102">3</reflink>], [<reflink idref="bib5" id="ref103">5</reflink>]). However, it is not known to what extent the hypercorrection effect operates in authentic learning situations in mathematics, and testing this is an important aim of this study.</p> <hd id="AN0155342115-6">Method</hd> <p></p> <hd id="AN0155342115-7">Participants</hd> <p>Data were collected from students who used the <emph>Eedi</emph> online platform (eedi.co.uk)[<reflink idref="bib2" id="ref104">2</reflink>] between 7 March 2019 and 9 May 2020. A total of 327,737 students accessed the platform during this period, from a total of 3083 schools. This study draws its findings from a sample of UK students who were assigned two quizzes on the same topic, 3 weeks apart, completed the second quiz no later than 4 weeks after the first, answered at least one question from the first quiz and gave a measure of confidence. In between the two quizzes, students would typically work on different topics from those addressed in the quizzes.</p> <p>We focused on UK students because these were the ones for whom we had access to additional metadata of gender, age and social disadvantage (Free School Meals[<reflink idref="bib3" id="ref105">3</reflink>] and Pupil Premium[<reflink idref="bib4" id="ref106">4</reflink>] statuses). We excluded students who answered fewer than 5 questions in total, and we excluded answers which were more than a week late. This resulted in a dataset consisting of <emph>N</emph> = 7302 students from 152 schools, with a total of 219,826 responses (<emph>M</emph> = 30.1 responses per student, <emph>SD</emph> = 42.7). For 119,211 of these responses, there was a corresponding answer in quiz B, and these were from 4123 unique users.</p> <p>For these students, demographic data revealed that 3586 were female (49.1%) and 1541 students (21.1%) were disadvantaged, as measured by eligibility for either Free School Meals (Gorard, [<reflink idref="bib28" id="ref107">28</reflink>]) or Pupil Premium (Gov.UK, [<reflink idref="bib29" id="ref108">29</reflink>]) (see Table 1 for the breakdown of the sample by gender, disadvantage and age). Currently, across England, 17.3% of pupils are known to be eligible for Free School Meals (Gov.UK, [<reflink idref="bib30" id="ref109">30</reflink>]), so, on this measure, our sample is slightly more disadvantaged than the national average.</p> <p>Table 1 Sociodemographic characteristics of participants</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th rowspan="2"><p>Baseline characteristic</p></th><th colspan="2"><p>Partial sample (September 2019 to May 2020)</p></th><th colspan="2"><p>Full sample (March 2019 to May 2020)</p></th></tr><tr><th><p><italic>N</italic> = 5382</p></th><th><p>%</p></th><th><p><italic>N</italic> = 7302</p></th><th><p>%</p></th></tr></thead><tbody><tr><td align="left" colspan="5"><p>Gender</p></td></tr><tr><td><p> Female</p></td><td><p>2561</p></td><td><p>48%</p></td><td><p>3716</p></td><td><p>51%</p></td></tr><tr><td><p> Male</p></td><td><p>2821</p></td><td><p>52%</p></td><td><p>3586</p></td><td><p>49%</p></td></tr><tr><td align="left" colspan="5"><p>Age</p></td></tr><tr><td><p> 5–6</p></td><td><p>0</p></td><td><p>0%</p></td><td><p>59</p></td><td><p>1%</p></td></tr><tr><td><p> 7–10</p></td><td><p>529</p></td><td><p>10%</p></td><td><p>2910</p></td><td><p>40%</p></td></tr><tr><td><p> 11–13</p></td><td><p>3339</p></td><td><p>62%</p></td><td><p>2680</p></td><td><p>37%</p></td></tr><tr><td><p> 14–16</p></td><td><p>1514</p></td><td><p>28%</p></td><td><p>1653</p></td><td><p>23%</p></td></tr><tr><td align="left" colspan="5"><p>Disadvantage</p></td></tr><tr><td><p> Advantaged</p></td><td><p>4186</p></td><td><p>78%</p></td><td><p>5761</p></td><td><p>79%</p></td></tr><tr><td><p> Disadvantaged</p></td><td><p>1196</p></td><td><p>22%</p></td><td><p>1541</p></td><td><p>21%</p></td></tr></tbody></table> </ephtml> </p> <p> <sups>a</sups>Age is students' age at the beginning of the academic year in which the trial started. This was 1 September 2018 for the full sample and 1 September 2019 for the partial sample.</p> <p>For the analysis by age, we restricted the dataset to responses given between September 2019 and May 2020 (within one academic year), so that each student's answers fell within a single academic year, allowing us to classify each student within a single year. This gave us a dataset consisting of <emph>N</emph> = 5382 students from 92 schools, with a total of 168,922 responses (see Table 1).</p> <hd id="AN0155342115-8">Instrument</hd> <p>Although the diagnostic questions on the <emph>Eedi</emph> platform are not formally trialled or calibrated with students before use in order to ascertain that the reasons for the students' incorrect responses align precisely with those intended by the question writers, the students did have the opportunity to respond in a free-text box with a reason for their answers. Examination of these responses suggests that there is generally a very strong alignment between the students' reasoning and the question writers' intentions behind the distractors, which are founded on their extensive practice-based knowledge and experience. We are confident of the validity of these questions, since they have been very widely used in thousands of schools, and <emph>Eedi</emph> applies a robust quality-control process of internal peer review on all questions submitted.</p> <p>As described above, if students choose to express how confident they are about their answer (A, B, C or D), then they select from one of 5 emojis, from sad to happy. Using emoji scales with children can lead to more reliable data, since the faces may be easier for children to interpret than words or numbers (see Kaye et al., [<reflink idref="bib34" id="ref110">34</reflink>]; Massey, [<reflink idref="bib39" id="ref111">39</reflink>]). We coded the five confidence emojis as 0 , 25 , 50 , 75 and 100 , so that their mean values could be directly compared with mean percentage facilities (percentage of questions answered correctly) for each student, assuming a linear scale of confidence.</p> <p>As explained above, our data for investigating the hypercorrection effect derived from students who answered both quiz A and quiz B. For this analysis, it is important that each pair of analogous questions should be of equal difficulty, and in the Appendix, we provide evidence that this is the case. Although the questions we analysed are grouped into quizzes, in this analysis, we make the assumption that the hypercorrection effect can be measured between paired questions, without considering the overall quiz performance.</p> <hd id="AN0155342115-9">Ethics</hd> <p>By using the <emph>Eedi</emph> website, all participants consented for their anonymised data to be used in this analysis. This paper uses personal data collected by <emph>Eedi</emph> under the lawful basis of legitimate interests (GDPR Art. 6 (<reflink idref="bib1" id="ref112">1</reflink>) (f)). The privacy notices provided to participants using the <emph>Eedi</emph> service explain that their data will be further analysed, under the same lawful basis, for research purposes, and that any results published will only include aggregated statistics. The dataset publicly available on <emph>GitHub</emph> contains only anonymised data. Because this study involved analysing only anonymous pre-existing data, and because all participants had consented for their data to be used for research purposes, no institutional ethical approval was required.</p> <hd id="AN0155342115-10">Overview of the analytic approach</hd> <p>We used a range of quantitative methods to address the two research questions, including linear and logistic regression, non-parametric tests and mediation analysis. Mediation analysis is a method for revealing whether the effect of one variable on another is direct or is transmitted partially or fully via a third variable.</p> <p>For our first research question (<emph>How are students' confidence ratings related to their mean facility, age, gender and socioeconomic disadvantage, and how do these variables interact?</emph>), we fitted two linear regression models for Confidence, both using all four predictors (Facility, Age, Gender, Advantaged), but the first with no interaction terms, and the second including all of the 2-way interactions. The results suggested that Advantaged may be partially mediated through the other predictors, and so we conducted a post hoc mediation analysis to investigate whether the effect of Advantage on Confidence operates via Facility, which was by far the strongest predictor. We also conducted a post hoc mediation analysis to see whether the effect of Age on Confidence also might be mediated by Facility.</p> <p>We also conducted some additional analyses to help understand the nature of students' calibration (the correlation between Facility and Confidence). We did this both (i) by student and (ii) by question.</p> <p>For (i), we calculated the Pearson correlation between each student's mean confidence on the questions on which they were correct and their mean confidence on the questions on which they were incorrect. We also used a Wilcoxon signed-rank test to determine whether there was a difference between students' confidence levels on questions on which they were correct and those on which they were incorrect, and a Kruskal-Wallis test to determine whether the number of correct attempts differed by confidence level. Non-parametric tests were used in both cases, because the conditions for parametric tests, such as normality, were not satisfied (see Baguley, [<reflink idref="bib1" id="ref113">1</reflink>]).</p> <p>For (ii), we carried out a similar Pearson correlation analysis, followed by a Mann-Whitney <emph>U</emph> test to determine whether the standard deviation of the confidence was greater for questions than for students. We also used Mann-Whitney <emph>U</emph> tests to determine whether there was a difference between boys and girls in the number of questions answered or in the number of questions answered correctly, and to determine whether disadvantaged students answered fewer questions than advantaged students, answered fewer questions correctly or had lower mean confidence. Again, non-parametric Mann-Whitney <emph>U</emph> tests were appropriate here, because the conditions for parametric tests, such as normality, were not satisfied (see Baguley, [<reflink idref="bib1" id="ref114">1</reflink>]).</p> <p>To address our second research question (<emph>Is there evidence for the hypercorrection effect in students' responses to a second set of questions (quiz B) administered 3 weeks after the first (quiz A)?</emph>), we needed to account for the fact that higher confidence on quiz A could be a proxy for higher "ability" and therefore would be likely to correlate with higher success on quiz B even without a hypercorrection effect. So, in order to partial out facility on quiz A, we carried out a logistic regression–logistic this time, since the outcome variable (facility on quiz B) was dichotomous. We included all 5 predictors in the model, as a conservative strategy to avoid making unjustified assumptions about the character of the best-fitting model.</p> <p>All data analysis was carried out using Python, with the <emph>statsmodels</emph> package (Seabold & Perktold, [<reflink idref="bib57" id="ref115">57</reflink>]), and all <emph>Python</emph> code is provided at https://github.com/Eedi/diagnostic-questions-and-student-confidence.</p> <hd id="AN0155342115-11">Results</hd> <p>We now report our findings under each of the two research questions set out above.</p> <hd id="AN0155342115-12">How are students' confidence ratings related to their mean facility, age, gender and socioeco...</hd> <p>The correlation matrix given in Table 2 shows that there was a positive association between the mean facility (the proportion of questions correct) for each student, and their mean confidence (<emph>r</emph><subs><emph>s</emph></subs> =.504, <emph>p</emph> <.001). In general, confidence was higher for boys (<emph>r</emph><subs><emph>s</emph></subs> =.134, with female coded 0 and male coded 1) and for more socioeconomically advantaged students (<emph>r</emph><subs><emph>s</emph></subs> =.094, with socioeconomically disadvantaged coded 0 and socioeconomically advantaged coded 1) but decreased with age (<emph>r</emph><subs><emph>s</emph></subs> = −.140). Facility was higher for advantaged students (<emph>r</emph><subs><emph>s</emph></subs> =.141) and decreased a little with age (<emph>r</emph><subs><emph>s</emph></subs> = −.053, all <emph>p</emph>s <.001). The small <emph>p</emph> values in Table 2 for all of the correlations just mentioned, even those correlations that are small in absolute size, mean that the correlations are statistically significantly different from zero.</p> <p>Table 2 Correlation matrix for the 5 variables</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th /><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p>Confidence</p></th><th><p>Facility</p></th><th><p>Age</p></th><th><p>Gender</p></th><th><p>Advantaged</p></th></tr></thead><tbody><tr><td><p>Confidence</p></td><td><p>68.32</p></td><td><p>22.04</p></td><td /><td><p>.504<sup>***</sup></p></td><td><p>−.140<sup>***</sup></p></td><td><p>.134<sup>***</sup></p></td><td><p>.094<sup>***</sup></p></td></tr><tr><td><p>Facility</p></td><td><p>57.81</p></td><td><p>22.07</p></td><td><p><.001</p></td><td /><td><p>−.053<sup>***</sup></p></td><td><p>−.013</p></td><td><p>.141<sup>***</sup></p></td></tr><tr><td><p>Age</p></td><td><p>11.41</p></td><td><p>2.01</p></td><td><p><.001</p></td><td><p><.001</p></td><td /><td><p>.032<sup>**</sup></p></td><td><p>.013</p></td></tr><tr><td><p>Gender</p></td><td /><td /><td><p><.001</p></td><td><p>.253</p></td><td><p>.006</p></td><td /><td><p>.002</p></td></tr><tr><td><p>Advantaged</p></td><td /><td /><td><p><.001</p></td><td><p><.001</p></td><td><p>.257</p></td><td><p>.854</p></td><td /></tr></tbody></table> </ephtml> </p> <p>The upper right triangle gives Spearman correlation coefficients and the lower left triangle the <emph>p</emph> values <sups>*</sups>.01 < <emph>p</emph> ≤.05; <sups>**</sups>.001 < <emph>p</emph> ≤.01; <sups>***</sups>.0001 < <emph>p</emph> ≤.001</p> <p>We fitted two linear regression models, with standardised[<reflink idref="bib5" id="ref116">5</reflink>] mean confidence as the dependent variable, and, among the predictors, we standardised Facility and Age, but not the two binary categorical predictors, Gender and Advantaged. The first model contained just the four predictors (Table 3); the second model included all of the 2-way interactions (Table 4).</p> <p>Table 3 Multiple regression with 4 predictors of confidence</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th rowspan="2"><p>Effect</p></th><th rowspan="2"><p>Estimate <italic>β</italic></p></th><th rowspan="2"><p>SE</p></th><th colspan="2"><p>95% CI for <italic>β</italic></p></th><th rowspan="2"><p><italic>t</italic></p></th><th rowspan="2"><p><italic>p</italic></p></th></tr><tr><th><p>LL</p></th><th><p>UL</p></th></tr></thead><tbody><tr><td><p>Intercept</p></td><td><p>−0.206</p></td><td><p>0.024</p></td><td><p>−0.253</p></td><td><p>−0.159</p></td><td><p>−8.588</p></td><td><p><.001</p></td></tr><tr><td><p>Facility</p></td><td><p>0.487</p></td><td><p>0.010</p></td><td><p>0.467</p></td><td><p>0.507</p></td><td><p>48.451</p></td><td><p><.001</p></td></tr><tr><td><p>Age</p></td><td><p>−0.117</p></td><td><p>0.010</p></td><td><p>−0.136</p></td><td><p>−0.097</p></td><td><p>−11.715</p></td><td><p><.001</p></td></tr><tr><td><p>Gender</p></td><td><p>0.279</p></td><td><p>0.020</p></td><td><p>0.240</p></td><td><p>0.318</p></td><td><p>14.026</p></td><td><p><.001</p></td></tr><tr><td><p>Advantaged</p></td><td><p>0.081</p></td><td><p>0.025</p></td><td><p>0.033</p></td><td><p>0.130</p></td><td><p>3.308</p></td><td><p>.001</p></td></tr></tbody></table> </ephtml> </p> <p>Adjusted <emph>R</emph>-squared = 0.279, <emph>F</emph>(<reflink idref="bib4" id="ref117">4</reflink>, 7297) = 706.4, <emph>p</emph> <.001</p> <p>Table 4 Multiple regression with 4 predictors and all 2-way interactions</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th rowspan="2"><p>Effect</p></th><th rowspan="2"><p>Estimate <italic>β</italic></p></th><th rowspan="2"><p>SE</p></th><th colspan="2"><p>95% CI for <italic>β</italic></p></th><th rowspan="2"><p><italic>t</italic></p></th><th rowspan="2"><p><italic>p</italic></p></th></tr><tr><th><p>LL</p></th><th><p>UL</p></th></tr></thead><tbody><tr><td><p>Intercept<sup>****</sup></p></td><td><p>−0.186</p></td><td><p>0.032</p></td><td><p>−0.248</p></td><td><p>−0.124</p></td><td><p>−5.882</p></td><td><p><.001</p></td></tr><tr><td><p>Facility<sup>****</sup></p></td><td><p>0.522</p></td><td><p>0.024</p></td><td><p>0.475</p></td><td><p>0.569</p></td><td><p>21.893</p></td><td><p><.001</p></td></tr><tr><td><p>Age<sup>****</sup></p></td><td><p>−0.121</p></td><td><p>0.026</p></td><td><p>−0.171</p></td><td><p>−0.071</p></td><td><p>−4.731</p></td><td><p><.001</p></td></tr><tr><td><p>Gender<sup>****</sup></p></td><td><p>0.243</p></td><td><p>0.044</p></td><td><p>0.157</p></td><td><p>0.329</p></td><td><p>5.560</p></td><td><p><.001</p></td></tr><tr><td><p>Advantaged</p></td><td><p>0.056</p></td><td><p>0.036</p></td><td><p>−0.014</p></td><td><p>0.126</p></td><td><p>1.574</p></td><td><p>.115</p></td></tr><tr><td><p>Facility × Age</p></td><td><p>0.014</p></td><td><p>0.010</p></td><td><p>−0.005</p></td><td><p>0.034</p></td><td><p>1.449</p></td><td><p>.147</p></td></tr><tr><td><p>Facility × Gender<sup>*</sup></p></td><td><p>−0.052</p></td><td><p>0.020</p></td><td><p>−0.091</p></td><td><p>−0.012</p></td><td><p>−2.555</p></td><td><p>.011</p></td></tr><tr><td><p>Facility × Advantaged</p></td><td><p>−0.011</p></td><td><p>0.024</p></td><td><p>−0.059</p></td><td><p>0.037</p></td><td><p>−0.454</p></td><td><p>.650</p></td></tr><tr><td><p>Age × Gender</p></td><td><p>0.031</p></td><td><p>0.020</p></td><td><p>−0.008</p></td><td><p>0.070</p></td><td><p>1.536</p></td><td><p>.124</p></td></tr><tr><td><p>Age × Advantaged</p></td><td><p>−0.018</p></td><td><p>0.026</p></td><td><p>−0.069</p></td><td><p>0.034</p></td><td><p>−0.672</p></td><td><p>.502</p></td></tr><tr><td><p>Advantaged × Gender</p></td><td><p>0.047</p></td><td><p>0.049</p></td><td><p>−0.050</p></td><td><p>0.143</p></td><td><p>0.949</p></td><td><p>.342</p></td></tr></tbody></table> </ephtml> </p> <p>Adjusted <emph>R</emph>-squared = 0.279, F(<reflink idref="bib10" id="ref118">10</reflink>, 7291) = 284, <emph>p</emph> <.001 <sups>*</sups>.01 < <emph>p</emph> ≤.05; <sups>**</sups>.001 < <emph>p</emph> ≤.01; <sups>***</sups>.0001 < <emph>p</emph> ≤.001; <sups>****</sups><emph>p</emph> ≤.0001</p> <p>The regression coefficients for Facility and Age in this multiple regression model (Table 3) are consistent with the correlation matrix (Table 2). For Gender and Advantaged, it is not possible to directly compare the <emph>β</emph>s in Table 3 with the <emph>r</emph><subs><emph>s</emph></subs> values in Table 2, since those variables were not standardised. However, by running models with single predictors of Confidence, we calculated that the regression coefficients for Gender and Advantaged (in separate single-predictor models) were 0.257 [95% CI = 0.212 to 0.303] and 0.245 [95% CI = 0.189 to 0.301]. Comparing these with the values in Table 3, we can see that the regression coefficient for Gender is very similar, but the coefficient for Advantaged has decreased considerably (0.245 to 0.081), suggesting that Advantaged may be partially mediated through the other predictors. Below, we conduct a post hoc mediation analysis to investigate this.</p> <p>It is clear that Facility is the dominant predictor, so it is important to consider the possible interaction of other predictors with Facility. Table 4 shows results from the multiple regression model which includes all 2-way interactions. Including the interaction terms does not appreciably affect the regression coefficients for Facility, Age and Gender, but, again, the coefficient for Advantaged has now dropped a little further, from 0.081 to 0.056, also now becoming nonsignificant. This suggests that the effects of Advantaged are now fully mediated through some or all of the other predictors. The only significant 2-way interaction is between Facility and Gender, and the regression coefficient for this is small (−0.052).</p> <p>We now present more detailed analysis relating to each predictor.</p> <hd id="AN0155342115-13">Facility</hd> <p>Facility was by far the strongest predictor of Confidence (<emph>β</emph> = 0.522, <emph>p</emph> <.001), and the positive association between mean facility and mean confidence for each student was <emph>r</emph><subs><emph>s</emph></subs> =.504 (<emph>p</emph> <.001). This is close to Foster's ([<reflink idref="bib21" id="ref119">21</reflink>]) previously reported correlation of <emph>r</emph> =.546 between facility and mean confidence for 11–14-year-old students in the topic of directed numbers (<emph>N</emph> = 345), meaning students' level of calibration (see Fischhoff et al., [<reflink idref="bib20" id="ref120">20</reflink>], p. 552) in the current study is comparable to this. However, it is clear from Fig. 3 and Table 5 that there are students at every combination of facility and confidence.</p> <p>Graph: Fig. 3 Mean confidence against mean facility. The dashed line indicates equal Facility and Confidence (perfect calibration), and the size of the dots indicates the number of answers the student gave. This plot is for all students in the dataset (N = 7302)</p> <p>Table 5 The percentage of correct answers by confidence</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th rowspan="2"><p>Parameter</p></th><th colspan="2"><p>Confidence 0</p><p><italic>n</italic> = 20,694</p></th><th colspan="2"><p>Confidence 25</p><p><italic>n</italic> = 11,915</p></th><th colspan="2"><p>Confidence 50</p><p><italic>n</italic> = 36,021</p></th><th colspan="2"><p>Confidence 75</p><p><italic>n</italic> = 51,722</p></th><th colspan="2"><p>Confidence 100</p><p><italic>n</italic> = 99,474</p></th></tr><tr><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th></tr></thead><tbody><tr><td><p>Correct (%)</p></td><td><p>32.7</p></td><td><p>46.9</p></td><td><p>35.9</p></td><td><p>48.0</p></td><td><p>45.3</p></td><td><p>49.8</p></td><td><p>64.3</p></td><td><p>47.9</p></td><td><p>73.4</p></td><td><p>44.2</p></td></tr></tbody></table> </ephtml> </p> <p>To explore students' calibration in more detail, we calculated each student's mean confidence on the questions on which they were correct and their mean confidence on the questions on which they were incorrect (see scatterplot in Fig. 4a). To guard against extreme responses, in this analysis we only included students who had provided at least 50 answers, at least 5 of which were correct, and at least 5 of which were incorrect, giving a dataset of 115,437 answers from 1033 students. The fact that most of the points are above the diagonal line in Fig. 4a shows that students tended to show greater confidence on correct questions than on incorrect questions, but the strong positive correlation (<emph>r</emph><subs><emph>s</emph></subs>(1031) =.889, <emph>p</emph> <.001) indicates that students who gave higher confidence scores tended to do so both for questions on which they were correct <emph>and</emph> for those on which they were incorrect. The histogram of differences in mean confidence score (Fig. 4b) is positively skewed, with a bulge near zero, indicating a large number of students who gave the same confidence level, whether or not their answer was correct. A Wilcoxon signed-rank test indicated that, on average, students were more confident with questions on which they were correct (<emph>Mdn</emph> = 82.8) than with those on which they were incorrect (<emph>Mdn</emph> = 68.8, <emph>Z</emph> = 11,255, <emph>r</emph><subs><emph>s</emph></subs> =.956, <emph>p</emph> <.001). The percentage of correct answers increased markedly with confidence (Table 5). A Kruskal-Wallis test found a difference in the number of correct attempts by answers with different confidence (<emph>H</emph>(<reflink idref="bib4" id="ref121">4</reflink>) = 20,487.1, <emph>p</emph> <.001).</p> <p>Graph: Fig. 4 Students' mean confidence on questions that they answered correctly versus their mean confidence on questions that they answered incorrectly. In a scatterplot (a), each disc represents one student, and the size of the disc represents the number of questions the student answered. The differences are plotted as a histogram in (b). These plots are for students who each answered more than 50 questions, at least 5 of which were correct and 5 were incorrect (N = 1033)</p> <p>A similar analysis but <emph>by question</emph> (Fig. 5) showed a positive but weaker correlation (<emph>r</emph><subs><emph>s</emph></subs>(<reflink idref="bib1" id="ref122">1</reflink>,<reflink idref="bib137" id="ref123">137</reflink>) =.477, <emph>p</emph> <.001). Similar to before, for this analysis, we only included questions with at least 50 answers, at least 5 of which were correct and at least 5 of which were incorrect, giving 110,283 responses across 1139 questions. For each student and each question, we calculated the standard deviation of the confidence. A Mann-Whitney <emph>U</emph> test indicated that the standard deviation of the confidence was greater for questions (<emph>Mdn</emph> = 30.8) than for students (<emph>Mdn</emph> = 22.2, <emph>U</emph> = 284,839.5, <emph>p</emph> <.001). This indicated that the confidence rating given was more strongly associated with the student than with the question.</p> <p>Graph: Fig. 5 Scatterplot of each question's mean confidence from students who were correct against the mean confidence from students who were incorrect. In a scatterplot (a), each disc represents one question, and the size of the disc represents the number of students who answered the question. The differences are plotted as a histogram in (b). Note that these plots are for questions with more than 50 answers, at least 5 of which were correct and 5 were incorrect (N = 1139). The number of answers to the questions ranged from 51 to 320 (M = 96.8, SD = 55.3)</p> <hd id="AN0155342115-14">Age</hd> <p>The correlation matrix given in Table 2 shows that mean confidence decreased with student age (<emph>r</emph><subs><emph>s</emph></subs> = −.140, <emph>p</emph> <.001), and facility also decreased a little with age (<emph>r</emph><subs><emph>s</emph></subs> = −.053, <emph>p</emph> <.001). To explore this further, we restricted the dataset to responses during one academic year: between September 2019 and May 2020 (<emph>N</emph> = 5382). We did this so that each student could be assigned to a single school year. We grouped the students into bands according to their school year: key stage 2 (ages 7–11), key stage 3 (ages 11–14) and key stage 4 (ages 14–16). Kruskal-Wallis tests comparing the parameter distributions between the groups found statistically significant differences among the key stages for the number of questions answered, the mean facility and also the mean confidence (see Table 6). For almost all facility levels, there is a clear decrease in confidence as age increases, and the drop in confidence from key stage 2 to key stage 3 appears to be generally larger than the drop from key stage 3 to key stage 4 (see Fig. 6).</p> <p>Table 6 Mean confidence and facility by key stage</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th rowspan="2"><p>Parameter</p></th><th colspan="2"><p>Key stage 2 (<italic>n</italic> = 529)</p></th><th colspan="2"><p>Key stage 3 (<italic>n</italic> = 3339)</p></th><th colspan="2"><p>Key stage 4 (<italic>n</italic> = 1514)</p></th><th rowspan="2"><p><italic>H</italic>(2)</p></th><th rowspan="2"><p><italic>p</italic></p></th></tr><tr><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th></tr></thead><tbody><tr><td><p>Number of questions answered</p></td><td><p>38.6</p></td><td><p>57.8</p></td><td><p>32.8</p></td><td><p>42.5</p></td><td><p>25.8</p></td><td><p>40.0</p></td><td><p>129.2</p></td><td><p><.001</p></td></tr><tr><td><p>Mean facility</p></td><td><p>66.3</p></td><td><p>21.9</p></td><td><p>56.3</p></td><td><p>21.2</p></td><td><p>57.7</p></td><td><p>23.3</p></td><td><p>104.0</p></td><td><p><.001</p></td></tr><tr><td><p>Mean confidence</p></td><td><p>78.8</p></td><td><p>20.0</p></td><td><p>67.8</p></td><td><p>21.7</p></td><td><p>65.6</p></td><td><p>22.6</p></td><td><p>172.2</p></td><td><p><.001</p></td></tr></tbody></table> </ephtml> </p> <p>Graph: Fig. 6 Confidence against facility by key stage, with facility split into decades. The grouped histogram shows the distribution of students by key stage in the different facility decades. Above each group, we have added the results of a Mann-Whitney U test, two-sided with Bonferroni correction, for the key stage 2 and 4 groups. ns:.05 < p; *.01 < p ≤.05; **.001 < p ≤.01; ***.0001 < p ≤.001</p> <p>To investigate the possibility that the decrease in confidence with increasing age might be mediated by the difficulty of the mathematics, we conducted a post hoc mediation analysis using the statsmodels <emph>mediation</emph> package in <emph>Python</emph> (Seabold & Perktold, [<reflink idref="bib57" id="ref124">57</reflink>]) to compute 95% confidence intervals (95% CI) over 1000 simulations to test for significant indirect effects (Fig. 7). Age displayed a significant direct effect on Confidence (<emph>β</emph> = −0.117, 95% CI = −0.136 to −0.097, <emph>p</emph> <.001) and a significant indirect effect on Confidence, with Facility as mediator (<emph>β</emph> = −0.027, 95% CI = −0.046 to −0.008, <emph>p</emph> =.008). Age was associated with Confidence, but only 18.7% (95% CI = 6.2% to 29.4%, <emph>p</emph> =.008) of this relationship was mediated by decreased Facility.</p> <p>Graph: Fig. 7 Model to investigate mediation of the effect of age by facility. Note that for clarity in this figure, we did not include the predictors Advantage and Gender, but these were included in all of the statistical analyses. ***p <.001. This analysis was for the restricted dataset (September 2019–May 2020, N = 5382)</p> <hd id="AN0155342115-15">Gender</hd> <p>Table 7 indicates that mean confidence was higher for boys than for girls (<emph>r</emph><subs><emph>s</emph></subs> =.134, <emph>p</emph> <.001) and, when analysed by decade of facility (see Fig. 8), the same pattern is striking across all levels of facility. Mann-Whitney <emph>U</emph> tests found no significant difference between boys and girls on the number of questions answered or on the number of questions answered correctly (Table 7), suggesting that boys' higher confidence constitutes <emph>over</emph>confidence. We found a significant interaction between Facility and Gender (<emph>β</emph> = −0.052, <emph>p</emph> =.011), meaning that confidence increases more slowly with Facility for boys than it does for girls. This means that the overconfidence of boys is more marked with lower-attaining students.</p> <p>Graph: Fig. 8 Confidence against facility by disadvantage, with facility split into decades. The paired histogram shows the distribution of students by disadvantage in the different facility decades. Above each group, we have added the results of a Mann-Whitney U test two-sided with Bonferroni correction. ns:.05 < p, *.01 < p ≤. 05, **.001 < p ≤.01, ***.0001 < p ≤.001</p> <p>Table 7 Mean confidence and facility by gender</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th rowspan="2"><p>Parameter</p></th><th colspan="2"><p>Male (<italic>n</italic> = 3716)</p></th><th colspan="2"><p>Female (<italic>n</italic> = 3586)</p></th><th rowspan="2"><p><italic>U</italic></p></th><th rowspan="2"><p><italic>p</italic></p></th></tr><tr><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th></tr></thead><tbody><tr><td><p>Number of questions answered</p></td><td><p>30.1</p></td><td><p>43.2</p></td><td><p>30.2</p></td><td><p>42.1</p></td><td><p>6,568,697.0</p></td><td><p>.296</p></td></tr><tr><td><p>Mean facility</p></td><td><p>57.5</p></td><td><p>22.3</p></td><td><p>58.2</p></td><td><p>21.8</p></td><td><p>6,559,851.0</p></td><td><p>. 253</p></td></tr><tr><td><p>Mean confidence</p></td><td><p>71.1</p></td><td><p>21.7</p></td><td><p>65.4</p></td><td><p>22.0</p></td><td><p>7,696,906.0</p></td><td><p><.001</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0155342115-16">Advantage</hd> <p>As noted before, both Confidence and Facility were higher for more socioeconomically advantaged students (<emph>r</emph><subs><emph>s</emph></subs> =.094 and <emph>r</emph><subs><emph>s</emph></subs> =.141, respectively, both <emph>p</emph>s <.001). When we included all 2-way interactions in our multiple regression model, the coefficient for Advantaged became nonsignificant, suggesting that the effects of Advantaged were fully mediated through some or all of the other predictors. To investigate this, we again conducted a post hoc mediation analysis using the statsmodels <emph>mediation</emph> package in <emph>Python</emph> (Seabold & Perktold, [<reflink idref="bib57" id="ref125">57</reflink>]), this time on the full dataset (<emph>N</emph> = 7302), to compute 95% confidence intervals (95% CI) over 1000 simulations (Fig. 9). Advantage displayed a significant direct effect on Confidence (<emph>β</emph> = 0.082, 95% CI = 0.030 to 0.128, <emph>p</emph> <.001) and a significant indirect effect on Confidence with Facility as mediator (<emph>β</emph> = 0.166, 95% CI = 0.136 to 0.197, <emph>p</emph> <.001). Advantage was associated with Confidence, and 67.0% (95% CI = 55.0% to 84.4%, <emph>p</emph> <.001) of this relationship was mediated by increased Facility.</p> <p>Graph: Fig. 9 Model to investigate mediation of the effect of Advantage by Facility. Note that for clarity in this figure, we did not include the predictors Age and Gender, but these were included in all of the statistical analyses. ***p <.001. This analysis was for the full dataset (N = 7302)</p> <p>Figure 8 presents the distribution of students (advantaged and disadvantaged) by decade of facility, showing a peak for the disadvantaged students in the 40–50 facility interval, whereas for the advantaged students, the peak is in the 60–70 facility interval. Two-sided Mann-Whitney <emph>U</emph> tests found that disadvantaged students answered fewer questions than advantaged students (<emph>r</emph><subs><emph>s</emph></subs> = −.086, <emph>p</emph> <.001), answered fewer questions correctly (<emph>r</emph><subs><emph>s</emph></subs> = −.199, <emph>p</emph> <.001), and the mean confidence for disadvantaged students was significantly lower than for advantaged students (<emph>r</emph><subs><emph>s</emph></subs> = −.133, <emph>p</emph> <.001) (Table 8). For most (but not all) decades of facility, confidence was lower for disadvantaged students (see Fig. 8).</p> <p>Table 8 Mean confidence and facility by disadvantage</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th rowspan="2"><p>Parameter</p></th><th colspan="2"><p>Advantaged (<italic>n</italic> = 5761)</p></th><th colspan="2"><p>Disadvantaged (<italic>n</italic> = 1541)</p></th><th rowspan="2"><p><italic>U</italic></p></th><th rowspan="2"><p><italic>p</italic></p></th></tr><tr><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th></tr></thead><tbody><tr><td><p>Number of questions answered</p></td><td><p>31.4</p></td><td><p>44.7</p></td><td><p>25.1</p></td><td><p>33.3</p></td><td><p>4,055,896.5</p></td><td><p><.001</p></td></tr><tr><td><p>Mean facility</p></td><td><p>59.4</p></td><td><p>21.8</p></td><td><p>51.9</p></td><td><p>22.1</p></td><td><p>3,553,918.5</p></td><td><p><.001</p></td></tr><tr><td><p>Mean confidence</p></td><td><p>69.5</p></td><td><p>21.5</p></td><td><p>64.0</p></td><td><p>23.6</p></td><td><p>3,849,712.5</p></td><td><p><.001</p></td></tr></tbody></table> </ephtml> </p> <p>Table 9 Students attempting an analogous question, following an incorrect answer, and the proportion of these which were correct</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th rowspan="2"><p>Parameter</p></th><th colspan="2"><p>Confidence</p><p>0</p><p>(<italic>n</italic> = 13,921)</p></th><th colspan="2"><p>Confidence 25</p><p>(<italic>n</italic> = 7642)</p></th><th colspan="2"><p>Confidence 50</p><p>(<italic>n</italic> = 19,695)</p></th><th colspan="2"><p>Confidence 75</p><p>(<italic>n</italic> = 18,440)</p></th><th colspan="2"><p>Confidence 100</p><p>(<italic>n</italic> = 26,446)</p></th><th rowspan="2"><p><italic>H</italic>(4)</p></th></tr><tr><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th></tr></thead><tbody><tr><td align="left"><p>Attempting analogous question (%)</p></td><td><p>48.9</p></td><td><p>50.0</p></td><td><p>49.3</p></td><td><p>50.0</p></td><td><p>50.1</p></td><td><p>50.0</p></td><td><p>52.7</p></td><td><p>49.9</p></td><td><p>54.3</p></td><td><p>49.8</p></td><td><p>160.6<sup>***</sup></p></td></tr><tr><td align="left"><p>Analogous question correct (%)</p></td><td><p>39.0</p></td><td><p>48.8</p></td><td><p>39.1</p></td><td><p>48.8</p></td><td><p>40.2</p></td><td><p>49.0</p></td><td><p>46.6</p></td><td><p>49.9</p></td><td><p>50.5</p></td><td><p>50.0</p></td><td><p>430.1<sup>***</sup></p></td></tr></tbody></table> </ephtml> </p> <p> <sups>***</sups> <emph>p</emph> <.001</p> <hd id="AN0155342115-17">Is there evidence for the hypercorrection effect in students' responses to a second set of qu...</hd> <p>The hypercorrection effect (Butterfield & Metcalfe, [<reflink idref="bib8" id="ref126">8</reflink>]) predicts that a student who answers a question incorrectly but with high confidence will be more likely to be successful with the same or a similar question subsequently. To test this in our data, we analysed the 86,144 answers from a total of <emph>N</emph> = 7002 students within quiz A sessions where the answer was incorrect, a confidence rating was given and an analogous question was subsequently assigned 3 weeks later in quiz B (see Section 3.1). Table 9 and Fig. 10 show that the percentage of students making second attempts increased with the confidence level expressed on the first attempt.[<reflink idref="bib6" id="ref127">6</reflink>] We see a clear increase in facility with confidence, which might seem to demonstrate the hypercorrection effect: the students who were more confident about their original incorrect answer were more likely to answer correctly 3 weeks later in quiz B. However, it is important to distinguish hypercorrection from regression to the mean (Baguley, [<reflink idref="bib1" id="ref128">1</reflink>]), where facility is a confounder. Students expressing high confidence in quiz A, despite being incorrect on that occasion, are likely on average to be higher-facility students, since facility and confidence are correlated. Consequently, even without a hypercorrection effect, they would be expected to be more likely to succeed on quiz B anyway. This means that, in order to tease out any hypercorrection effect of confidence over and above an "ability effect", we need to carry out a logistic regression.</p> <p>Graph: Fig. 10 Students attempting a similar question 3 weeks later (quiz B), following an incorrect answer, and the proportion of these which were correct. Dashed lines indicate the percentages for students who provided no measure of confidence. These plots are for pairs for which the first answer was incorrect; this reduced the dataset to 7002 students and 86,144 answer pairs</p> <p>The data consisted of 44,524 incorrect answers by 3838 students who had attempted analogous questions 3 weeks later in quiz B (see Table 10). Of these students, 19,885 (44.7%) answered the quiz B question correctly and 24,639 (55.3%) incorrectly. The hypercorrection hypothesis is that the probability that the quiz B question is answered correctly is higher when the student's confidence in their original mistake was higher, after controlling for facility.</p> <p>Table 10 Distribution of data for logistic regression</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th rowspan="2"><p>Correct in quiz B?</p></th><th><p><italic>N</italic></p></th><th colspan="2"><p>Facility</p></th><th colspan="2"><p>Confidence</p></th><th colspan="2"><p>Age</p></th><th colspan="2"><p>Socioeconomic advantage</p></th><th colspan="2"><p>Gender</p></th></tr><tr><th /><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p><italic>M</italic></p></th><th><p><italic>SD</italic></p></th><th><p>Adv</p></th><th><p>Dis</p></th><th><p>F</p></th><th><p>M</p></th></tr></thead><tbody><tr><td><p>Yes</p></td><td><p>19,885</p></td><td><p>57.6</p></td><td><p>17.2</p></td><td><p>65.4</p></td><td><p>34.6</p></td><td><p>11.2</p></td><td><p>1.9</p></td><td><p>16,233</p></td><td><p>3652</p></td><td><p>10,288</p></td><td><p>9597</p></td></tr><tr><td><p>No</p></td><td><p>24,639</p></td><td><p>50.4</p></td><td><p>16.6</p></td><td><p>58.9</p></td><td><p>35.4</p></td><td><p>11.4</p></td><td><p>1.8</p></td><td><p>19,348</p></td><td><p>5291</p></td><td><p>12,552</p></td><td><p>12,087</p></td></tr><tr><td><p>Summary</p></td><td><p>44,524</p></td><td><p>53.6</p></td><td><p>17.2</p></td><td><p>61.8</p></td><td><p>35.2</p></td><td><p>11.3</p></td><td><p>1.8</p></td><td><p>35,581</p></td><td><p>8943</p></td><td><p>22,840</p></td><td><p>21,684</p></td></tr></tbody></table> </ephtml> </p> <p>In order to make as few modelling assumptions as possible, we fitted a five-predictor logistic model, using the Logit method in the <emph>statsmodels</emph> package, version 0.10.1 (Seabold & Perktold, [<reflink idref="bib57" id="ref129">57</reflink>]), so as to allow effects of any of these predictors to be accounted for. The model may be expressed as:</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mtext>logit</mtext><mfenced close=")" open="("><mi>Y</mi></mfenced><mo>=</mo><msub><mi>β</mi><mn>0</mn></msub><mo>+</mo><msub><mi>β</mi><mn>1</mn></msub><msub><mi>X</mi><mn>1</mn></msub><mo>+</mo><msub><mi>β</mi><mn>2</mn></msub><msub><mi>X</mi><mn>2</mn></msub><mo>+</mo><msub><mi>β</mi><mn>3</mn></msub><msub><mi>X</mi><mn>3</mn></msub><mo>+</mo><msub><mi>β</mi><mn>4</mn></msub><msub><mi>X</mi><mn>4</mn></msub><mo>+</mo><msub><mi>β</mi><mn>5</mn></msub><msub><mi>X</mi><mn>5</mn></msub></math> </ephtml> </p> <p>Graph</p> <p>where the outcome variable <emph>Y</emph> is whether the quiz B question was answered correctly (0 = incorrect, 1 = correct), <emph>X</emph><subs>1</subs> is the mean student facility on quiz A (0–100), <emph>X</emph><subs>2</subs> is the student's confidence in their original incorrect response on quiz A (0–100), <emph>X</emph><subs>3</subs> is the student's age (6–16), <emph>X</emph><subs>4</subs> indicates whether the student was advantaged (0 = disadvantaged, 1 = advantaged) and <emph>X</emph><subs>5</subs> is the student's gender (0 = female, 1 = male).</p> <p>The result was:</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mtable displaystyle="true"><mtr><mtd><mtext>Predicted logit of</mtext><mspace width="0.25em" /><mfenced close=")" open="("><mrow><mtext>Correct Quiz</mtext><mspace width="0.25em" /><mi mathvariant="normal">B</mi><mspace width="0.25em" /><mtext>Answer</mtext></mrow></mfenced><mo>=</mo><mo>−</mo><mn>1.4578</mn></mtd></mtr><mtr><mtd><mo>+</mo><mn>0.0235</mn><mo>×</mo><mtext>Student Facility</mtext></mtd></mtr><mtr><mtd><mtable><mtr><mtd><mo>+</mo><mn>0.0028</mn><mo>×</mo><mtext>Confidence in Quiz</mtext><mspace width="0.25em" /><mi mathvariant="normal">A</mi><mspace width="0.25em" /><mtext>Mistake</mtext></mtd></mtr><mtr><mtd><mo>−</mo><mn>0.0188</mn><mo>×</mo><mi>Age</mi></mtd></mtr><mtr><mtd><mtable><mtr><mtd><mo>+</mo><mn>0.0402</mn><mo>×</mo><mtext>Is Advantaged</mtext></mtd></mtr><mtr><mtd><mo>−</mo><mn>0.0461</mn><mo>×</mo><mtext>Is Male</mtext></mtd></mtr></mtable></mtd></mtr></mtable></mtd></mtr></mtable></math> </ephtml> </p> <p>Graph</p> <p>According to the model (see Table 11), the log odds of a student answering correctly was, as expected, positively related to their overall facility (<emph>p</emph> <.001). However, and confirming our hypercorrection hypothesis, it was also positively related to their confidence in their quiz A mistake (<emph>p</emph> <.001). The log odds of a student answering correctly was <emph>negatively</emph> related to their age (<emph>p</emph> <.001) and to gender (<emph>p</emph> =.019). Whether they were advantaged was not statistically significant (<emph>p</emph> =.105).</p> <p>Table 11 Logistic regression analysis of 44,524 students' answers to questions 3 weeks after answering a similar question incorrectly</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th rowspan="2"><p><bold>Effect</bold></p></th><th rowspan="2"><p><bold>Estimate</bold><bold><italic>β</italic></bold></p></th><th rowspan="2"><p><bold>SE</bold></p></th><th colspan="2"><p><bold>95% CI for</bold><bold><italic>β</italic></bold></p></th><th rowspan="2"><p><bold>Wald's</bold><bold><italic>χ</italic></bold><sup><bold>2</bold></sup></p></th><th><p><bold><italic>df</italic></bold></p></th><th rowspan="2"><p><bold><italic>p</italic></bold></p></th><th rowspan="2"><p><bold><italic>e</italic></bold><sup><bold><italic>β</italic></bold></sup><bold>(odds ratio)</bold></p></th><th colspan="2"><p><bold>95% CI for</bold><bold><italic>e</italic></bold><sup><bold><italic>β</italic></bold></sup></p></th></tr><tr><th><p><bold>LL</bold></p></th><th><p><bold>UL</bold></p></th><th /><th><p><bold>LL</bold></p></th><th><p><bold>UL</bold></p></th></tr></thead><tbody><tr><td><p> Intercept</p></td><td><p>−1.4578</p></td><td><p>0.075</p></td><td><p>−1.606</p></td><td><p>−1.310</p></td><td><p>372.9</p></td><td><p>1</p></td><td><p><.001</p></td><td><p>0.233</p></td><td><p>0.201</p></td><td><p>0.270</p></td></tr><tr><td><p> Facility<sup>a</sup></p></td><td><p>0.0235</p></td><td><p>0.001</p></td><td><p>0.022</p></td><td><p>0.025</p></td><td><p>1491.1</p></td><td><p>1</p></td><td><p><.001</p></td><td><p>1.024</p></td><td><p>1.023</p></td><td><p>1.025</p></td></tr><tr><td><p> Confidence<sup>b</sup></p></td><td><p>0.0028</p></td><td><p>0.000</p></td><td><p>0.002</p></td><td><p>0.003</p></td><td><p>97.7</p></td><td><p>1</p></td><td><p><.001</p></td><td><p>1.003</p></td><td><p>1.002</p></td><td><p>1.003</p></td></tr><tr><td><p> Age<sup>c</sup></p></td><td><p>−0.0188</p></td><td><p>0.005</p></td><td><p>−0.029</p></td><td><p>−0.008</p></td><td><p>12.3</p></td><td><p>1</p></td><td><p><.001</p></td><td><p>0.981</p></td><td><p>0.971</p></td><td><p>0.992</p></td></tr><tr><td><p> Advantaged<sup>d</sup></p></td><td><p>0.0402</p></td><td><p>0.025</p></td><td><p>−0.008</p></td><td><p>0.089</p></td><td><p>2.6</p></td><td><p>1</p></td><td><p>.105</p></td><td><p>1.041</p></td><td><p>0.992</p></td><td><p>1.093</p></td></tr><tr><td><p> Gender<sup>e</sup></p></td><td><p>−0.0461</p></td><td><p>0.020</p></td><td><p>−0.085</p></td><td><p>–0.008</p></td><td><p>5.5</p></td><td><p>1</p></td><td><p>.019</p></td><td><p>0.955</p></td><td><p>0.919</p></td><td><p>0.992</p></td></tr><tr><td><p><bold>Test</bold></p></td><td /><td /><td /><td /><td><p><bold><italic>χ</italic></bold><sup><bold>2</bold></sup></p></td><td><p><bold><italic>df</italic></bold></p></td><td><p><bold><italic>p</italic></bold></p></td><td /><td /><td /></tr><tr><td colspan="3"><p> Overall model evaluation</p></td><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td colspan="3"><p> Likelihood ratio test</p></td><td /><td /><td><p>2059.2</p></td><td><p>5</p></td><td><p><.001</p></td><td /><td /><td /></tr><tr><td colspan="3"><p> Wald test</p></td><td /><td /><td><p>1936.4</p></td><td><p>5</p></td><td><p><.001</p></td><td /><td /><td /></tr><tr><td colspan="3"><p> Goodness-of-fit test</p></td><td /><td /><td /><td /><td /><td /><td /><td /></tr><tr><td colspan="3"><p> Hosmer & Lemeshow</p></td><td /><td /><td><p>39,330.0</p></td><td><p>8</p></td><td><p><.001</p></td><td /><td /><td /></tr></tbody></table> </ephtml> </p> <p>Number of answers = 44,524, number of students = 3838 McFadden's pseudo-<emph>R</emph>-squared = 0.0336 <emph>CI</emph> confidence interval, <emph>LL</emph> lower limit, <emph>UL</emph> upper limit <sups>a</sups>Ranging from 0 to 100 <sups>b</sups>Ranging from 0 to 100 <sups>c</sups>Ranging from 6 to 16 <sups>d</sups>0 = disadvantaged, 1 = advantaged <sups>e</sups>0 = female, 1 = male</p> <p>In other words, the higher the student's confidence in their quiz A mistake, the more likely it was that the student answered the quiz B question correctly, even after accounting for overall facility. The odds ratio when increasing from one emoji (e.g., 25) to the next (e.g., 50) was <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msup><mi>e</mi><mrow><mn>25</mn><msub><mi>β</mi><mn>2</mn></msub></mrow></msup><mo>=</mo><mn>1.07</mn></math> </ephtml> . In this dataset, the hypercorrection effect appeared to be stronger for younger students than for older students, and stronger for girls than for boys.</p> <p>Finally, we note that, as Fig. 11 suggests, confidence in quiz A is positively correlated with facility in quiz A, <emph>r</emph><subs><emph>s</emph></subs>(<reflink idref="bib1" id="ref130">1</reflink>,<reflink idref="bib654" id="ref131">654</reflink>) =.509, <emph>p</emph> <.001, and the relationship for quiz B is almost identical, <emph>r</emph><subs><emph>s</emph></subs>(<reflink idref="bib1" id="ref132">1</reflink>,<reflink idref="bib654" id="ref133">654</reflink>) =.502, <emph>p</emph> <.001, indicating that students were similarly well calibrated in both quizzes. Their calibration did not measurably change across the intervening 3-week period.</p> <p>Graph: Fig. 11 Student confidence versus facility in quiz A and quiz B. Each point represents one student. The size of the point indicates the number of answers for which the student answered the questions in quizzes A and B with confidence. These plots are for responses where confidence was given in both quiz A and quiz B. Then, we grouped by student and only retained students with 10 or more answers; the result was 1656 students</p> <hd id="AN0155342115-18">Discussion</hd> <p>Analysis of data from the <emph>Eedi</emph> platform succeeded in confirming several findings from previous studies, and did so using a large dataset in an authentic learning context, in which students were answering questions set by their teachers as a normal part of their school learning, not specifically for a research study. In addition to this, we were able to uncover greater detail concerning relationships between students' confidence and a variety of demographic variables, including gender, age and socioeconomic disadvantage. Finally, we found the first evidence for the hypercorrection effect in an authentic mathematics learning context.</p> <hd id="AN0155342115-19">The relationship between confidence and other variables</hd> <p>Previous research has shown that confidence is lower for girls, for older students and for students characterised as socioeconomically disadvantaged (e.g., Ganley & Lubienski, [<reflink idref="bib27" id="ref134">27</reflink>]; Mullis et al., [<reflink idref="bib46" id="ref135">46</reflink>]; OECD, [<reflink idref="bib47" id="ref136">47</reflink>], [<reflink idref="bib48" id="ref137">48</reflink>]). However, exactly how these variables are interrelated was not known; in particular, whether the relative under-confidence of girls varies depending on age or socioeconomic disadvantage, or whether the decrease in student confidence with age can be fully accounted for by decreasing attainment, or whether the lower confidence of socioeconomically disadvantaged students might be completely accounted for by lower attainment.</p> <p>In line with previous research (e.g., Foster, [<reflink idref="bib21" id="ref138">21</reflink>]; Mullis et al., [<reflink idref="bib46" id="ref139">46</reflink>]), the students in this dataset were generally well calibrated (Fischhoff et al., [<reflink idref="bib20" id="ref140">20</reflink>]), showing a positive correlation between the mean facility (the proportion of questions correct) for each student and the mean confidence (<emph>r</emph><subs><emph>s</emph></subs> =.504, <emph>p</emph> <.001). This means that in general, students showed good awareness of which questions they could answer correctly and which they could not. We also confirmed previous research showing a higher mean confidence for boys than for girls (e.g., Foster, [<reflink idref="bib21" id="ref141">21</reflink>]; Frost et al., [<reflink idref="bib24" id="ref142">24</reflink>]; Ganley & Lubienski, [<reflink idref="bib27" id="ref143">27</reflink>]; Mullis et al., [<reflink idref="bib46" id="ref144">46</reflink>]; OECD, [<reflink idref="bib47" id="ref145">47</reflink>], [<reflink idref="bib48" id="ref146">48</reflink>]). Figure 12 shows that the mean confidence for boys was higher for every decade of facility, although boys did not answer more questions, or answer more questions correctly, than girls. It might be hoped that the overconfidence of boys, and under-confidence of girls, could both be addressed through repeated use of confidence assessment over time, where accurate placement of confidence is incentivised (Foster, [<reflink idref="bib21" id="ref147">21</reflink>], [<reflink idref="bib22" id="ref148">22</reflink>]). For the students in this dataset, there was a significant interaction between facility and gender (<emph>β</emph> = −0.052, <emph>p</emph> =.011), indicating that the relative overconfidence of boys was greater for lower-attaining students, which is a new finding. Helping lower-attaining students, both boys and girls, to gain an accurate sense of their strengths and weaknesses could be important in enabling improved student metacognition as an important facet of more effective learning.</p> <p>Graph: Fig. 12 Confidence against facility by gender, with facility split into decades. The paired histogram shows the distribution of students by gender in the different facility decades. Above each pair, we show the results of a Mann-Whitney U test, two-sided with Bonferroni correction. ns:.05 < p, *.01 < p ≤.05, **.001 < p ≤.01, ***.0001 < p ≤.001, ****p ≤.0001</p> <p>We also confirmed previous research showing that attitudes to mathematics tend to decrease across the primary to secondary school transition and continue to fall throughout lower secondary school (Galton et al., [<reflink idref="bib26" id="ref149">26</reflink>]; Greany et al., [<reflink idref="bib31" id="ref150">31</reflink>]; Mullis et al., [<reflink idref="bib46" id="ref151">46</reflink>]; Zanobini & Usai, [<reflink idref="bib63" id="ref152">63</reflink>]). In our study, this pattern was present across almost all decades of facility but was particularly marked across the primary to secondary transition (see Fig. 6). A new result from our mediation analysis was that only 18.7% of the relationship between age and confidence was mediated by decreased facility, meaning that there would appear to be more to students' decreasing confidence than merely experiencing harder mathematics as they progress through school. However, it is important to note that the present study was cross-sectional and did not track <emph>the same</emph> students over time, so it is possible that these effects could arise from other changes, such as in teaching or curriculum, that were not principally age-related. Further longitudinal research should explore how students' progress in mathematics from primary to secondary school, as increasingly challenging and abstract concepts are introduced, might be achieved without detrimental effects on their confidence.</p> <p>Finally, we confirmed the findings of studies reporting that confidence decreases with increasing socioeconomic disadvantage (OECD, [<reflink idref="bib47" id="ref153">47</reflink>], [<reflink idref="bib48" id="ref154">48</reflink>]). In our study, students classified as socioeconomically disadvantaged (defined as being eligible for either Free School Meals [Gorard, [<reflink idref="bib28" id="ref155">28</reflink>]] or Pupil Premium [Gov.UK, [<reflink idref="bib29" id="ref156">29</reflink>]]) answered fewer questions than advantaged students, answered fewer questions correctly and had lower mean confidence. The extensive dataset available for this study allowed us to probe this relationship in more detail than had previously been possible. Our mediation analysis found that 67.0% of the relationship between advantage and confidence was mediated by increased facility, meaning that the remaining 32.5% was either directly related to advantage or mediated by other variables (measured or not). Consequently, our model predicts that for two students of equal mathematics attainment, but differing in socioeconomic disadvantage, the more disadvantaged student would have lower confidence. This implies that the experience of a disadvantaged student studying mathematics at school may be considerably worse than for an advantaged student in <emph>multiple</emph> ways, since not only are disadvantaged students on average less successful in their mathematics, but they also on average feel less sure of their mathematics than an equally well-achieving more advantaged student does. This could be a consequence of messages conveyed through school, such as setting practices, for example (see Connolly et al., [<reflink idref="bib14" id="ref157">14</reflink>]), or of factors outside of school, or a combination.</p> <hd id="AN0155342115-20">The hypercorrection effect in an authentic learning context</hd> <p>We also found support in this study for the hypercorrection effect (Butterfield & Metcalfe, [<reflink idref="bib8" id="ref158">8</reflink>], [<reflink idref="bib9" id="ref159">9</reflink>]), for the first time in an authentic mathematics learning context. We found that the higher a student's confidence in their initial incorrect response in quiz A, the more likely they were to answer the subsequent analogous question in quiz B correctly, 3 weeks later, even after controlling for student facility. Each one-unit increase on our 5-point emoji confidence scale for the incorrect response in quiz A was associated with a 7% increase in the probability of a correct answer on quiz B. In this dataset, the hypercorrection effect was stronger for girls and for younger students, and validating these findings in a new data set would be valuable. One major benefit of incorporating confidence assessment into diagnostic questions may be to prompt students to reflect on their confidence level when incorrect and so take advantage of the hypercorrection effect (see Barton, [<reflink idref="bib3" id="ref160">3</reflink>], [<reflink idref="bib5" id="ref161">5</reflink>]). This study provides encouragement that this effect operates beyond the tightly controlled research studies previously reported.</p> <hd id="AN0155342115-21">Limitations</hd> <p>This study explored students' responses online, and caution may be needed in generalising from this to face-to-face classroom situations. However, the data were derived from real students in normal learning situations that incorporate both in-school and out-of-school engagement with the platform, so the data derive from an authentic learning situation. The diagnostic mathematics questions used covered a broad range of mathematics topics, and, although the questions used were all short questions, they spanned factual, procedural and conceptual understanding, as outlined in Section 1.</p> <p>An important limitation of analysing confidence scores is that students may exaggerate their confidence for strategic purposes, beyond what they truly think, in order to present themselves more favourably to their teacher or peers (see Foster, [<reflink idref="bib21" id="ref162">21</reflink>], [<reflink idref="bib22" id="ref163">22</reflink>]). We cannot completely rule out this possibility; however, the fact that confidence scores ranged across all of the emojis is evidence that large numbers of students were willing to register low confidence. One way in which to collect confidence measures with higher validity could be to implement confidence-<emph>weighted</emph> assessment, in which the student's mark is <emph>increased</emph> when the student expresses high confidence and is correct but is <emph>decreased</emph> when the student expresses high confidence but is incorrect (see Dirkzwager, [<reflink idref="bib17" id="ref164">17</reflink>]; Foster, [<reflink idref="bib21" id="ref165">21</reflink>], [<reflink idref="bib22" id="ref166">22</reflink>]; Rosewell, [<reflink idref="bib56" id="ref167">56</reflink>]; Sparck et al., [<reflink idref="bib60" id="ref168">60</reflink>]). Such an approach is intended over time to incentivise truthful confidence ratings and is particularly easy to implement in a multiple-choice model, where students can be asked to allocate, say, 100 marks across the four options A–D. This would allow a student to give a response of 0, 0, 50, 50, for instance, to express their feeling that they were sure that neither A nor B was correct but were completely unsure regarding C and D. We plan to explore the effects of such a model in our future research.</p> <hd id="AN0155342115-22">Conclusion</hd> <p>Students' confidence in the responses that they give to mathematics questions gives insight into an important aspect of their learning of mathematics (Barton, [<reflink idref="bib5" id="ref169">5</reflink>]; Foster, [<reflink idref="bib21" id="ref170">21</reflink>], [<reflink idref="bib22" id="ref171">22</reflink>]). Students' confidence has implications for their achievement, as well as for their experience of studying mathematics in school, and their inclination, or otherwise, to pursue mathematics-related study beyond school (Paulsen & Betz, [<reflink idref="bib50" id="ref172">50</reflink>]). Most previous research into students' confidence in mathematics has been carried out on a small scale or in situations deliberately contrived for research purposes. In the present study, we used a large dataset of students' responses to diagnostic multiple-choice mathematics questions, generated in the course of students' ordinary school learning, to confirm previous findings about the relationships between students' confidence and other variables. We were also able to report new findings, such as that the relative overconfidence of boys was greater for lower-attaining students, and that the decline in confidence with increasing age could not be fully accounted for by a decrease in students' success on harder mathematics. We also found that socioeconomically disadvantaged students on average feel less confident than equally well-achieving more advantaged students, suggesting that targeted support to socioeconomically disadvantaged students to improve their confidence could be valuable, alongside attempts to help raise their attainment.</p> <p>Finding the first support for the hypercorrection effect in an authentic school mathematics learning context suggests that there could be benefits to students' learning in exploiting this effect in lessons, as has been previously suggested (Barton, [<reflink idref="bib5" id="ref173">5</reflink>]). Future studies should, unlike the present study, track the same students over time, to investigate whether changes in confidence are associated with changes in teaching or the curriculum. It would also be important to examine school-level data, and the ways in which the <emph>Eedi</emph> platform is used differently in different schools or classes or with different teachers. For example, we do not know the extent to which teachers may have encouraged the students to <emph>reflect</emph> on their confidence explicitly (see Barton, [<reflink idref="bib5" id="ref174">5</reflink>]). In further research, we intend to examine different ways in which teachers might harness the confidence assessment feature of the platform to help students become better calibrated over time and benefit more from the hypercorrection effect.</p> <hd id="AN0155342115-23">Acknowledgements</hd> <p>We would like to thank the editor and the anonymous reviewers for their extremely helpful comments on previous versions of this paper.</p> <hd id="AN0155342115-24">Availability of data and material</hd> <p>All data is available at https://github.com/Eedi/diagnostic-questions-and-student-confidence.</p> <hd id="AN0155342115-25">Code availability</hd> <p>All <emph>Python</emph> code is provided at https://github.com/Eedi/diagnostic-questions-and-student-confidence.</p> <hd id="AN0155342115-26">Declarations</hd> <p></p> <hd id="AN0155342115-27">Conflict of interest</hd> <p>The second and third authors are co-founders of <emph>Eedi</emph>.</p> <hd id="AN0155342115-28">Appendix. Comparability of quiz A and quiz B questions</hd> <p>For the analysis of the hypercorrection effect, we need to know how similar the analogous questions are in quiz A and quiz B. We can test this by comparing the facility of questions in quiz A to the corresponding questions in quiz B. When calculating the facility of questions in quiz B, we excluded any answers from students who had already answered the corresponding question in quiz A. We might expect that quiz A, administered immediately after the teaching, would show higher facility than quiz B, which is a delayed test given some 3 weeks later. This might be expected to be particularly the case, given that, for quiz B, we include only those students who were <emph>not</emph> present for quiz A, and a likely reason for this could be that they were absent from school at that time, meaning that they are likely to have missed some of the teaching on that topic.</p> <p>There were 6370 question pairs in the dataset from 7302 students in total. We identified <emph>N</emph> = 6232 for which there were at least 20 answers to the quiz A question, and at least 20 answers to the quiz B question where the student had not answered the question in quiz A. A Wilcoxon signed-rank test found no evidence that the distribution of the paired differences was unsymmetrical about the median (<emph>Mdn</emph> = −1.38, <emph>Z</emph> = 9,709,954.5, <emph>p</emph> =.994, see Fig. 13). Together with the fact that the median is very close to 0, this leads us to conclude that, despite the plausible concerns above, on average the difference in difficulty between quiz A and quiz B questions is too small to be of any concern.</p> <p>Graph: Fig. 13 Comparison of question difficulty between quizzes A and B</p> <hd id="AN0155342115-29">Publisher's note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0155342115-30"> <title> References </title> <blist> <bibl id="bib1" idref="ref40" type="bt">1</bibl> <bibtext> Baguley, T. (2012). Serious stats: A guide to advanced statistics for the behavioral sciences. Macmillan International Higher Education.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref11" type="bt">2</bibl> <bibtext> Bandura A. Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review. 1977; 84; 2: 191-215. 10.1037/0033-295X.84.2.191</bibtext> </blist> <blist> <bibl id="bib3" idref="ref37" type="bt">3</bibl> <bibtext> Barton C. How I wish I'd taught maths: Lessons learned from research, conversations with experts, and 12 years of mistakes. 2018John Catt Educational Limited</bibtext> </blist> <blist> <bibl id="bib4" idref="ref38" type="bt">4</bibl> <bibtext> Barton C. On formative assessment in math: How diagnostic questions can help. American Educator. 2018; 42; 2: 33</bibtext> </blist> <blist> <bibl id="bib5" idref="ref35" type="bt">5</bibl> <bibtext> Barton C. Reflect, expect, check, explain: Sequences and behaviour to enable mathematical thinking in the classroom. 2020; John Catt Educational Ltd.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref15" type="bt">6</bibl> <bibtext> Burton L. "Confidence is everything"—Perspectives of teachers and students on learning mathematics. Journal of Mathematics Teacher Education. 2004; 7; 4: 357-381. 10.1007/s10857-004-3355-y</bibtext> </blist> <blist> <bibl id="bib7" idref="ref62" type="bt">7</bibl> <bibtext> Butler AC, Fazio LK, Marsh EJ. The hypercorrection effect persists over a week, but high-confidence errors return. Psychonomic Bulletin & Review. 2011; 18; 6: 1238-1244. 10.3758/s13423-011-0173-y</bibtext> </blist> <blist> <bibl id="bib8" idref="ref41" type="bt">8</bibl> <bibtext> Butterfield B, Metcalfe J. Errors committed with high confidence are hypercorrected. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2001; 27: 1491-1494. 10.1037/0278-7393.27.6.1491</bibtext> </blist> <blist> <bibl id="bib9" idref="ref91" type="bt">9</bibl> <bibtext> Butterfield B, Metcalfe J. The correction of errors committed with high confidence. Metacognition and Learning. 2006; 1; 1: 69-84. 10.1007/s11409-006-6894-z</bibtext> </blist> <blist> <bibtext> Carpenter SK, Haynes CL, Corral D, Yeung KL. Hypercorrection of high-confidence errors in the classroom. Memory. 2018; 26; 10: 1379-1384. 10.1080/09658211.2018.1477164</bibtext> </blist> <blist> <bibtext> Clarkson, L. M. C, Love, Q. U, & Ntow, F. D. (2017). How confidence relates to mathematics achievement: A new framework. In A Chronaki (Ed.), Mathematics Education and Life at Times of Crisis, Proceedings of the Ninth International Mathematics Education and Society Conference (Vol. 2, pp. 441–451). University of Thessaly Press.</bibtext> </blist> <blist> <bibtext> Clark-Wilson, A, Robutti, O, & Thomas, M. (2020). Teaching with digital technology. ZDM-Mathematics Education, 52(7), 1223–1242. https://doi.org/10.1007/s11858-020-01196-0</bibtext> </blist> <blist> <bibtext> Code W, Merchant S, Maciejewski W, Thomas M, Lo J. The Mathematics Attitudes and Perceptions Survey: An instrument to assess expert-like views and dispositions among undergraduate mathematics students. International Journal of Mathematical Education in Science and Technology. 2016; 47; 6: 917-937. 10.1080/0020739X.2015.1133854</bibtext> </blist> <blist> <bibtext> Connolly P, Taylor B, Francis B, Archer L, Hodgen J, Mazenod A, Tereshchenko A. The misallocation of students to academic sets in maths: A study of secondary schools in England. British Educational Research Journal. 2019; 45; 4: 873-897. 10.1002/berj.3530</bibtext> </blist> <blist> <bibtext> de Bruin AB, van Gog T. Improving self-monitoring and self-regulation: From cognitive psychology to the classroom. Learning and Instruction. 2012; 22; 4: 245-252. 10.1016/j.learninstruc.2012.01.003</bibtext> </blist> <blist> <bibtext> Di Martino P, Zan R. 'Me and maths': Towards a definition of attitude grounded on students' narratives. Journal of Mathematics Teacher Education. 2010; 13; 1: 27-48. 10.1007/s10857-009-9134-z</bibtext> </blist> <blist> <bibtext> Dirkzwager A. Multiple evaluation: A new testing paradigm that exorcizes guessing. International Journal of Testing. 2003; 3; 4: 333-352. 10.1207/S15327574IJT0304_3</bibtext> </blist> <blist> <bibtext> Efklides A. Commentary: How readily can findings from basic cognitive psychology research be applied in the classroom?. Learning and Instruction. 2012; 22; 4: 290-295. 10.1016/j.learninstruc.2012.01.001</bibtext> </blist> <blist> <bibtext> Fennema E, Sherman JA. Fennema-Sherman mathematics attitudes scales: Instruments designed to measure attitudes toward the learning of mathematics by females and males. Journal for Research in Mathematics Education. 1976; 7; 5: 324-326. 10.2307/748467</bibtext> </blist> <blist> <bibtext> Fischhoff B, Slovic P, Lichtenstein S. Knowing with certainty: The appropriateness of extreme confidence. Journal of Experimental Psychology. 1977; 3; 4: 552-564. 10.1037/0096-1523.3.4.552</bibtext> </blist> <blist> <bibtext> Foster C. Confidence and competence with mathematical procedures. Educational Studies in Mathematics. 2016; 91; 2: 271-288. 10.1007/s10649-015-9660-9</bibtext> </blist> <blist> <bibtext> Foster, C. (2021). Implementing confidence assessment in low-stakes, formative mathematics assessments. International Journal of Science and Mathematics Education. Advance online publication. https://doi.org/10.1007/s10763-021-10207-9</bibtext> </blist> <blist> <bibtext> Franz-Odendaal TA, Blotnicky KA, Joy P. Math self-efficacy and the likelihood of pursuing a STEM-based career: A gender-based analysis. Canadian Journal of Science, Mathematics and Technology Education. 2020; 20: 538-556. 10.1007/s42330-020-00105-7</bibtext> </blist> <blist> <bibtext> Frost LA, Hyde JS, Fennema E. Gender, mathematics performance, and mathematics-related attitudes and affect: A meta-analytic synthesis. International Journal of Educational Research. 1994; 21; 4: 373-385. 10.1016/S0883-0355(06)80026-1</bibtext> </blist> <blist> <bibtext> Galbraith P, Haines C. Disentangling the nexus: Attitudes to mathematics and technology in a computer learning environment. Educational Studies in Mathematics. 1998; 36; 3: 275-290. 10.1023/A:1003198120666</bibtext> </blist> <blist> <bibtext> Galton, M, Gray, J, & Rudduck, J. (2003). Transfer and transitions in the middle years of schooling (7-14): Continuities and discontinuities in learning. DfES.</bibtext> </blist> <blist> <bibtext> Ganley CM, Lubienski ST. Mathematics confidence, interest, and performance: Examining gender patterns and reciprocal relations. Learning and Individual Differences. 2016; 47: 182-193. 10.1016/j.lindif.2016.01.002</bibtext> </blist> <blist> <bibtext> Gorard, S. (2012). Who is eligible for free school meals? Characterising free school meals as a measure of disadvantage in England. British Educational Research Journal, 38(6), 1003–1017. https://doi.org/10.1080/01411926.2011.608118</bibtext> </blist> <blist> <bibtext> Gov.UK (2020a). Pupil premium. https://<ulink href="http://www.gov.uk/government/publications/pupil-premium/pupil-premium">www.gov.uk/government/publications/pupil-premium/pupil-premium</ulink></bibtext> </blist> <blist> <bibtext> Gov.UK (2020b). Schools, pupils and their characteristics. https://<ulink href="http://www.gov.uk/government/statistics/schools-pupils-and-their-characteristics-january-2020">www.gov.uk/government/statistics/schools-pupils-and-their-characteristics-january-2020</ulink></bibtext> </blist> <blist> <bibtext> Greany, T, Barnes, I, Mostafa, T, Pensiero, N, & Swensson, C. (2016). Trends in Maths and Science Study (TIMSS): National Report for England. UCL Institute of Education.</bibtext> </blist> <blist> <bibtext> Hannula MLerman S. Affect in mathematics education. Encyclopedia of mathematics education. 2014; Springer: 23-27. 10.1007/978-94-007-4978-8_174</bibtext> </blist> <blist> <bibtext> Hannula, M. S, Leder, G. C, Morselli, F, Vollstedt, M, & Zhang, Q. (2019). Affect and mathematics education: Fresh perspectives on motivation, engagement, and identity. Springer Nature. https://doi.org/10.1007/978-3-030-13761-8</bibtext> </blist> <blist> <bibtext> Kaye LK, Malone SA, Wall HJ. Emojis: Insights, affordances, and possibilities for psychological science. Trends in Cognitive Sciences. 2017; 21; 2: 66-68. 10.1016/j.tics.2016.10.007</bibtext> </blist> <blist> <bibtext> Kyriacou C. The impact of daily mathematics lessons in England on pupil confidence and competence in early mathematics: A systematic review. British Journal of Educational Studies. 2005; 53; 2: 168-186. 10.1111/j.1467-8527.2005.00289.x</bibtext> </blist> <blist> <bibtext> Lim SY, Chapman E. Development of a short form of the attitudes toward mathematics inventory. Educational Studies in Mathematics. 2013; 82; 1: 145-164. 10.1007/s10649-012-9414-x</bibtext> </blist> <blist> <bibtext> Ma X, Kishor N. Assessing the relationship between attitude toward mathematics and achievement in mathematics: A meta-analysis. Journal for Research in Mathematics Education. 1997; 28; 1: 26-47. 10.2307/749662</bibtext> </blist> <blist> <bibtext> Marsh HW, Pekrun R, Parker PD, Murayama K, Guo J, Dicke T, Arens AK. The murky distinction between self-concept and self-efficacy: Beware of lurking jingle-jangle fallacies. Journal of Educational Psychology. 2019; 111; 2: 331-353. 10.1037/edu0000281</bibtext> </blist> <blist> <bibtext> Massey, S. (2021). Using emojis and drawings in surveys to measure children's attitudes to mathematics. International Journal of Social Research Methodology, 1-13. Advance online publication. https://doi.org/10.1080/13645579.2021.1940774</bibtext> </blist> <blist> <bibtext> Mendick H, Francis B. Boffin and geek identities: Abject or privileged?. Gender and Education. 2012; 24; 1: 15-24. 10.1080/09540253.2011.564575</bibtext> </blist> <blist> <bibtext> Metcalfe J. Learning from errors. Annual Review of Psychology. 2017; 68: 465-489. 10.1146/annurev-psych-010416-044022</bibtext> </blist> <blist> <bibtext> Metcalfe J, Finn B. People's hypercorrection of high-confidence errors: Did they know it all along?. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2011; 37; 2: 437-448. 10.1037/a0021962</bibtext> </blist> <blist> <bibtext> Metcalfe J, Finn B. Hypercorrection of high confidence errors in children. Learning and Instruction. 2012; 22; 4: 253-261. 10.1016/j.learninstruc.2011.10.004</bibtext> </blist> <blist> <bibtext> Metcalfe J, Huelser BJ. Learning from errors is attributable to episodic recollection rather than semantic mediation. Neuropsychologia. 2020; 138: 107296. 10.1016/j.neuropsychologia.2019.107296</bibtext> </blist> <blist> <bibtext> Morsanyi K, Cheallaigh NN, Ackerman R. Mathematics anxiety and metacognitive processes: Proposal for a new line of inquiry. Psihologijske Teme. 2019; 28; 1: 147-169. 10.31820/pt.28.1.8</bibtext> </blist> <blist> <bibtext> Mullis, I. V. S, Martin, M. O, Foy, P, Kelly, D. L, & Fishbein, B. (2020). TIMSS 2019 International Results in Mathematics and Science. Retrieved from Boston College, TIMSS & PIRLS International Study Center website: https://timssandpirls.bc.edu/timss2019/international-results/</bibtext> </blist> <blist> <bibtext> OECD. (2013). Mathematics self-beliefs and participation in mathematics-related activities. In PISA 2012 Results: Ready to learn (Volume III), Students' engagement, drive and self-beliefs (pp. 87–112). OECD Publishing. https://doi.org/10.1787/9789264201170-en</bibtext> </blist> <blist> <bibtext> OECD. PISA 2018 Results (Volume III): What school life means for students' lives. 2019; PISA, OECD Publishing. 10.1787/acd78851-en</bibtext> </blist> <blist> <bibtext> Panaoura A, Gagatsis A, Demetriou A. An intervention to the metacognitive performance: Self-regulation in mathematics and mathematical modeling. Acta Didactica Universitatis Comenianae Mathematics. 2009; 9: 63-79</bibtext> </blist> <blist> <bibtext> Paulsen AM, Betz NE. Basic confidence predictors of career decision-making self-efficacy. The Career Development Quarterly. 2004; 52; 4: 354-362. 10.1002/j.2161-0045.2004.tb00951.x</bibtext> </blist> <blist> <bibtext> Pepin B, Roesken-Winter B. From beliefs to dynamic affect systems in mathematics education: Exploring a mosaic of relationships and interactions. 2014; Springer. 10.1007/978-3-319-06808-4</bibtext> </blist> <blist> <bibtext> Pierce R, Stacey K. A framework for monitoring progress and planning teaching towards the effective use of computer algebra systems. International Journal of Computers for Mathematical Learning. 2004; 9: 59-93. 10.1023/B:IJCO.0000038246.98119.14</bibtext> </blist> <blist> <bibtext> Pinxten M, Marsh HW, De Fraine B, Van Den Noortgate W, Van Damme J. Enjoying mathematics or feeling competent in mathematics? Reciprocal effects on mathematics achievement and perceived math effort expenditure. British Journal of Educational Psychology. 2014; 84; 1: 152-174. 10.1111/bjep.12028</bibtext> </blist> <blist> <bibtext> Pomerantz EM, Altermatt ER, Saxon JL. Making the grade but feeling distressed: Gender differences in academic performance and internal distress. Journal of Educational Psychology. 2002; 94; 2: 396-404. 10.1037/0022-0663.94.2.396</bibtext> </blist> <blist> <bibtext> Rohrer D, Taylor K. The shuffling of mathematics problems improves learning. Instructional Science. 2007; 35; 6: 481-498. 10.1007/s11251-007-9015-8</bibtext> </blist> <blist> <bibtext> Rosewell, J. P. (2011) Opening up multiple-choice: Assessing with confidence. Presented at the 2011 International Computer Assisted Assessment (CAA) Conference: Research into e-Assessment, 5-6 July 2011, Southampton, UK. <ulink href="http://oro.open.ac.uk/32150">http://oro.open.ac.uk/32150</ulink></bibtext> </blist> <blist> <bibtext> Seabold, S. & Perktold, J. (2010). Statsmodels: Econometric and statistical modeling with Python. 9th Python in Science Conference. https://doi.org/10.25080/Majora-92bf1922-011</bibtext> </blist> <blist> <bibtext> Sitzman DM, Rhodes MG, Tauber SK, Liceralde VRT. The role of prior knowledge in error correction for younger and older adults: Correction. Aging, Neuropsychology, and Cognition. 2020; 27; 4: 636-637. 10.1080/13825585.2019.1640443</bibtext> </blist> <blist> <bibtext> Soderstrom NC, Bjork RA. Learning versus performance: An integrative review. Perspectives on Psychological Science. 2015; 10; 2: 176-199. 10.1177/1745691615569000</bibtext> </blist> <blist> <bibtext> Sparck EM, Bjork EL, Bjork RA. On the learning benefits of confidence-weighted testing. Cognitive Research: Principles and Implications. 2016; 1; 1: 3. 10.1186/s41235-016-0003-x</bibtext> </blist> <blist> <bibtext> Stankov L, Lee J, Luo W, Hogan DJ. Confidence: A better predictor of academic achievement than self-efficacy, self-concept and anxiety?. Learning and Individual Differences. 2012; 22; 6: 747-758. 10.1016/j.lindif.2012.05.013</bibtext> </blist> <blist> <bibtext> van Loon MH, Dunlosky J, van Gog T, van Merriënboer JJ, de Bruin AB. Refutations in science texts lead to hypercorrection of misconceptions held with high confidence. Contemporary Educational Psychology. 2015; 42: 39-48. 10.1016/j.cedpsych.2015.04.003</bibtext> </blist> <blist> <bibtext> Zanobini M, Usai MC. Domain-specific self-concept and achievement motivation in the transition from primary to low middle school. Educational Psychology. 2002; 22; 2: 203-217. 10.1080/01443410120115265</bibtext> </blist> </ref> <ref id="AN0155342115-31"> <title> Footnotes </title> <blist> <bibtext> See https://app.eedi.com/113906/collections/ULTIMATE_SOW.</bibtext> </blist> <blist> <bibtext> Students could answer quizzes on the <emph>Diagnostic Questions</emph> website (<ulink href="http://www.diagnosticquesitons.com">www.diagnosticquesitons.com</ulink>), the <emph>Eedi</emph> website (<ulink href="http://www.eedi.co.uk">www.eedi.co.uk</ulink>) or the <emph>Eedi</emph> mobile app. Confidence could not be recorded on <emph>Diagnostic Questions</emph> and some versions of the <emph>Eedi</emph> mobile app, so we include here only answers given on the <emph>Eedi</emph> website.</bibtext> </blist> <blist> <bibtext> Free School Meal eligibility is a measure of low parental income, often used as an individual indicator of socioeconomic disadvantage (see Gorard, [28]).</bibtext> </blist> <blist> <bibtext> The pupil premium is extra funding given to schools in England to raise the attainment of disadvantaged pupils (Gov.UK, [29]).</bibtext> </blist> <blist> <bibtext> Standardisation was by subtracting the mean and dividing by the standard deviation.</bibtext> </blist> <blist> <bibtext> We note that the <emph>n</emph>s for each confidence level suggest a preference for confidence ratings of 0, 50 and 100 over 25 and 75. For this dataset, 29% of students used 0, 50, 100 only for at least 90% of their answers. For the full dataset, 23% of students used 0, 50, 100 for at least 90% of their answers, suggesting that this reduction of a 5-point scale to a 3-point scale was fairly common.</bibtext> </blist> </ref> <aug> <p>By Colin Foster; Simon Woodhead; Craig Barton and Alison Clark-Wilson</p> <p>Reported by Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib21" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib22" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib32" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib13" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib16" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib33" firstref="ref6"></nolink> <nolink nlid="nl7" bibid="bib35" firstref="ref7"></nolink> <nolink nlid="nl8" bibid="bib51" firstref="ref8"></nolink> <nolink nlid="nl9" bibid="bib11" firstref="ref9"></nolink> <nolink nlid="nl10" bibid="bib38" firstref="ref10"></nolink> <nolink nlid="nl11" bibid="bib61" firstref="ref14"></nolink> <nolink nlid="nl12" bibid="bib25" firstref="ref16"></nolink> <nolink nlid="nl13" bibid="bib52" firstref="ref17"></nolink> <nolink nlid="nl14" bibid="bib19" firstref="ref18"></nolink> <nolink nlid="nl15" bibid="bib36" firstref="ref19"></nolink> <nolink nlid="nl16" bibid="bib46" firstref="ref20"></nolink> <nolink nlid="nl17" bibid="bib37" firstref="ref21"></nolink> <nolink nlid="nl18" bibid="bib27" firstref="ref22"></nolink> <nolink nlid="nl19" bibid="bib53" firstref="ref23"></nolink> <nolink nlid="nl20" bibid="bib45" firstref="ref26"></nolink> <nolink nlid="nl21" bibid="bib23" firstref="ref27"></nolink> <nolink nlid="nl22" bibid="bib50" firstref="ref28"></nolink> <nolink nlid="nl23" bibid="bib47" firstref="ref31"></nolink> <nolink nlid="nl24" bibid="bib48" firstref="ref32"></nolink> <nolink nlid="nl25" bibid="bib12" firstref="ref36"></nolink> <nolink nlid="nl26" bibid="bib55" firstref="ref43"></nolink> <nolink nlid="nl27" bibid="bib59" firstref="ref44"></nolink> <nolink nlid="nl28" bibid="bib42" firstref="ref63"></nolink> <nolink nlid="nl29" bibid="bib43" firstref="ref64"></nolink> <nolink nlid="nl30" bibid="bib20" firstref="ref69"></nolink> <nolink nlid="nl31" bibid="bib49" firstref="ref71"></nolink> <nolink nlid="nl32" bibid="bib24" firstref="ref72"></nolink> <nolink nlid="nl33" bibid="bib54" firstref="ref75"></nolink> <nolink nlid="nl34" bibid="bib40" firstref="ref78"></nolink> <nolink nlid="nl35" bibid="bib26" firstref="ref82"></nolink> <nolink nlid="nl36" bibid="bib31" firstref="ref83"></nolink> <nolink nlid="nl37" bibid="bib63" firstref="ref84"></nolink> <nolink nlid="nl38" bibid="bib62" firstref="ref89"></nolink> <nolink nlid="nl39" bibid="bib10" firstref="ref90"></nolink> <nolink nlid="nl40" bibid="bib44" firstref="ref93"></nolink> <nolink nlid="nl41" bibid="bib58" firstref="ref96"></nolink> <nolink nlid="nl42" bibid="bib15" firstref="ref99"></nolink> <nolink nlid="nl43" bibid="bib18" firstref="ref100"></nolink> <nolink nlid="nl44" bibid="bib41" firstref="ref101"></nolink> <nolink nlid="nl45" bibid="bib28" firstref="ref107"></nolink> <nolink nlid="nl46" bibid="bib29" firstref="ref108"></nolink> <nolink nlid="nl47" bibid="bib30" firstref="ref109"></nolink> <nolink nlid="nl48" bibid="bib34" firstref="ref110"></nolink> <nolink nlid="nl49" bibid="bib39" firstref="ref111"></nolink> <nolink nlid="nl50" bibid="bib57" firstref="ref115"></nolink> <nolink nlid="nl51" bibid="bib137" firstref="ref123"></nolink> <nolink nlid="nl52" bibid="bib654" firstref="ref131"></nolink> <nolink nlid="nl53" bibid="bib14" firstref="ref157"></nolink> <nolink nlid="nl54" bibid="bib17" firstref="ref164"></nolink> <nolink nlid="nl55" bibid="bib56" firstref="ref167"></nolink> <nolink nlid="nl56" bibid="bib60" firstref="ref168"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1326555
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: School Students' Confidence When Answering Diagnostic Questions Online
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Foster%2C+Colin%22">Foster, Colin</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1648-7485">0000-0003-1648-7485</externalLink>)<br /><searchLink fieldCode="AR" term="%22Woodhead%2C+Simon%22">Woodhead, Simon</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-2192-9797">0000-0002-2192-9797</externalLink>)<br /><searchLink fieldCode="AR" term="%22Barton%2C+Craig%22">Barton, Craig</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3566-2795">0000-0003-3566-2795</externalLink>)<br /><searchLink fieldCode="AR" term="%22Clark-Wilson%2C+Alison%22">Clark-Wilson, Alison</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-6479-4677">0000-0002-6479-4677</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Educational+Studies+in+Mathematics%22"><i>Educational Studies in Mathematics</i></searchLink>. Mar 2022 109(3):491-521.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 31
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Secondary+School+Students%22">Secondary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Diagnostic+Tests%22">Diagnostic Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Confidence+Testing%22">Confidence Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Gender+Differences%22">Gender Differences</searchLink><br /><searchLink fieldCode="DE" term="%22Socioeconomic+Status%22">Socioeconomic Status</searchLink><br /><searchLink fieldCode="DE" term="%22Age+Differences%22">Age Differences</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Tests%22">Mathematics Tests</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22United+Kingdom%22">United Kingdom</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s10649-021-10084-7
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0013-1954
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In this paper, we analyse a large, opportunistic dataset of responses (N = 219,826) to online, diagnostic multiple-choice mathematics questions, provided by 6-16-year-old UK school mathematics students (N = 7302). For each response, students were invited to indicate on a 5-point Likert-type scale how confident they were that their response was correct. Using demographic data available from the online platform, we examine the relationships between confidence and facility (the proportion of questions correct), as well as gender, age and socioeconomic disadvantage. We found a positive correlation between student confidence and mean facility, higher confidence for boys than for girls and lower confidence for students classified as socioeconomically disadvantaged, even after accounting for facility. We found that confidence was lower for older students, and this was particularly marked across the primary to secondary school transition. An important feature of the online platform used is that, when students answer a question incorrectly, they are presented with an analogous question about 3 weeks later. We exploited this feature to obtain the first evidence in an authentic school mathematics context for the "hypercorrection effect" (Butterfield & Metcalfe J EXP PSYCHOL 27:1491-1494, 2001), which is the observation that errors made with higher confidence are more likely to be corrected. These findings have implications for classroom practices that have the potential to support more effective and efficient learning of mathematics.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1326555
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1326555
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10649-021-10084-7
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 31
        StartPage: 491
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Elementary School Students
        Type: general
      – SubjectFull: Secondary School Students
        Type: general
      – SubjectFull: Multiple Choice Tests
        Type: general
      – SubjectFull: Diagnostic Tests
        Type: general
      – SubjectFull: Confidence Testing
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Gender Differences
        Type: general
      – SubjectFull: Socioeconomic Status
        Type: general
      – SubjectFull: Age Differences
        Type: general
      – SubjectFull: Mathematics Tests
        Type: general
      – SubjectFull: United Kingdom
        Type: general
    Titles:
      – TitleFull: School Students' Confidence When Answering Diagnostic Questions Online
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Foster, Colin
      – PersonEntity:
          Name:
            NameFull: Woodhead, Simon
      – PersonEntity:
          Name:
            NameFull: Barton, Craig
      – PersonEntity:
          Name:
            NameFull: Clark-Wilson, Alison
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-print
              Value: 0013-1954
          Numbering:
            – Type: volume
              Value: 109
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Educational Studies in Mathematics
              Type: main
ResultId 1