View in EDS HTML Full Text PDF Full Text

Does Supportive Feedback on Class Rank Improve Scores for Intermediate-Level Microeconomics?

Saved in:
Bibliographic Details
Title: Does Supportive Feedback on Class Rank Improve Scores for Intermediate-Level Microeconomics?
Language: English
Authors: Chanita C. Holmes, Marlon R. Tracey
Source: Journal of Economic Education. 2025 56(1):22-29.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 8
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Class Rank, Economics Education, Grades (Scholastic), Advanced Courses, Intervention, Student Improvement, Educational Practices, Microeconomics, Homework, Undergraduate Students, State Universities, Electronic Mail, Tests
Geographic Terms: Virginia
DOI: 10.1080/00220485.2024.2399591
ISSN: 0022-0485
2152-4068
Abstract: Instructors may use low-cost, light-touch strategies to help students achieve optimal effort in demanding upper-level courses. The authors of this study exploit an intervention that provides a series of personalized feedback emails to students about their relative performance, which is tied to approving messages or tips that encourage improvement. The authors analyze the efficacy of this supportive rank feedback in an intermediate-level microeconomics theory course, given the challenges it poses to students. Evidence shows gains from feedback across the grade distribution, but they are short-lived and decline toward zero as the course progresses.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1455827
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwF3vXOPxJu2OJX0O31SI2bsAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDGRCz2MxV7f4uuQgegIBEICBm34jo1NNWu4h2hvmVBb59MPCaI2AVMO07RaTKLD99hAnvtBiKml9dDxaPTTVV9CMxWQuLJZxP93cboV3eQxsvCdz135cKedQ7K0H7Fe9VBK0dU0yiq_jXHgDh_O24KvWKHuguV0LMFFI_ae47SFy7C4VgDXFlR4QtWYkoZSFSHFp-o91yFPu7g1g3e9sh2kVQWJAXczGeoLqhwwZ
Text:
  Availability: 1
  Value: <anid>AN0181888593;jmd01jan.25;2024Dec30.05:05;v2.2.500</anid> <title id="AN0181888593-1">Does supportive feedback on class rank improve scores for intermediate-level microeconomics? </title> <p>Instructors may use low-cost, light-touch strategies to help students achieve optimal effort in demanding upper-level courses. The authors of this study exploit an intervention that provides a series of personalized feedback emails to students about their relative performance, which is tied to approving messages or tips that encourage improvement. The authors analyze the efficacy of this supportive rank feedback in an intermediate-level microeconomics theory course, given the challenges it poses to students. Evidence shows gains from feedback across the grade distribution, but they are short-lived and decline toward zero as the course progresses.</p> <p>Keywords: Academic performance; feedback; low-cost intervention; rank incentives; relative performance</p> <p>Economics courses, especially upper-level calculus-based ones, require considerable effort from students to perform well. Instructor behavior potentially plays a key role in helping students to exert optimal effort for success. When an instructor announces the class average on an assignment, this not only conveys overall absolute performance but also informs students of whether they performed above or below average. The instructor then either commends the students' efforts or offers tips to help them improve. Such feedback can update students' beliefs about their ability or relative productivity, which could impact their efforts on later assignments. Therefore, we assess how scores in an intermediate-level microeconomics course respond to a series of feedback emails that provide class rankings tied to encouraging advice.</p> <p>The expected effect of rank feedback is not obvious. One theory proposes that students use the information on peer performance to update incomplete beliefs about their ability (Azmat and Iriberri [<reflink idref="bib2" id="ref1">2</reflink>]; Dobrescu et al. [<reflink idref="bib8" id="ref2">8</reflink>]). Specifically, high-rank feedback signals that one has high ability, which would raise their self-efficacy and thus grades; however, low-rank (negative) feedback reveals low ability that could discourage effort. An alternate view on negative feedback is that individuals disregard it if they attribute it to bad luck or view it as atypical (Compte and Postlewaite [<reflink idref="bib6" id="ref3">6</reflink>]), in which case no grade effect is expected. It appears then that the direction of the relationship could vary across the ability distribution. The competitive preference theory (Azmat and Iriberri [<reflink idref="bib2" id="ref4">2</reflink>]; Dobrescu et al. [<reflink idref="bib8" id="ref5">8</reflink>]) challenges this. It suggests that students of any ability prefer to outdo others and, thus, use rank feedback to glean their peers' relative productivity in order to choose optimal effort. Gains are therefore expected at each level of ability, with some attenuation at the upper end of the distribution. Tying rank feedback to encouragement further adds to the ambiguity. While supportive feedback could raise self-concept and belief in one's ability, it may adversely affect grades if students feel pressured or less intrinsically motivated (Damgaard and Nielsen [<reflink idref="bib7" id="ref6">7</reflink>]).</p> <p>A small but growing empirical literature explores the link between performance and rank feedback in higher-education settings. Using a semester-long English course at a Vietnamese university, Tran and Zeckhauser ([<reflink idref="bib11" id="ref7">11</reflink>]) are the first to show this link. They find that students with private or public information about their class rank on biweekly in-course tests later got better scores on their standardized English proficiency tests. Studies on economics courses focus on the first-year introductory level. Kajitani, Morimoto, and Suzuki ([<reflink idref="bib9" id="ref8">9</reflink>]) show that when Japanese university students receive rank feedback based on their midterm exam in an introductory economics course, they do better on their final exam, especially if their midterm score is low. In principles of microeconomics at an Australian University, Dobrescu et al. ([<reflink idref="bib8" id="ref9">8</reflink>]) studied how feedback affects exams and a semester-long online assignment (sets of exercises). They exploit feedback measured as 5-minute updates to a leaderboard ranking of the online assignment in terms of completion and success rates. The authors find slightly declining but positive effects across a majority of the grade distribution if feedback is provided when a student's relative position changes, with spillovers only to courses taken one semester later.</p> <p>Other studies assess overall performance for multiple programs rather than a specific course, adding conflicting findings. Azmat et al. ([<reflink idref="bib1" id="ref10">1</reflink>]) report negative GPA effects of semiannual rank feedback for second-year students at a Spanish university, particularly those understating their prior beliefs about their rank. Meanwhile, for first-year university students in German business and engineering programs, Brade, Himmler, and Jäckle ([<reflink idref="bib3" id="ref11">3</reflink>]) find that feedback on first-semester accumulated credits affects only above-average students by raising their second-semester credit points. It appears that Carrell and Kurlaender ([<reflink idref="bib5" id="ref12">5</reflink>]) are the only ones to investigate the efficacy of supportive personalized feedback on <emph>absolute</emph> performance in higher education. They find that faculty encouragement or advice raises course grades for first-year underrepresented minorities at a California university.</p> <p>We examine a stage of educational output and an effort-inducing feedback intervention that differs from the literature in three main ways. First, student performance is assessed in a semester-long microeconomics course at the intermediate level. Kajitani, Morimoto, and Suzuki ([<reflink idref="bib9" id="ref13">9</reflink>]) and Dobrescu et al. ([<reflink idref="bib8" id="ref14">8</reflink>]) report gains from rank feedback for at least some principles students, but that does not promise gains for upper-level students with more demanding courses and greater interest in economics. Transitioning from introductory to intermediate-level economics courses can be a large leap for students, even for some who performed well at the introductory level. Second, emails are sent with <emph>both</emph> supportive (encouraging) and rank feedback. This is likely the case in actual classroom settings, as instructor encouragement is especially needed when students know they rank poorly relative to their peers. Also, because studies generally show positive returns to each kind of feedback, providing both (as opposed to none) makes it more likely to discover gains in a challenging, calculus-based course. Finally, to sufficiently target effort, feedback emails are sent consistently after every homework and exam, which are worth 95 percent of the course grade. Thus, unlike previous studies, feedback is not based on overall program performance (e.g., Azmat et al. [<reflink idref="bib1" id="ref15">1</reflink>]), online exercises only (Dobrescu et al. [<reflink idref="bib8" id="ref16">8</reflink>]), or exams only (e.g., Kajitani, Morimoto, and Suzuki [<reflink idref="bib9" id="ref17">9</reflink>]).</p> <p>Difference-in-differences (DiD) models are used to compare grade outcomes for a cohort of students (fall 2022) who received supportive rank feedback emails (i.e., the treated) and a similar but untreated cohort (fall 2021). The outcomes are semester GPA and the distribution of scores on each assessment during the semester. The key finding is higher scores from feedback at different levels of student ability in the first half of the semester, but gains dissipate later and do not spill over to other courses taken during the same semester.</p> <hd id="AN0181888593-2">Intervention design</hd> <p>A series of personalized feedback emails were sent to students of an intermediate-level microeconomics theory course in the fall of 2022 at a large public university in Virginia.[<reflink idref="bib1" id="ref18">1</reflink>] The emails provide precise relative performance information framed in a way that is not discouraging. Mail Merge was used to help personalize each email and automate the sending process via the instructor's university email address. Figure 1 displays the typical body of the emails, which differed among three groups based on letter grade performance (i.e., B + to A, C to B, and F to C−) on each assessment.[<reflink idref="bib2" id="ref19">2</reflink>] Each email body has four key components: (i) the student's absolute score; (ii) the student's percentile score; (iii) a pie chart comparing the size of the student's letter-grade group to the other groups; and (iv) some encouragement or advice for students. The latter component is structured similarly to Carrell and Kurlaender ([<reflink idref="bib5" id="ref20">5</reflink>]). That is, B + to A students were commended and encouraged to continue doing well as they mainly demonstrated accurate and complete knowledge of course content and skills. Because C to B students demonstrated only knowledge of basic content and skills, they were commended on their effort but advised to improve their scores through additional practice, class engagement, and seeking support. The university designates work earning a grade of C − or below as unsatisfactory. Thus, feedback emails to students at that level were designed to console students (by reminding them there is time to recover), provide tips for doing well (e.g., ask questions, redo problems), urge office hour attendance, and offer support.</p> <p>Graph: Figure 1. Typical email body sent to three groups of students.</p> <p>The course comprises a mixture of instructor-graded homework and exams (two midterms and one final).[<reflink idref="bib3" id="ref21">3</reflink>]Figure 2 outlines the timing of assessments throughout the fall 2022 semester. Once students completed an assessment, 4–5 days later, they were emailed feedback based on their performance on that assessment. For example, a student with a grade of D on HW #1 received Email #1 with feedback content corresponding to group F to C − in Figure 1. If the student then earns a C + on HW #2, they will receive Email #2 with the group C to B feedback content, and so on. Adequate time was available after feedback emails for students to adjust their effort level, which could impact subsequent assessments.</p> <p>Graph: Figure 2. Timeline of feedback messages and assessments in fall 2022.</p> <p>In fall 2022, the instructor designed the course to have the same format as fall 2021 when students did not receive supportive rank feedback.[<reflink idref="bib4" id="ref22">4</reflink>] On both occasions, the course modality was traditional (classroom-based), and the instructor used the same teaching material, assessments (timed consistently), and grading rubrics.[<reflink idref="bib5" id="ref23">5</reflink>] There were also no school policy changes that would affect class instruction and composition. Thus, the fall 2021 cohort serves as the pre-determined comparison group.</p> <p>If feedback emails were randomly assigned to a class of 30 to 40 students, there could be ethical concerns, low statistical power, or cross-contamination as students could share feedback or any benefits from them. To avoid these issues, the ideal approach is to use at least six clusters of students taking the course with the same instructor and randomly assign feedback to each cluster.[<reflink idref="bib6" id="ref24">6</reflink>] Our study's intervention is analogous, with clusters representing cohorts of students who take the course together.[<reflink idref="bib7" id="ref25">7</reflink>] Unfortunately, only two cohorts are available (fall 2021 and fall 2022), which means standard errors cannot be clustered at the cohort level. Moreover, identifying the effect of feedback by comparing fall 2021 to fall 2022 assumes no unobserved cohort effects. One could argue, however, that the fall 2021 cohort experienced a learning loss in virtual principles courses due to the COVID-19 pandemic. If such an unobserved cohort effect is invariant across assessments or time, a DiD framework will account for it.</p> <hd id="AN0181888593-3">Empirical model</hd> <p>To identify the impact of supportive rank feedback emails on student</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>i</mi></mrow></math> </ephtml> 's score on assessment</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>j</mi></mrow></math> </ephtml> , a DiD regression is specified as follows:</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mrow><mtext>Score</mtext></mrow></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub><mo>=</mo><msub><mrow><mi>α</mi></mrow><mi>j</mi></msub><mo>+</mo><msub><mrow><mi>η</mi></mrow><mi>i</mi></msub><mo>+</mo><msubsup><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>X</mi></mstyle></mrow><mi>i</mi><mo>′</mo></msubsup><msub><mrow><mi mathvariant="bold-italic">β</mi></mrow><mi>j</mi></msub><mo>+</mo><msub><mrow><mi>γ</mi></mrow><mi>j</mi></msub><msub><mrow><mrow><mtext>Feedback</mtext></mrow></mrow><mi>i</mi></msub><mo>+</mo><msub><mrow><mi>ε</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub></mrow></math> </ephtml> (<reflink idref="bib1" id="ref26">1</reflink>)</p> <p>where the fixed effects</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>α</mi></mrow><mrow><mi>j</mi></mrow></msub></math> </ephtml> adjust for differences in assessment topic, timing, and type (homework or exam), as well as any fall semester trends (e.g., holidays). Equation (<reflink idref="bib1" id="ref27">1</reflink>) accounts for student characteristics in two ways. First, individual fixed effects</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>η</mi></mrow><mrow><mi>i</mi></mrow></msub></math> </ephtml> adjust for latent ability and between-cohort learning differences, among other assessment-invariant traits. Second, the extent to which observable inputs</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi mathvariant="bold">X</mi></mrow><mrow><mi>i</mi></mrow></msub></math> </ephtml> are productive,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mi mathvariant="bold-italic">β</mi></mrow><mi>j</mi></msub></mrow></math> </ephtml> , depends on assessment</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>j</mi></mrow></math> </ephtml> . For example, high-GPA students may perform better at difficult topics or on earlier assessments when other students are less concerned about their grades. The treatment indicator,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mtext mathvariant="italic">Feedback</mtext></mrow><mrow><mi>i</mi></mrow></msub></math> </ephtml> , takes a value of 1 if student</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>i</mi></mrow></math> </ephtml> was sent emails providing supportive rank feedback, and 0 otherwise.</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mtext mathvariant="italic">Feedback</mtext></mrow><mrow><mi>i</mi></mrow></msub></math> </ephtml> has coefficient</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>γ</mi></mrow><mrow><mi>j</mi></mrow></msub></math> </ephtml> , the average treatment effect (ATE), which varies by assessment relative to the first assessment. Conceivably,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>γ</mi></mrow><mrow><mi>j</mi></mrow></msub></math> </ephtml> also varies across the grade distribution in that feedback may affect low- and high-performing students differently. To check this, equation (<reflink idref="bib1" id="ref28">1</reflink>) is also estimated as a quantile regression at three quantiles of the conditional distribution of scores. The 0.25 and 0.75 quantiles are used to compare low- and high-performing students, whereas the 0.5 quantile checks the influence of outliers. Given the intervention setting and the control variables,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mtext mathvariant="italic">Feedback</mtext></mrow><mrow><mi>i</mi></mrow></msub></math> </ephtml> is taken as exogenous to the error term</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>ε</mi></mrow><mrow><mtext mathvariant="italic">ij</mtext></mrow></msub></math> </ephtml> , and thus, estimates are unbiased and consistent. Inferences on these estimates are conducted using student-level clustered standard errors because feedback status varies across only two cohorts.</p> <p>If feedback emails matter to the intervention course, it may also affect the transfer of inputs to other courses. Perhaps feedback diverts effort from other courses or causes students to adopt better study habits. Exploring these spillover effects involves specifying this DiD regression:</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mrow><mrow><mtext>GPA</mtext></mrow></mrow><mrow><mtext mathvariant="italic">fall</mtext><mo>,</mo><mi>i</mi></mrow></msub><mo>−</mo><msub><mrow><mrow><mtext>GPA</mtext></mrow></mrow><mrow><mtext mathvariant="italic">spr</mtext><mo>,</mo><mi>i</mi></mrow></msub><mo>=</mo><mi>α</mi><mo>+</mo><msubsup><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>X</mi></mstyle></mrow><mi>i</mi><mo>′</mo></msubsup><mi mathvariant="bold-italic">β</mi><mo>+</mo><mi>γ</mi><msub><mrow><mrow><mtext>Feedback</mtext></mrow></mrow><mi>i</mi></msub><mo>+</mo><msub><mrow><mi>ε</mi></mrow><mi>i</mi></msub></mrow></math> </ephtml> (<reflink idref="bib2" id="ref29">2</reflink>)</p> <p>where the outcome variable is the difference between GPA in the fall and spring semesters for student</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>i</mi></mrow></math> </ephtml> . As such, the equation adjusts for all time-invariant unobserved effects and allows observed covariates to affect the change in GPA. The effect of feedback,</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi>γ</mi></mrow></math> </ephtml> , is estimated when the fall semester GPA includes and excludes the grade for the intervention course. Here, the estimated standard errors are robust to heteroskedasticity.</p> <hd id="AN0181888593-4">Data</hd> <p>Assessment scores and characteristics are available for 32 students (sent feedback) in fall 2022 and 35 students (with no feedback) in fall 2021.[<reflink idref="bib8" id="ref30">8</reflink>] The analysis is based on eight assessment scores per student except for two students who each missed one assessment due to illness, resulting in a total of 534 student-assessment observations.[<reflink idref="bib9" id="ref31">9</reflink>] Both cohorts span the full grading distribution, ranging from some students failing to many students receiving As. Figure 3 further compares the distributions of scores by assessment type and feedback status.</p> <p>Graph: Figure 3. Distribution of average homework and exam scores by feedback status.</p> <p>The data on student characteristics come from administrative records and student surveys. Besides typical demographics, there is information on students' underserved status (i.e., Pell-eligible, first-generation, and veterans), current number of credits, and GPA (cumulative, current semester, and previous semester). Table 1 provides the complete list of baseline covariates and summary statistics by feedback status. Students are typically 19.8 years old, with 57 percent being males. While some are minority students (13%) or transfer students (9%), many (40%) identify as underserved. Most students (83%) are sophomores or juniors, likely from the department offering the intervention course. At the start of the fall semester, students averaged 14.7 credit hours and a cumulative GPA of 3.20. There are noteworthy differences between feedback status in terms of gender and academic department, with only the latter being statistically significant. At any rate, the regressions adjust for all covariates.</p> <p>Table 1. Baseline student characteristics.</p> <p> <ephtml> <table><thead><tr><td /><td>Sent feedback</td></tr><tr><td /><td>Overall</td><td>S.D.</td><td>No</td><td>Yes</td></tr></thead><tbody valign="top"><tr><td>Age</td><td char=".">19.81</td><td>(0.909)</td><td char=".">19.86</td><td char=".">19.75</td></tr><tr><td>Male</td><td char=".">0.567</td><td>(0.499)</td><td char=".">0.486</td><td char=".">0.656</td></tr><tr><td>Minority</td><td char=".">0.134</td><td>(0.344)</td><td char=".">0.171</td><td char=".">0.094</td></tr><tr><td>Underserved<sup>a</sup></td><td char=".">0.403</td><td>(0.494)</td><td char=".">0.371</td><td char=".">0.438</td></tr><tr><td>Transferred</td><td char=".">0.090</td><td>(0.288)</td><td char=".">0.114</td><td char=".">0.063</td></tr><tr><td>Sophomore</td><td char=".">0.373</td><td>(0.487)</td><td char=".">0.314</td><td char=".">0.434</td></tr><tr><td>Junior</td><td char=".">0.403</td><td>(0.494)</td><td char=".">0.457</td><td char=".">0.343</td></tr><tr><td>Senior</td><td char=".">0.224</td><td>(0.420)</td><td char=".">0.229</td><td char=".">0.219</td></tr><tr><td>Offering dept<sup>b</sup></td><td char=".">0.776</td><td>(0.420)</td><td char=".">0.657</td><td char=".">0.906<sup>c</sup></td></tr><tr><td>No. of credits</td><td char=".">14.72</td><td>(2.088)</td><td char=".">14.74</td><td char=".">14.69</td></tr><tr><td>Cumulative GPA<sup>d</sup></td><td char=".">3.198</td><td>(0.531)</td><td char=".">3.265</td><td char=".">3.125</td></tr><tr><td>No. of students</td><td char=".">67</td><td char=".">35</td><td char=".">32</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note</emph>: Values are mean, with standard deviation in parentheses. <sups>a</sups>Underserved students include those who are Pell-eligible, first-generation, and veterans. <sups>b</sups>Offering Dept. is the department offering the intervention course. <sups>c</sups>Indicates a 10% significant difference between students who received feedback and those who did not. <sups>d</sups>Cumulative GPA excludes the current semester.</p> <hd id="AN0181888593-5">Results</hd> <p>Table 2 presents the estimated effects of feedback emails on scores for each assessment.[<reflink idref="bib10" id="ref32">10</reflink>] The assessments are sequenced in their assigned order, each following a feedback email. Average effects for homework and exams are computed separately based on the assessment-specific effects. Results are reported for the typical student (i.e., the ATE and the 0.5 quantile) and those at the ends of the grade distribution (i.e., 0.25 and 0.75 quantiles). Based on the estimated ATE from equation (<reflink idref="bib1" id="ref33">1</reflink>) (with</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msup><mrow><mi>R</mi></mrow><mrow><mn>2</mn></mrow></msup></math> </ephtml> = 57%), students with feedback emails score, on average, 20.1 percentage points (pp) higher on homework #2 than their peers with no feedback emails. They continue to perform better on midterm exam #1 by 7.8 pp and subsequently on homework #3 by 10.2 pp. Gains appear short-lived, however, as the estimated impacts on homework #4 and later assessments are relatively small and statistically insignificant. The average of the estimated effects for all homework is 7.4 pp, compared to a nonsignificant 3.9 pp for all exams. Given that exams are weighted more heavily than homework (a 4:1 ratio), the effect on overall course performance would closely reflect the average exam effect. Smith et al. ([<reflink idref="bib10" id="ref34">10</reflink>]) also find diminishing effects and small gains (about 4 pp) in sports economics and fundamentals of microeconomics when they send grade nudges—messages about an assignment's impact on one's final grade.</p> <p>Table 2. Estimated impact of feedback emails.</p> <p> <ephtml> <table><thead><tr><td /><td>Score quartiles</td></tr><tr><td>Post-feedback assessment</td><td>ATE</td><td char=".">0.25</td><td char=".">0.50</td><td char=".">0.75</td></tr></thead><tbody valign="top"><tr><td>Homework #2</td><td char=".">0.201<xref ref-type="table-fn" rid="tfn2">***</xref></td><td char=".">0.245<xref ref-type="table-fn" rid="tfn2">***</xref></td><td char=".">0.193<xref ref-type="table-fn" rid="tfn2">***</xref></td><td char=".">0.151<xref ref-type="table-fn" rid="tfn2">***</xref></td></tr><tr><td>(0.056)</td><td>(0.073)</td><td>(0.053)</td><td>(0.043)</td></tr><tr><td>Midterm exam #1</td><td char=".">0.078<xref ref-type="table-fn" rid="tfn2">**</xref></td><td char=".">0.090<xref ref-type="table-fn" rid="tfn2">*</xref></td><td char=".">0.076<xref ref-type="table-fn" rid="tfn2">**</xref></td><td char=".">0.066<xref ref-type="table-fn" rid="tfn2">*</xref></td></tr><tr><td>(0.039)</td><td>(0.052)</td><td>(0.038)</td><td>(0.039)</td></tr><tr><td>Homework #3</td><td char=".">0.102<xref ref-type="table-fn" rid="tfn2">**</xref></td><td char=".">0.120<xref ref-type="table-fn" rid="tfn2">**</xref></td><td char=".">0.099<xref ref-type="table-fn" rid="tfn2">**</xref></td><td char=".">0.083<xref ref-type="table-fn" rid="tfn2">*</xref></td></tr><tr><td>(0.049)</td><td>(0.055)</td><td>(0.048)</td><td>(0.049)</td></tr><tr><td>Homework #4</td><td char=".">0.033</td><td char=".">0.051</td><td char=".">0.030</td><td char=".">0.014</td></tr><tr><td>(0.055)</td><td>(0.065)</td><td>(0.053)</td><td>(0.050)</td></tr><tr><td>Midterm exam #2</td><td char=".">0.013</td><td char=".">0.022</td><td char=".">0.012</td><td char=".">0.005</td></tr><tr><td>(0.046)</td><td>(0.060)</td><td>(0.044)</td><td>(0.041)</td></tr><tr><td>Homework #5</td><td char=".">−0.037</td><td char=".">−0.056</td><td char=".">−0.034</td><td char=".">−0.018</td></tr><tr><td>(0.040)</td><td>(0.054)</td><td>(0.038)</td><td>(0.037)</td></tr><tr><td>Final exam</td><td char=".">0.026</td><td char=".">0.037</td><td char=".">0.025</td><td char=".">0.015</td></tr><tr><td>(0.042)</td><td>(0.049)</td><td>(0.041)</td><td>(0.044)</td></tr><tr><td>Avg. homework effects</td><td char=".">0.074<xref ref-type="table-fn" rid="tfn2">**</xref></td><td char=".">0.091<xref ref-type="table-fn" rid="tfn2">**</xref></td><td char=".">0.072<xref ref-type="table-fn" rid="tfn2">**</xref></td><td char=".">0.057</td></tr><tr><td>(0.037)</td><td>(0.042)</td><td>(0.036)</td><td>(0.035)</td></tr><tr><td>Avg. exam effects</td><td char=".">0.039</td><td char=".">0.050</td><td char=".">0.038</td><td char=".">0.029</td></tr><tr><td>(0.034)</td><td>(0.042)</td><td>(0.033)</td><td>(0.033)</td></tr></tbody></table> </ephtml> </p> <p>2 <emph>Notes</emph>: Obs. = 534. ATE is the average treatment effect (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><msub><mrow><mi>γ</mi></mrow><mrow><mi>j</mi></mrow></msub></math> </ephtml> ) from estimating equation (<reflink idref="bib1" id="ref35">1</reflink>). Other columns report distributional effects at quartiles when equation (<reflink idref="bib1" id="ref36">1</reflink>) is estimated as a quantile regression. Control variables are those in table 1, as well as age squared. Standard errors in parentheses are clustered at the individual level. *</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mi>p</mi><mo><</mo><mtext>0.10</mtext></math> </ephtml> ; **</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mi>p</mi><mo><</mo><mtext>0.05;</mtext><mi /></math> </ephtml> ***</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mi>p</mi><mo><</mo><mtext>0.01</mtext></math> </ephtml> </p> <p>Despite the skewed outcomes (see figure 3), the median (0.5 quantile) estimates are similar to ATE, showcasing the robustness of the findings to outliers. Regarding low (0.25 quantile) and high (0.75 quantile) achievers, it appears the former benefits somewhat more from supportive feedback on relative performance, particularly for homework. The gains across all homework average 9.1 pp for low achievers but decline to a nonsignificant 5.7 pp for the best-performing students. Still, at least in the first half of the semester, students perform better across the grade distribution. Dobrescu et al. ([<reflink idref="bib8" id="ref37">8</reflink>]) observe a similar pattern in principles of microeconomics, implying that students hold competitive preferences.</p> <p>Finally, the impact of feedback on student performance in other courses in the same semester is examined. The previous findings suggest that students' efforts in the intervention course quickly dissipated during the semester. Thus, one would expect feedback to divert little effort from other courses or do little to improve study habits. To check this, equation (<reflink idref="bib2" id="ref38">2</reflink>) is estimated when the fall semester GPA excludes the intervention course grade, with results indicating that the effect of feedback is −0.049 (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mi>s</mi><mo>.</mo><mi>e</mi><mo>.</mo></math> </ephtml> = 0.139). Taking this point estimate at face value, it seems feedback crowded out effort in concurrent courses, but consistent with Dobrescu et al. it is not significant. For comparison, when the fall semester GPA includes the intervention course, the estimated impact is −0.008 (</p> <p>Graph</p> <p> <ephtml> <math display="inline" xmlns="http://www.w3.org/1998/Math/MathML"><mi>s</mi><mo>.</mo><mi>e</mi><mo>.</mo></math> </ephtml> = 0.138). This increase in effect size coincides with the relatively small gains in overall course performance alluded to earlier.</p> <hd id="AN0181888593-6">Conclusion</hd> <p>Students may invest effort at a level instructors deem suboptimal for success in demanding, upper-level courses. Perhaps they make such investment choices uncertain of their ability or how their productivity compares with others. Therefore, we exploit an intervention that provides a series of personalized feedback emails that share class rank and advice for success with students in an intermediate-level microeconomics course.</p> <p>Evidence shows that scores increased on three consecutive assessments (two homework and a midterm exam) in the first half of the semester after three feedback emails. However, subsequent emails appear ineffective, and there are no spillover effects on other courses taken concurrently. Another finding is that gains from early in the semester were realized across the entire grade distribution, suggesting students hold competitive preferences—they choose effort to outdo others. Given this, there are two possible reasons for the diminishing effects during the semester. First, feedback might boost competition early in the semester when the informational gap is largest, and students would otherwise be less concerned about their grades. Second, it may partly reflect the intervention design, as it provides feedback to <emph>all</emph> students in a class—a realistic implementation of a feedback scheme in an actual classroom setting. Thus, if students of any ability are initially performing better, their relative class positions will hardly improve, and they may eventually ignore feedback.</p> <p>Our study supports feedback email as a low-cost, light-touch tool that instructors can use to induce effort at least early in a semester. The small gain in overall course performance is statistically insignificant, but this may be due to the relatively small size of an intermediate-level course yielding a lack of statistical power.</p> <hd id="AN0181888593-7">Disclosure statement</hd> <p>No potential conflict of interest was reported by the author(s).</p> <ref id="AN0181888593-8"> <title> References </title> <blist> <bibl id="bib1" idref="ref10" type="bt">1</bibl> <bibtext> Azmat, G., M. Bagues, A. Cabrales, N. Iriberri. 2019. What you don't know... can't hurt you? A natural field experiment on relative performance feedback in higher education. Management Science 65 (8): 3714 – 36. doi: 10.1287/mnsc.2018.3131.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref1" type="bt">2</bibl> <bibtext> Azmat, G., and N. Iriberri. 2010. The importance of relative performance feedback information: Evidence from a natural experiment using high school students. Journal of Public Economics 94 (7–8): 435 – 52. doi: 10.1016/j.jpubeco.2010.04.001.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref11" type="bt">3</bibl> <bibtext> Brade, R., O. Himmler, and R. Jäckle. 2022. Relative performance feedback and the effects of being above average—Field experiment and replication. Economics of Education Review 89 (August): 102268. doi: 10.1016/j.econedurev.2022.102268.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref22" type="bt">4</bibl> <bibtext> Cameron, A. C., J. B. Gelbach, and D. L. Miller. 2008. Bootstrap-based improvements for inference with clustered errors. Review of Economics and Statistics 90 (3): 414 – 27. doi: 10.1162/rest.90.3.414.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref12" type="bt">5</bibl> <bibtext> Carrell, S. E., and M. Kurlaender. 2023. My professor cares: Experimental evidence on the role of faculty engagement. American Economic Journal: Economic Policy 15 (4): 113 – 41. doi: 10.1257/pol.20210699.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref3" type="bt">6</bibl> <bibtext> Compte, O., and A. Postlewaite. 2004. Confidence-enhanced performance. American Economic Review 94 (5): 1536 – 57. doi: 10.1257/0002828043052204.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref6" type="bt">7</bibl> <bibtext> Damgaard, M. T., and H. S. Nielsen. 2018. Nudging in education. Economics of Education Review 64 (June): 313 – 42. doi: 10.1016/j.econedurev.2018.03.008.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref2" type="bt">8</bibl> <bibtext> Dobrescu, L. I., M. Faravelli, R. Megalokonomou, and A. Motta. 2021. Relative performance feedback in education: Evidence from a randomised controlled trial. The Economic Journal 131 (640): 3145 – 81. doi: 10.1093/ej/ueab043.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref8" type="bt">9</bibl> <bibtext> Kajitani, S., K. Morimoto, and S. Suzuki. 2020. Information feedback in relative grading: Evidence from a field experiment. PloS One 15 (4): e0231548. doi: 10.1371/journal.pone.0231548.</bibtext> </blist> <blist> <bibtext> Smith, B. O., D. R. White, P. C. Kuzyk, and J. E. Tierney. 2018. Improved grade outcomes with an e-mailed "grade nudge. Journal of Economic Education 49 (1): 1 – 7. doi: 10.1080/00220485.2017.1397570.</bibtext> </blist> <blist> <bibtext> Tran, A., and R. Zeckhauser. 2012. Rank as an inherent incentive: Evidence from a field experiment. Journal of Public Economics 96 (9–10): 645 – 50. doi: 10.1016/j.jpubeco.2012.05.004.</bibtext> </blist> </ref> <ref id="AN0181888593-9"> <title> Footnotes </title> <blist> <bibtext> This study was submitted to the university's Institutional Review Board (IRB #: 22-868), which approved and exempted it from further review. Consistent with IRB standards, data were de-identified and students were asked for consent to be sent feedback emails. They were told only that the emails allow the instructor to provide additional feedback.</bibtext> </blist> <blist> <bibtext> The university uses the +/− suffix with letter grading (except for A+) on a 4.0 scale. An "A" (3.7–4.0) is for students who earned above 90; "B" (2.7–3.3) if scored 80–89; "C" (1.7–2.3) for 70–79; "D" (0.7–1.3) for 60–69; and "F" (0.0) is below 60.</bibtext> </blist> <blist> <bibtext> Open-book quizzes were also assigned to reinforce concepts during lectures, but no feedback messages were sent for this assessment. Students yielded high scores (averaging 4.7% out of 5%), prompting little or no need for feedback.</bibtext> </blist> <blist> <bibtext> The course was taught by a different instructor in spring 2022.</bibtext> </blist> <blist> <bibtext> The course was assigned one Teaching Assistant (TA) who took the same course in the previous semester and received an "A." While TAs differed, anecdotally, their services were underutilized and even more so in fall 2022.</bibtext> </blist> <blist> <bibtext> Cameron, Gelbach, and Miller ([4]) show that for as few as six clusters, correct inferences in DiD models can be achieved with the residual or wild cluster bootstrap-t method.</bibtext> </blist> <blist> <bibtext> Because the intervention course has one section per semester, the clusters cannot represent multiple sections.</bibtext> </blist> <blist> <bibtext> Seven students (four in fall 2021 and three in fall 2022) who dropped the course before the first feedback email were excluded from the data. However, two fall 2022 students who dropped out after midterm #2 were retained. To avoid selection bias, their homework #5 and final exam scores were imputed using the 25th percentile because they tended to perform below average (albeit not the lowest scores)—their expected course grades were C + and D−. One student who repeated the course and two students with missing GPA data were also excluded.</bibtext> </blist> <blist> <bibtext> Because the two students each missed one assessment, they were not dropped from the data. Wherever their scores are known (which is 7 out of 8 assessments), they are used to estimate the assessment-specific effects; otherwise, their scores are treated as missing.</bibtext> </blist> <blist> <bibtext> Given the many coefficients for all the covariates (because they vary by assessment), their estimates are not reported but are available upon request.</bibtext> </blist> </ref> <aug> <p>By Chanita C. Holmes and Marlon R. Tracey</p> <p>Reported by Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib11" firstref="ref7"></nolink> <nolink nlid="nl2" bibid="bib10" firstref="ref32"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1455827
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Does Supportive Feedback on Class Rank Improve Scores for Intermediate-Level Microeconomics?
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Chanita+C%2E+Holmes%22">Chanita C. Holmes</searchLink><br /><searchLink fieldCode="AR" term="%22Marlon+R%2E+Tracey%22">Marlon R. Tracey</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Economic+Education%22"><i>Journal of Economic Education</i></searchLink>. 2025 56(1):22-29.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 8
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Class+Rank%22">Class Rank</searchLink><br /><searchLink fieldCode="DE" term="%22Economics+Education%22">Economics Education</searchLink><br /><searchLink fieldCode="DE" term="%22Grades+%28Scholastic%29%22">Grades (Scholastic)</searchLink><br /><searchLink fieldCode="DE" term="%22Advanced+Courses%22">Advanced Courses</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Improvement%22">Student Improvement</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Practices%22">Educational Practices</searchLink><br /><searchLink fieldCode="DE" term="%22Microeconomics%22">Microeconomics</searchLink><br /><searchLink fieldCode="DE" term="%22Homework%22">Homework</searchLink><br /><searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22State+Universities%22">State Universities</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+Mail%22">Electronic Mail</searchLink><br /><searchLink fieldCode="DE" term="%22Tests%22">Tests</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Virginia%22">Virginia</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/00220485.2024.2399591
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0022-0485<br />2152-4068
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Instructors may use low-cost, light-touch strategies to help students achieve optimal effort in demanding upper-level courses. The authors of this study exploit an intervention that provides a series of personalized feedback emails to students about their relative performance, which is tied to approving messages or tips that encourage improvement. The authors analyze the efficacy of this supportive rank feedback in an intermediate-level microeconomics theory course, given the challenges it poses to students. Evidence shows gains from feedback across the grade distribution, but they are short-lived and decline toward zero as the course progresses.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1455827
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1455827
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/00220485.2024.2399591
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 8
        StartPage: 22
    Subjects:
      – SubjectFull: Class Rank
        Type: general
      – SubjectFull: Economics Education
        Type: general
      – SubjectFull: Grades (Scholastic)
        Type: general
      – SubjectFull: Advanced Courses
        Type: general
      – SubjectFull: Intervention
        Type: general
      – SubjectFull: Student Improvement
        Type: general
      – SubjectFull: Educational Practices
        Type: general
      – SubjectFull: Microeconomics
        Type: general
      – SubjectFull: Homework
        Type: general
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: State Universities
        Type: general
      – SubjectFull: Electronic Mail
        Type: general
      – SubjectFull: Tests
        Type: general
      – SubjectFull: Virginia
        Type: general
    Titles:
      – TitleFull: Does Supportive Feedback on Class Rank Improve Scores for Intermediate-Level Microeconomics?
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Chanita C. Holmes
      – PersonEntity:
          Name:
            NameFull: Marlon R. Tracey
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0022-0485
            – Type: issn-electronic
              Value: 2152-4068
          Numbering:
            – Type: volume
              Value: 56
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Journal of Economic Education
              Type: main
ResultId 1