View in EDS HTML Full Text PDF Full Text

Does Students' Evaluation of Teaching Improve Teaching Quality? Improvement versus the Reversal Effect

Saved in:
Bibliographic Details
Title: Does Students' Evaluation of Teaching Improve Teaching Quality? Improvement versus the Reversal Effect
Language: English
Authors: Chen, Yanyan (ORCID 0000-0002-8892-2012)
Source: Assessment & Evaluation in Higher Education. 2023 48(8):1195-1207.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Page Count: 13
Publication Date: 2023
Document Type: Journal Articles
Reports - Research
Tests/Questionnaires
Education Level: Higher Education
Postsecondary Education
Descriptors: Foreign Countries, Universities, College Faculty, Student Evaluation of Teacher Performance, Teacher Effectiveness, Teacher Improvement, Shift Studies, Teacher Response, Educational Quality, Teacher Evaluation, Evaluation Problems
Geographic Terms: China
DOI: 10.1080/02602938.2023.2177252
ISSN: 0260-2938
1469-297X
Abstract: Using students' assessments of teaching at a top university in China from 2016 to 2021, this study examines whether evaluation of teaching improves teaching quality. Given the many doubts about the validity of students' evaluation of teaching, this study adopts a methodology to distinguish teaching quality improvement from the reversal effect. It shows that instructors with a poor (good) ranking in the previous evaluation are more likely to receive an improved (decreased) ranking in the current evaluation. This reversal effect is more pronounced among the instructors who are associate/assistant professors, younger and female. This study provides evidence that supports the reversal effect rather than the improvement.
Abstractor: As Provided
Entry Date: 2023
Accession Number: EJ1402755
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEOMKkic8sQTJPSRIkMDmj0AAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDC0IhhqkwptMo7XuUgIBEICBm44_gLA6P66uvR1SY28FNxTDkboGgIIyAz0hoYtQZPakniVm9O07ALmwcU5u2GKSGpTWDO9KqQdSfKr8GRWPSveER0gUmrpQfxinKbpe4bxNCuPahRDEuZs5SBI4ZlB5I2_RBt4k6hjCqvJobyzT2GNmddSNsdP6NMlRUSflhaipQwE2BZ_9Q8pnWAv1RechL-_BVbk1SdmqaFGJ
Text:
  Availability: 1
  Value: <anid>AN0173779477;eva01dec.23;2023Nov24.05:14;v2.2.500</anid> <title id="AN0173779477-1">Does students' evaluation of teaching improve teaching quality? Improvement versus the reversal effect </title> <p>Using students' assessments of teaching at a top university in China from 2016 to 2021, this study examines whether evaluation of teaching improves teaching quality. Given the many doubts about the validity of students' evaluation of teaching, this study adopts a methodology to distinguish teaching quality improvement from the reversal effect. It shows that instructors with a poor (good) ranking in the previous evaluation are more likely to receive an improved (decreased) ranking in the current evaluation. This reversal effect is more pronounced among the instructors who are associate/assistant professors, younger and female. This study provides evidence that supports the reversal effect rather than the improvement.</p> <p>Keywords: Student evaluation of teaching; teaching quality improvement; reversal effect</p> <hd id="AN0173779477-2">Introduction</hd> <p>Student evaluation of teaching (hereafter referred to as SET) is a survey universities conduct to obtain students' feedback about their courses. Students participate in the teaching process and are the recipients of teaching effectiveness, which makes it rational to let students evaluate teaching quality. SET is widely used in course design and improvement, tenure and promotion decisions, and merit raises (Blair and Noel [<reflink idref="bib3" id="ref1">3</reflink>]; Stroebe [<reflink idref="bib40" id="ref2">40</reflink>]; Park and Cho [<reflink idref="bib31" id="ref3">31</reflink>]). Views are mixed regarding its validity, with evidence for and against its use in universities. Notably, as indicated in two literature reviews (Wallace, Lewis, and Allen [<reflink idref="bib44" id="ref4">44</reflink>]; Heffernan [<reflink idref="bib20" id="ref5">20</reflink>]), opponents' arguments have gradually come to dominate in the past decade. There are four lines of criticism of SET: poor reliability; low effectiveness; noise caused by non-teaching factors; and grade inflation.</p> <p>One of the primary purposes of SET is to improve teaching quality (Palmer [<reflink idref="bib30" id="ref6">30</reflink>]; Hammonds et al. [<reflink idref="bib19" id="ref7">19</reflink>]). Despite the extensive research into the weaknesses of SET, few studies have directly examined whether SET improves teaching quality in universities. If a poor evaluation in the previous period leads to a better appraisal in the current evaluation, it may be evidence that SET improves teaching quality. However, the abovementioned shortcomings of SET may cause a "random walk" in SET scores. The reversal effect emerges as a random walk – an instructor with a poor (good) evaluation in the previous period is more likely to have a better (worse) evaluation in the current period. Hence, an instructor's improved evaluation does not necessarily indicate that his/her teaching quality has improved. It could be a reversal effect in a random walk.</p> <p>This study adopts a new methodology to distinguish teaching quality improvement from the reversal effect, and provides robust evidence of the effect of SET on teaching quality improvement. This study is the first to examine whether SET improves teaching quality. It also sheds light on research of SET by showing that weaknesses in SET lead to its failure to improve teaching quality.</p> <hd id="AN0173779477-3">SET and teaching quality improvement</hd> <p></p> <hd id="AN0173779477-4">Improving teaching quality</hd> <p>SET is generally used as an important evaluation indicator and guideline in higher education (Park and Cho [<reflink idref="bib31" id="ref8">31</reflink>]). Since many universities are using student evaluations as part of their performance review system, it would be imprudent for instructors to completely ignore the potential of SET as an agent of change (Blair and Noel [<reflink idref="bib3" id="ref9">3</reflink>]). Despite doubts about the reliability and effectiveness of SET, it still offers valuable insight into the student experience (Zhao and Gallant [<reflink idref="bib46" id="ref10">46</reflink>]), and it tends to be a valid indicator of teaching performance (Marsh and Bailey [<reflink idref="bib26" id="ref11">26</reflink>]; Hellman [<reflink idref="bib21" id="ref12">21</reflink>]). Beyond questions about its validity, the ubiquity of SET offers a level of authority.</p> <p>In scrutinizing students' feedback to instructors, Ginns, Prosser, and Barrie ([<reflink idref="bib17" id="ref13">17</reflink>]) explicitly examine the factors that may influence the level and quality of response. They find five factors that predict students' overall levels of satisfaction: good teaching, clear goals, appropriate assessments, appropriate workload, and skills in helping students develop their ability to plan their work. Richardson, Slater, and Wilson ([<reflink idref="bib35" id="ref14">35</reflink>]) find that teaching skills and teacher support have the highest correlation with overall satisfaction.</p> <p>Many universities use student evaluation systems as a way of highlighting course and lecturer strengths and areas for improvement (Blair and Noel [<reflink idref="bib3" id="ref15">3</reflink>]). Cook-Sather ([<reflink idref="bib11" id="ref16">11</reflink>]) suggests that capitalizing on student feedback can benefit instructors' professional development. The literature has indicated that SET is a valuable tool for instructors to improve their teaching quality.</p> <hd id="AN0173779477-5">Failure to improve teaching quality</hd> <p>One of the primary purposes of SET is to improve teaching quality (Palmer [<reflink idref="bib30" id="ref17">30</reflink>]; Hammonds et al. [<reflink idref="bib19" id="ref18">19</reflink>]). However, evidence suggests that SET has drifted far from that purpose in the past decade (Stroebe [<reflink idref="bib40" id="ref19">40</reflink>]). Research on the shortcomings of SET can be summarized in four categories: poor reliability; low effectiveness; the influence of factors unrelated to teaching quality; and grade inflation.</p> <p>Reliability is a fundamental criterion of SET and a precondition for its validity as a teaching quality indicator. The most common measure of inter-rater reliability for SET is the intra-class correlation, defined as the correlation between assessments of randomly determined pairs of students evaluating the same course or instructor. It could be regarded as the proportion of the total variance of student evaluations that can be explained by courses or instructors. The variance component of courses or instructors is target variance, and reliability is maximized when maximizing the proportion of target variance (Feistauer and Richter [<reflink idref="bib14" id="ref20">14</reflink>]). Several studies have shown that nearly 25% of the systematic variance in evaluations is due to students, 25% is due to courses or instructors, and 50% remains unexplained (Spooren [<reflink idref="bib36" id="ref21">36</reflink>]; Rantanen [<reflink idref="bib33" id="ref22">33</reflink>]; Feistauer and Richter [<reflink idref="bib14" id="ref23">14</reflink>]). The systematic influence of student characteristics is almost as strong as the effects of course or instructor characteristics, indicating that the low reliability generated by SET is not a sufficient foundation for establishing validity.</p> <p>A valid measure of teaching quality should be associated with students' learning (Otto, Sanford, and Ross [<reflink idref="bib29" id="ref24">29</reflink>]). Learning is a gain in performance related to the purpose of a course (Clayson [<reflink idref="bib10" id="ref25">10</reflink>]). Recent studies have found no evidence supporting the belief that students learn more from instructors who receive higher SET ratings (Galbraith, Merrill, and Kline [<reflink idref="bib16" id="ref26">16</reflink>]; Uttl, White, and Gonzalez [<reflink idref="bib42" id="ref27">42</reflink>]; Carpenter, Witherby, and Tauber [<reflink idref="bib7" id="ref28">7</reflink>]). In addition, there appears to be a negative relationship between the student ratings of a course and student performance in subsequent courses (Carrell and West [<reflink idref="bib8" id="ref29">8</reflink>]; Braga, Paccagnella, and Pellizzari [<reflink idref="bib6" id="ref30">6</reflink>]; Stroebe [<reflink idref="bib39" id="ref31">39</reflink>]).</p> <p>Research has revealed that factors unrelated to teaching quality, such as the specific attributes of the instructor, have a remarkable effect on SET (Wallace, Lewis, and Allen [<reflink idref="bib44" id="ref32">44</reflink>]; Heffernan [<reflink idref="bib20" id="ref33">20</reflink>]). Instructors' characteristics, such as their experience, title, and reputation (Read, Rama, and Raghunandan [<reflink idref="bib34" id="ref34">34</reflink>]; Worthington [<reflink idref="bib45" id="ref35">45</reflink>]), age (Ragan and Walia [<reflink idref="bib32" id="ref36">32</reflink>]; Stonebraker and Stone [<reflink idref="bib37" id="ref37">37</reflink>]), gender (Boring [<reflink idref="bib4" id="ref38">4</reflink>]; Fan et al. [<reflink idref="bib13" id="ref39">13</reflink>]; Valencia [<reflink idref="bib43" id="ref40">43</reflink>]), perceived competence (Bavishi, Hebl, and Madera [<reflink idref="bib1" id="ref41">1</reflink>]; Storage et al. [<reflink idref="bib38" id="ref42">38</reflink>]), likability (Hessler et al. [<reflink idref="bib22" id="ref43">22</reflink>]; Clayson [<reflink idref="bib10" id="ref44">10</reflink>]), physical attractiveness or appearance (Read, Rama, and Raghunandan [<reflink idref="bib34" id="ref45">34</reflink>]; Worthington [<reflink idref="bib45" id="ref46">45</reflink>]; Freng and Webber [<reflink idref="bib15" id="ref47">15</reflink>]), and race/ethnicity (Bavishi, Hebl, and Madera [<reflink idref="bib1" id="ref48">1</reflink>]; Fan et al. [<reflink idref="bib13" id="ref49">13</reflink>]), can singularly and collectively affect student perceptions and evaluations.</p> <p>SET is also influenced by grade inflation, which is also known as grading leniency (Griffin [<reflink idref="bib18" id="ref50">18</reflink>]). Students who receive high grades or expect to gain high grades if the SET is completed before the final examination give high SET scores to instructors as a reward, whereas students who receive or expect to receive low grades give low SET scores to instructors in retaliation. Many studies have found a high correlation between students' grade expectations and SET scores (Cho, Baek, and Cho [<reflink idref="bib9" id="ref51">9</reflink>]; Berezvai, Lukátsc and Molontay [<reflink idref="bib2" id="ref52">2</reflink>]; Park and Cho [<reflink idref="bib31" id="ref53">31</reflink>]), consistent with the grading leniency hypothesis. These findings raise concerns that instructors may be motivated to alter their assessment processes or give generous grades to be rewarded with high SET scores, which causes grade inflation. Stroebe ([<reflink idref="bib40" id="ref54">40</reflink>]) asserts that SET encourages grade inflation, rewards poor teaching, and punishes those who grade strictly or instruct in challenging courses.</p> <p>Given the invalidity of SET, instructors have questioned its use as an accurate measure of teaching performance in higher education. In addition, instructors' regular exposure to SET contributes to stress and adversely affects mental health and well-being (Lakeman et al. [<reflink idref="bib24" id="ref55">24</reflink>]). Hence, instructors would not be motivated to improve teaching quality if they did not believe in SET.</p> <p>With mixed arguments regarding the validity of SET, it is difficult to theoretically determine whether SET improves teaching quality. This study aims to empirically explore the effect of SET on teaching quality improvement by using SET data from a top university in China.</p> <hd id="AN0173779477-6">Moderating role of instructors' characteristics</hd> <p>Research has revealed that the specific characteristics of the instructor, such as their title, age and gender, can affect student perceptions and evaluations (Wallace, Lewis, and Allen [<reflink idref="bib44" id="ref56">44</reflink>]; Heffernan [<reflink idref="bib20" id="ref57">20</reflink>]). This study further explores the moderating role of instructors' characteristics in the relationship between SET and teaching quality improvement.</p> <hd id="AN0173779477-7">Instructors' title</hd> <p>Studies have shown that an instructor's title has a positive impact on student evaluation (Read, Rama, and Raghunandan [<reflink idref="bib34" id="ref58">34</reflink>]; Worthington [<reflink idref="bib45" id="ref59">45</reflink>]). An instructor's title is an indicator of academic achievement. A high achievement demonstrates that instructors keep up with cutting-edge knowledge in their field, which leads to positive course ratings (Ting [<reflink idref="bib41" id="ref60">41</reflink>]). High academic achievement also helps to create a positive image of the instructor. Students assume that professors are more capable of teaching than associate/assistant professors, regardless of their real teaching performance.</p> <p>However, there are numerous examples of good instructors who are not high-achieving researchers; research skills are not the same as teaching skills (Johnston [<reflink idref="bib23" id="ref61">23</reflink>]). Moreover, too much involvement in non-teaching activities may force instructors to compromise teaching quality. Hence, the bias against non-professor instructors may affect the relationship between SET and improvement in teaching quality.</p> <hd id="AN0173779477-8">Instructors' age</hd> <p>The evidence for a relationship between instructors' age and SET scores in higher education is mixed. Some studies have shown that an instructor's age negatively affects SET scores (Meshkani and Hossein [<reflink idref="bib28" id="ref62">28</reflink>]; McPherson, Jewell, and Kim [<reflink idref="bib27" id="ref63">27</reflink>]). Stonebraker and Stone ([<reflink idref="bib37" id="ref64">37</reflink>]) find that age hurts SET scores, with this effect beginning once instructors reach their mid-forties. In contrast, Ragan and Walia ([<reflink idref="bib32" id="ref65">32</reflink>]) find that new instructors tend to receive lower scores, but this disadvantage changes in a few years. Spooren ([<reflink idref="bib36" id="ref66">36</reflink>]) finds no significant effect of instructors' age in their sample.</p> <p>As younger instructors gain experience, their teaching performance should improve, but other factors eventually push their SET scores in the opposite direction (Stonebraker and Stone [<reflink idref="bib37" id="ref67">37</reflink>]). For example, the teaching prowess of older instructors might suffer from an inability to stay current in their fields, or they might be allocating relatively more time to administrative rather than classroom pursuits (McPherson, Jewell, and Kim [<reflink idref="bib27" id="ref68">27</reflink>]). Additionally, students might find it easier to connect with instructors closer to their age. Although the relationship between instructors' age and SET scores is mixed, it is still worth exploring whether instructors' age affects the relationship between SET and improvement in teaching quality.</p> <hd id="AN0173779477-9">Instructors' gender</hd> <p>Researchers have shown that instructors' gender makes a difference in SET scores and is highly prejudiced against women (Boring [<reflink idref="bib4" id="ref69">4</reflink>]; Fan et al. [<reflink idref="bib13" id="ref70">13</reflink>]; Valencia [<reflink idref="bib43" id="ref71">43</reflink>]). Some studies have found that gender biases result in large and significant variations in SET results. Fan et al. ([<reflink idref="bib13" id="ref72">13</reflink>]) show that at the extreme, women instructors received SET scores 37 percentage points lower than their male counterparts. Boring, Ottoboni, and Stark ([<reflink idref="bib5" id="ref73">5</reflink>]) demonstrate similar results. Gender biases are so strong that more effective female instructors end up in lower SET grading brackets than their less effective male counterparts.</p> <p>These findings are consistent with the theory of role congruity (Eagly and Karau [<reflink idref="bib12" id="ref74">12</reflink>]). Students expect women to behave as female gender stereotypes and men as male gender stereotypes. Students are more likely to rate male instructors as brilliant or knowledgeable; by the same token, they judge female instructors as bossy or annoying. Students perceive college teaching as a male occupation and evaluate teaching performance based on the characteristics of stereotypical male instructors (MacNell, Driscoll, and Hunt [<reflink idref="bib25" id="ref75">25</reflink>]). Given the gender bias in SET, this study further explores whether bias against female instructors affects the relationship between SET and teaching quality improvement.</p> <hd id="AN0173779477-10">Data</hd> <p>This study uses SET data submitted by undergraduate students enrolled at the College of Economics and Management, South China Agricultural University, a top Chinese university. The university requires students to evaluate each instructor of the courses selected in any given semester. The evaluation must be completed before the final examination. Instructors learn their teaching evaluation scores and ranking in the college through the educational administration system two weeks after the beginning of the following semester.</p> <p>The evaluation is completed by each student individually by logging into the university's online system. It consists of eight single-choice questions with five options (strongly agree, agree, neutral, disagree and strongly disagree). The responses are quantified and converted into points. The total points are 100, with five 10-point questions, two 15-point questions and one 20-point question. The questions are as follows:</p> <p></p> <ulist> <item> The instructor was enthusiastic and took responsibility for what was taught (10 points).</item> <p></p> <item> The instructor gave a clear explanation of the course's objectives and requirements (10 points).</item> <p></p> <item> The teaching was clear, and the key points were highlighted and well arranged (10 points).</item> <p></p> <item> The instructor integrated the discipline's cutting-edge knowledge and practice into the teaching (15 points).</item> <p></p> <item> The flexible teaching methods and practical teaching content stimulated my interest in this course (15 points).</item> <p></p> <item> The instructor motivated me to think actively by communicating and interacting with the students (10 points).</item> <p></p> <item> The instructor cared about the students' learning and gave timely feedback and guidance in response to questions (10 points).</item> <p></p> <item> I gained new knowledge and improved my relevant skills by taking this course (20 points).</item> </ulist> <p>SET data covering 11 semesters, from the autumn semester of 2016–2017 to the autumn semester of 2021–2022, were collected from the college. Because of the lagged variables in the regression, the sample consists of 10 semesters from the spring semester of 2016–2017 to the autumn semester of 2021–2022. If an instructor had two or more courses during a semester, the average score is used. Each instructor is given a score and percentile ranking in the college each semester. This study takes in 832 instructor-semester observations.</p> <hd id="AN0173779477-11">Methodology</hd> <p>If a poor evaluation in the period immediately prior prompts an increase in the subsequent period, it may be evidence that SET improves teaching quality. However, the shortcomings of SET, such as low reliability, low effectiveness, disturbance by non-teaching factors and grade inflation, may produce a random walk in students' evaluation scores. The reversal effect emerges as a random walk; that is, an instructor with a poor (good) score in the previous period is more likely to receive a better (worse) score in the current period. Hence, an instructor's increased evaluation does not necessarily indicate that the instructor improves the teaching quality. It could be a reversal effect in a random walk.</p> <p>In this study, I adopt a new design to distinguish the improvement from the reversal effect. I examine the impact of the last poor and good evaluation separately on the current evaluation change. I expect two scenarios. The first is that an instructor with a poor score in the previous period is more likely to receive a better score in the current period, whereas a last good score does not lead to a decline in the current period. This scenario indicates that SET improves teaching quality. The second is that an instructor with a poor (good) score in the previous period is more likely to have a better (worse) score in the current period. This scenario indicates the reversal effect.</p> <p>This study uses four logistic regression models. Equations (<reflink idref="bib1" id="ref76">1</reflink>) and (<reflink idref="bib2" id="ref77">2</reflink>) examine the effect of the last poor evaluation on the improvement in current evaluation. The dependent variable of Equation (<reflink idref="bib1" id="ref78">1</reflink>) is the ranking increase in the current evaluation (<emph>RANK_INCREASE</emph>), which equals 1 for a ranking increase and 0 for a ranking decrease. The key dependent variable is the indicator of poor evaluation in the previous period (<emph>RANK_BOTTOM</emph>), which equals 1 for evaluations ranked in the bottom 25% and 0 otherwise. The coefficient of <emph>RANK_BOTTOM</emph> is my focus. The control variables include instructor title (<emph>TITLE</emph>), age (<emph>AGE</emph>) and gender (<emph>GENDER</emph>). <emph>TERM</emph> indicates semester fixed effects. I add the interaction term of the variable <emph>X</emph> and <emph>RANK_BOTTOM</emph> in Equation (<reflink idref="bib2" id="ref79">2</reflink>). <emph>X</emph> stands for instructors' title (<emph>TITLE</emph>), age (<emph>AGE</emph>) and gender (<emph>GENDER</emph>). The coefficients of the interaction terms are the focus.</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi mathvariant="italic">RANK</mi><mo>_</mo><mi mathvariant="italic">INCREAS</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>=</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>0</mn></msub><mo>+</mo><msub><mrow><mi mathvariant="bold-italic">β</mi></mrow><mi mathvariant="bold-italic">1</mi></msub><mi>R</mi><mi mathvariant="italic">ANK</mi><mo>_</mo><mi mathvariant="italic">BOTTO</mi><msub><mrow><mi>M</mi></mrow><mrow><mi mathvariant="bold-italic">i,t-1</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>2</mn></msub><mi>T</mi><mi mathvariant="italic">ITL</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>3</mn></msub><mi>A</mi><mi>G</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>4</mn></msub><mi>G</mi><mi mathvariant="italic">ENDE</mi><msub><mrow><mi>R</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><mi mathvariant="italic">TER</mi><msub><mrow><mi>M</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>ε</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub></mrow></math> </ephtml> (<reflink idref="bib1" id="ref80">1</reflink>)</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi mathvariant="italic">RANK</mi><mo>_</mo><mi mathvariant="italic">INCREAS</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>=</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>0</mn></msub><mo>+</mo><msub><mrow><mi mathvariant="bold-italic">β</mi></mrow><mi mathvariant="bold-italic">1</mi></msub><mi>R</mi><mi mathvariant="italic">ANK</mi><mo>_</mo><mi mathvariant="italic">BOTTO</mi><msub><mrow><mi>M</mi></mrow><mrow><mi mathvariant="bold-italic">i,t-1</mi></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="bold-italic">β</mi></mrow><mi mathvariant="bold-italic">2</mi></msub><mi>X</mi><mo>*</mo><mi mathvariant="italic">RANK</mi><mo>_</mo><mi mathvariant="italic">BOTTO</mi><msub><mrow><mi>M</mi></mrow><mrow><mi mathvariant="bold-italic">i,t-1</mi></mrow></msub><mo>+</mo><msub><mrow><mi>β</mi></mrow><mn>3</mn></msub><mi>T</mi><mi mathvariant="italic">ITL</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>4</mn></msub><mi>A</mi><mi>G</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>5</mn></msub><mi>G</mi><mi mathvariant="italic">ENDE</mi><msub><mrow><mi>R</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><mi mathvariant="italic">TER</mi><msub><mrow><mi>M</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>ε</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub></mrow></math> </ephtml> (<reflink idref="bib2" id="ref81">2</reflink>)</p> <p>Equations (<reflink idref="bib3" id="ref82">3</reflink>) and (<reflink idref="bib4" id="ref83">4</reflink>) examine the impact of the last good evaluation on the decline of current evaluation. The dependent variable of Equation (<reflink idref="bib3" id="ref84">3</reflink>) is the ranking decrease in the current evaluation (<emph>RANK_DECREASE</emph>), which equals 1 for a ranking decrease and 0 for a ranking increase. The key dependent variable in Equation (<reflink idref="bib3" id="ref85">3</reflink>) is the indicator of a good evaluation in the previous period (<emph>RANK_TOP</emph>), which equals 1 for evaluations ranked in the top 25% and 0 otherwise. The coefficient of <emph>RANK_TOP</emph> is the focus. I add the interaction term of the variable <emph>X</emph> and <emph>RANK_TOP</emph> in Equation (<reflink idref="bib4" id="ref86">4</reflink>). <emph>X</emph> stands for instructors' title (<emph>TITLE</emph>), age (<emph>AGE</emph>) and gender (<emph>GENDER</emph>). The coefficients of the interaction terms are the focus. Table 1 provides detailed definitions of these variables.</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi mathvariant="italic">RANK</mi><mo>_</mo><mi mathvariant="italic">DECREAS</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>=</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>0</mn></msub><mo>+</mo><msub><mrow><mi mathvariant="bold-italic">β</mi></mrow><mi mathvariant="bold-italic">1</mi></msub><mi>R</mi><mi mathvariant="italic">ANK</mi><mo>_</mo><mi>T</mi><mi>O</mi><msub><mrow><mi>P</mi></mrow><mrow><mi mathvariant="bold-italic">i,t-1</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>2</mn></msub><mi>T</mi><mi mathvariant="italic">ITL</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi><mo>−</mo><mn>1</mn></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>3</mn></msub><mi>A</mi><mi>G</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>4</mn></msub><mi>G</mi><mi mathvariant="italic">ENDE</mi><msub><mrow><mi>R</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><mi mathvariant="italic">TER</mi><msub><mrow><mi>M</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>ε</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub></mrow></math> </ephtml> (<reflink idref="bib3" id="ref87">3</reflink>)</p> <p>Graph</p> <p> <ephtml> <math display="block" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi mathvariant="italic">RANK</mi><mo>_</mo><mi mathvariant="italic">DECREAS</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>=</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>0</mn></msub><mo>+</mo><msub><mrow><mi mathvariant="bold-italic">β</mi></mrow><mi mathvariant="bold-italic">1</mi></msub><mi>R</mi><mi mathvariant="italic">ANK</mi><mo>_</mo><mi>T</mi><mi>O</mi><msub><mrow><mi>P</mi></mrow><mrow><mi mathvariant="bold-italic">i,t-1</mi></mrow></msub><mo>+</mo><msub><mrow><mi mathvariant="bold-italic">β</mi></mrow><mi mathvariant="bold-italic">2</mi></msub><mi>X</mi><mo>*</mo><mi mathvariant="italic">RANK</mi><mo>_</mo><mi>T</mi><mi>O</mi><msub><mrow><mi>P</mi></mrow><mrow><mi mathvariant="bold-italic">i,t-1</mi></mrow></msub><mo>+</mo><msub><mrow><mi>β</mi></mrow><mn>3</mn></msub><mi>T</mi><mi mathvariant="italic">ITL</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>4</mn></msub><mi>A</mi><mi>G</mi><msub><mrow><mi>E</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>β</mi></mrow><mn>5</mn></msub><mi>G</mi><mi mathvariant="italic">ENDE</mi><msub><mrow><mi>R</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><mi mathvariant="italic">TER</mi><msub><mrow><mi>M</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>+</mo><mi mathvariant="normal" /><msub><mrow><mi>ε</mi></mrow><mrow><mi>i</mi><mo>,</mo><mi>t</mi></mrow></msub></mrow></math> </ephtml> (<reflink idref="bib4" id="ref88">4</reflink>)</p> <p>Table 1. Variable definitions.</p> <p> <ephtml> <table><thead><tr><td>Variable</td><td>Definition</td></tr></thead><tbody valign="top"><tr><td><italic>RANK_INCREASE</italic></td><td>Evaluation ranking increase in the current period, which equals 1 for a ranking increase and 0 for a ranking decrease.</td></tr><tr><td><italic>RANK_DECREASE</italic></td><td>Evaluation ranking decrease in the current period, which equals 1 for a ranking decrease and 0 for a ranking increase.</td></tr><tr><td><italic>RANK_BOTTOM</italic></td><td>Indicator of poor evaluation in the previous period, which equals 1 for evaluation ranked in the bottom 25% and 0 otherwise.</td></tr><tr><td><italic>RANK_TOP</italic></td><td>Indicator of good evaluation in the previous period, which equals 1 for evaluation ranked in the top 25% and 0 otherwise.</td></tr><tr><td><italic>TITLE</italic></td><td>A dummy variable equal to 1 if the instructor is an associate/assistant professor and 0 if the instructor is a professor.</td></tr><tr><td><italic>AGE</italic></td><td>A dummy variable equal to 1 if the instructor is aged 45 years or under and 0 if the instructor is aged over 45 years.</td></tr><tr><td><italic>GENDER</italic></td><td>A dummy variable equal to 1 for female instructors and 0 for male instructors.</td></tr><tr><td><italic>TERM</italic></td><td>Semester indicators (9 dummy variables for 10 semesters)</td></tr></tbody></table> </ephtml> </p> <hd id="AN0173779477-12">Results</hd> <p>Table 2 presents the distribution of instructor characteristics. Of the 832 instructor-semester observations, 74% are associate/assistant professors, 45% are aged 45 years or under, and 54% are women.</p> <p>Table 2. Distribution of instructors' characteristics.</p> <p> <ephtml> <table><thead><tr><td>Instructors' title</td><td>Professor (<italic>TITLE</italic> = 0)</td><td>Associate/assistant professor (<italic>TITLE</italic> = 1)</td><td>Total</td></tr><tr><td>220 (26.44%)</td><td>612 (73.56%)</td><td>832 (100%)</td></tr></thead><tbody valign="top"><tr><td>Instructors' age</td><td>Aged over 45 years (<italic>AGE</italic> = 0)</td><td>Aged 45 years or under (<italic>AGE</italic> = 1)</td><td>Total</td></tr><tr><td>455 (54.69%)</td><td>377 (45.31%)</td><td>832 (100%)</td></tr><tr><td>Instructors' gender</td><td>Male (<italic>GENDER</italic> = 0)</td><td>Female (<italic>GENDER</italic> = 1)</td><td>Total</td></tr><tr><td>375 (45.07%)</td><td>457 (54.93%)</td><td>832 (100%)</td></tr></tbody></table> </ephtml> </p> <p>Table 3 reports a chi-square test of the correlation between the last evaluation and the change in the current evaluation. Panel A shows the result of the last poor evaluation. 73% of the instructors most recently rated as poor see their ranking improve in the current evaluation. For the instructors whose previous evaluation was not poor, the proportion of ranking improvement is only 41%. The chi-square test is significant at the 1% level, which indicates that instructors most recently evaluated as poor are significantly more likely to improve their current evaluation ranking than their counterparts without the last poor evaluation. Panel B shows that 75% of the instructors most recently rated as good see their evaluation decline in the current ranking. For the instructors whose most recent prior evaluation was not good, the proportion of ranking decline is only 43%. The chi-square test is significant at the 1% level, indicating that instructors whose most recent prior evaluation was good are significantly more likely to see their current ranking decrease than their counterparts without the last good evaluation. These results demonstrate that the previous evaluation ranking usually reverses in the current period.</p> <p>Table 3. Chi-square test.</p> <p> <ephtml> <table><thead><tr><td>Panel A: Last poor evaluation and the improvement in current evaluation</td></tr></thead><tbody valign="top"><tr><td /><td>With last poor ranking (<italic>RANK_BOTTOM</italic> = 1)</td><td>Without last poor ranking (<italic>RANK_BOTTOM</italic> = 0)</td></tr><tr><td>Current ranking increase (<italic>RANK_ INCREASE</italic> = 1)</td><td char=".">153 (72.86%)</td><td char=".">253 (40.68%)</td></tr><tr><td>Current ranking decrease (<italic>RANK_INCREASE</italic> = 0)</td><td char=".">57 (27.14%)</td><td char=".">369 (59.32%)</td></tr><tr><td>Total</td><td char=".">210 (100%)</td><td char=".">622 (100%)</td></tr><tr><td /><td>Pearson chi2(1) = 65.0760 Pr. = 0.000</td></tr><tr><td>Panel B: Last good evaluation and the decline in current evaluation</td></tr><tr><td /><td>With last good ranking (<italic>RANK_TOP</italic> = 1)</td><td>Without last good ranking (<italic>RANK_TOP</italic> = 0)</td></tr><tr><td>Current ranking decrease (<italic>RANK_DECREASE</italic> = 1)</td><td char=".">152 (75.25%)</td><td char=".">274 (43.49%)</td></tr><tr><td>Current ranking increase (<italic>RANK_ DECREASE</italic> = 0)</td><td char=".">50 (24.75%)</td><td char=".">356 (56.51%)</td></tr><tr><td>Total</td><td char=".">202 (100%)</td><td char=".">630 (100%)</td></tr><tr><td /><td>Pearson chi2(1) = 61.7329 Pr. = 0.000</td></tr></tbody></table> </ephtml> </p> <p>In Table 4, I examine the effect of the last poor evaluation on the improvement of current evaluation. All of the regressions control for instructor characteristics (i.e., title, age and gender) and semester fixed effects. I report the odds ratios (exp (<emph>β)</emph>) instead of coefficients (<emph>β</emph>), as the independent variables are dummy variables. Panel A reports the results of the full sample. In Column (<reflink idref="bib1" id="ref89">1</reflink>), the odds ratio of the last poor evaluation (<emph>RANK_BOTTOM</emph>) is 4.5511, which is significant at the 1% level. Given that the other variables remain unchanged, the probability of the instructors whose most recent prior evaluation was poor seeing their current evaluation improve is 4.5511 times (355%) higher than that of those not previously evaluated as poor. I include the interaction term of the instructors' title and the previous poor evaluation (<emph>TITLE*RANK_BOTTOM</emph>). The odds ratio of the interaction term is 5.2868, which is significant at the 1% level. The probability of the associate/assistant professors whose most recent prior evaluation was poor seeing their current evaluation improve is 5.2868 times (429%) higher than that of the professors. I also include the interaction term of the instructors' age and the previous poor evaluation (<emph>AGE*RANK_BOTTOM</emph>). The odds ratio of <emph>AGE* RANK_BOTTOM</emph> is significantly positive (10.5440, p < 0.01). In the case of instructors aged 45 years or under whose previous evaluation was poor, the probability of their current evaluation improving is 10.5440 times (954%) higher than that of instructors aged over 45 years. Finally, I include the interaction term of instructors' gender and their prior poor evaluation (<emph>GENDER*RANK_BOTTOM</emph>). The odds ratio of <emph>GENDER*RANK_BOTTOM</emph> is significantly positive (6.1073, p < 0.01). In the case of the most recent prior poor evaluation, the probability of female instructors seeing their current evaluation improve is 6.1073 times (511%) higher than that of male instructors.</p> <p>Table 4. Last poor evaluation's effect on the improvement of current evaluation.</p> <p> <ephtml> <table><thead><tr><td>Dependent variable: <italic>RANK_INCREASE</italic></td></tr><tr><td /><td>(1)</td><td>(2)</td><td>(3)</td><td>(4)</td></tr></thead><tbody valign="top"><tr><td>Panel A: Full sample (N = 832)</td></tr><tr><td /><td /><td><italic>X = TITLE</italic></td><td><italic>X = AGE</italic></td><td><italic>X = GENDER</italic></td></tr><tr><td><italic>RANK_BOTTOM</italic></td><td char=".">4.5511***</td><td char=".">4.3695***</td><td char=".">3.8078***</td><td char=".">3.4155***</td></tr><tr><td>(8.30)</td><td>(4.63)</td><td>(6.28)</td><td>(5.01)</td></tr><tr><td><italic>X*RANK_BOTTOM</italic></td><td /><td char=".">5.2868***</td><td char=".">10.5440***</td><td char=".">6.1073***</td></tr><tr><td /><td>(6.37)</td><td>(5.71)</td><td>(6.46)</td></tr><tr><td><italic>TITLE</italic></td><td char=".">1.1576</td><td char=".">1.1389</td><td char=".">0.1529</td><td char=".">0.1901</td></tr><tr><td>(0.82)</td><td>(0.63)</td><td>(0.81)</td><td>(0.97)</td></tr><tr><td><italic>AGE</italic></td><td char=".">1.5114***</td><td char=".">1.5135***</td><td char=".">1.3602*</td><td char=".">1.5098***</td></tr><tr><td>(2.63)</td><td>(2.64)</td><td>(1.80)</td><td>(2.62)</td></tr><tr><td><italic>GENDER</italic></td><td char=".">1.1105</td><td char=".">1.1128</td><td char=".">1.1155</td><td char=".">0.9709</td></tr><tr><td>(0.71)</td><td>(0.72)</td><td>(0.74)</td><td>(-0.18)</td></tr><tr><td><italic>Semester FE</italic></td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td></tr><tr><td><italic>Pseudo. R<sup>2</sup></italic></td><td char=".">0.0734</td><td char=".">0.0734</td><td char=".">0.0757</td><td char=".">0.0759</td></tr><tr><td>Panel B: Excluding last good evaluation (N = 630)</td></tr><tr><td /><td /><td><italic>X = TITLE</italic></td><td><italic>X = AGE</italic></td><td><italic>X = GENDER</italic></td></tr><tr><td><italic>RANK_BOTTOM</italic></td><td char=".">3.3335***</td><td char=".">2.6550***</td><td char=".">2.8514***</td><td char=".">2.6506***</td></tr><tr><td>(6.29)</td><td>(2.89)</td><td>(4.67)</td><td>(3.81)</td></tr><tr><td><italic>X*RANK_BOTTOM</italic></td><td /><td char=".">3.1155***</td><td char=".">8.2936***</td><td char=".">4.5701***</td></tr><tr><td /><td>(3.99)</td><td>(5.03)</td><td>(5.26)</td></tr><tr><td><italic>TITLE</italic></td><td char=".">0.9404</td><td char=".">0.8411</td><td char=".">0.9369</td><td char=".">0.9566</td></tr><tr><td>(-0.30)</td><td>(-0.71)</td><td>(-0.32)</td><td>(-0.22)</td></tr><tr><td><italic>AGE</italic></td><td char=".">1.7822***</td><td char=".">1.7978***</td><td char=".">1.5802**</td><td char=".">1.7811***</td></tr><tr><td>(3.15)</td><td>(3.17)</td><td>(2.23)</td><td>(3.14)</td></tr><tr><td><italic>GENDER</italic></td><td char=".">1.2058</td><td char=".">1.2166</td><td char=".">1.2088</td><td char=".">1.0482</td></tr><tr><td>(1.11)</td><td>(1.16)</td><td>(1.12)</td><td>(0.24)</td></tr><tr><td><italic>Semester FE</italic></td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td></tr><tr><td><italic>Pseudo. R<sup>2</sup></italic></td><td char=".">0.0641</td><td char=".">0.0649</td><td char=".">0.0662</td><td char=".">0.0661</td></tr><tr><td>Panel C: Excluding current evaluation change between −5% and 5% (N = 673)</td></tr><tr><td /><td /><td><italic>X = TITLE</italic></td><td><italic>X = AGE</italic></td><td><italic>X = GENDER</italic></td></tr><tr><td><italic>RANK_BOTTOM</italic></td><td char=".">8.5145***</td><td char=".">6.3590***</td><td char=".">6.8938***</td><td char=".">5.4664***</td></tr><tr><td>(8.97)</td><td>(4.83)</td><td>(7.04)</td><td>(5.59)</td></tr><tr><td><italic>X*RANK_BOTTOM</italic></td><td /><td char=".">12.0710***</td><td char=".">23.7975***</td><td char=".">17.1772***</td></tr><tr><td /><td>(7.06)</td><td>(5.18)</td><td>(6.72)</td></tr><tr><td><italic>TITLE</italic></td><td char=".">1.2887</td><td char=".">1.1774</td><td char=".">1.2725</td><td char=".">1.3311</td></tr><tr><td>(1.23)</td><td>(0.73)</td><td>(1.19)</td><td>(1.38)</td></tr><tr><td><italic>AGE</italic></td><td char=".">1.4336**</td><td char=".">1.4408**</td><td char=".">1.3177</td><td char=".">1.4267**</td></tr><tr><td>(2.05)</td><td>(2.05)</td><td>(1.47)</td><td>(2.01)</td></tr><tr><td><italic>GENDER</italic></td><td char=".">1.2352</td><td char=".">1.2485</td><td char=".">1.2371</td><td char=".">1.0641</td></tr><tr><td>(1.25)</td><td>(1.31)</td><td>(1.26)</td><td>(0.34)</td></tr><tr><td><italic>Semester FE</italic></td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td></tr><tr><td><italic>Pseudo. R<sup>2</sup></italic></td><td char=".">0.1129</td><td char=".">0.1139</td><td char=".">0.1155</td><td char=".">0.1179</td></tr></tbody></table> </ephtml> </p> <p>1 This table reports the odds ratios instead of coefficients. See Table 1 for the definition of variables. The robust z statistics are in parentheses.</p> <p>2 indicate statistical significance at the 1%, 5%, and 10% level, respectively.</p> <p>In Panel A, I compare instructors whose prior evaluation was poor with those who were not rated poor previously. The latter group consists of the instructors whose most recent prior evaluation was medium (ranked between 25% and 75%) and those whose prior evaluation was good (ranked in the top 25%). I exclude the instructors whose prior evaluation was good and focus on the comparison between the instructors whose prior evaluation was poor and those who were rated medium. Panel B presents the regression results of the new sample. Considering that small changes in the evaluation should be subject to normal fluctuations, I remove changes in current evaluation between plus or minus 5% and present the results in Panel C. The findings of the two robustness checks are similar to the baseline results. Taken overall, Table 4 shows that there is a greater probability that instructors whose most recent prior evaluation was poor will improve their current evaluation, and the effect is more pronounced among the instructors who are associate/assistant professors, aged 45 years or under and female.</p> <p>In Table 5, I examine the effect of the last good evaluation on the decline of current evaluation. Panel A reports the results of the full sample. In Column (<reflink idref="bib1" id="ref90">1</reflink>), the odds ratio of the last good evaluation (<emph>RANK_TOP</emph>) is 7.3356, which is significant at the 1% level. Assuming that other variables remain unchanged, the probability that instructors whose most recent prior evaluation was good seeing their current evaluation decline is 7.3356 times (634%) higher than that of those whose previous rating was not good. Moreover, in Columns (<reflink idref="bib2" id="ref91">2</reflink>) to (<reflink idref="bib4" id="ref92">4</reflink>), I include the interaction terms of instructors' title, age and gender and the previous good evaluation (<emph>TITLE*RANK_TOP, AGE*RANK_TOP, GENDER*RANK_TOP</emph>). The odds ratios of the three interaction terms are all significantly larger than 1 (coefficients are all significantly positive). These results show that when the most recent prior evaluation is good, the probability of the current evaluation decreasing is higher among the instructors who are associate/assistant professors, aged 45 years or under and female.</p> <p>Table 5. Last good evaluation's effect on the decline of current evaluation.</p> <p> <ephtml> <table><thead><tr><td>Dependent variable: <italic>RANK_DECREASE</italic></td></tr></thead><tbody valign="top"><tr><td /><td>(1)</td><td>(2)</td><td>(3)</td><td>(4)</td></tr><tr><td>Panel A: Full sample (N = 832)</td></tr><tr><td /><td /><td><italic>X = TITLE</italic></td><td><italic>X = AGE</italic></td><td><italic>X = GENDER</italic></td></tr><tr><td><italic>RANK_TOP</italic></td><td char=".">7.3356***</td><td char=".">9.9784***</td><td char=".">4.4999***</td><td char=".">5.1762***</td></tr><tr><td>(7.83)</td><td>(4.87)</td><td>(5.37)</td><td>(4.86)</td></tr><tr><td><italic>X*RANK_TOP</italic></td><td /><td char=".">3.7180***</td><td char=".">3.2112***</td><td char=".">3.4492***</td></tr><tr><td /><td>(5.17)</td><td>(4.81)</td><td>(5.47)</td></tr><tr><td><italic>TITLE</italic></td><td char=".">0.9079</td><td char=".">1.0659</td><td char=".">0.9084</td><td char=".">0.9189</td></tr><tr><td>(-0.57)</td><td>(0.33)</td><td>(-0.56)</td><td>(-0.49)</td></tr><tr><td><italic>AGE</italic></td><td char=".">0.7520*</td><td char=".">0.7597*</td><td char=".">0.7612</td><td char=".">0.7506*</td></tr><tr><td>(-1.83)</td><td>(-1.77)</td><td>(-1.58)</td><td>(-1.84)</td></tr><tr><td><italic>GENDER</italic></td><td char=".">0.8296</td><td char=".">0.8550</td><td char=".">0.8299</td><td char=".">0.8666</td></tr><tr><td>(-1.27)</td><td>(-1.06)</td><td>(-1.26)</td><td>(-0.88)</td></tr><tr><td><italic>Semester FE</italic></td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td></tr><tr><td><italic>Pseudo. R<sup>2</sup></italic></td><td char=".">0.0677</td><td char=".">0.0719</td><td char=".">0.0678</td><td char=".">0.0681</td></tr><tr><td>Panel B: Excluding last poor evaluation (N = 622)</td></tr><tr><td /><td /><td><italic>X = TITLE</italic></td><td><italic>X = AGE</italic></td><td><italic>X = GENDER</italic></td></tr><tr><td><italic>RANK_TOP</italic></td><td char=".">3.0175***</td><td char=".">6.8251***</td><td char=".">2.8333***</td><td char=".">3.7793***</td></tr><tr><td>(5.62)</td><td>(3.91)</td><td>(3.53)</td><td>(3.72)</td></tr><tr><td><italic>X*RANK_TOP</italic></td><td /><td char=".">2.6303***</td><td char=".">2.0869***</td><td char=".">2.5820***</td></tr><tr><td /><td>(3.38)</td><td>(2.83)</td><td>(3.94)</td></tr><tr><td><italic>TITLE</italic></td><td char=".">0.8569</td><td char=".">1.0709</td><td char=".">0.8552</td><td char=".">0.8712</td></tr><tr><td>(-0.75)</td><td>(0.29)</td><td>(-0.76)</td><td>(-0.67)</td></tr><tr><td><italic>AGE</italic></td><td char=".">0.6790**</td><td char=".">0.6865**</td><td char=".">0.6603**</td><td char=".">0.6772**</td></tr><tr><td>(-2.16)</td><td>(-2.10)</td><td>(-2.02)</td><td>(-2.18)</td></tr><tr><td><italic>GENDER</italic></td><td char=".">0.8876</td><td char=".">0.9144</td><td char=".">0.8872</td><td char=".">0.9583</td></tr><tr><td>(-0.69)</td><td>(-0.51)</td><td>(-0.69)</td><td>(-0.21)</td></tr><tr><td><italic>Semester FE</italic></td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td></tr><tr><td><italic>Pseudo. R<sup>2</sup></italic></td><td char=".">0.0594</td><td char=".">0.0643</td><td char=".">0.0595</td><td char=".">0.0602</td></tr><tr><td>Panel C: Excluding current evaluation change between −5% and 5% (N = 673)</td></tr><tr><td /><td /><td><italic>X = TITLE</italic></td><td><italic>X = AGE</italic></td><td><italic>X = GENDER</italic></td></tr><tr><td><italic>RANK_TOP</italic></td><td char=".">5.9718***</td><td char=".">11.8227***</td><td char=".">7.3945***</td><td char=".">5.9783***</td></tr><tr><td>(8.14)</td><td>(4.77)</td><td>(5.89)</td><td>(4.76)</td></tr><tr><td><italic>X*RANK_TOP</italic></td><td /><td char=".">5.0402***</td><td char=".">4.3223***</td><td char=".">4.4167***</td></tr><tr><td /><td>(5.49)</td><td>(5.11)</td><td>(5.52)</td></tr><tr><td><italic>TITLE</italic></td><td char=".">0.9075</td><td char=".">1.0357</td><td char=".">0.9120</td><td char=".">0.9076</td></tr><tr><td>(-0.50)</td><td>(0.16)</td><td>(-0.47)</td><td>(-0.50)</td></tr><tr><td><italic>AGE</italic></td><td char=".">0.8025</td><td char=".">0.8103</td><td char=".">0.8566</td><td char=".">0.8025</td></tr><tr><td>(-1.25)</td><td>(-1.20)</td><td>(-0.80)</td><td>(-1.25)</td></tr><tr><td><italic>GENDER</italic></td><td char=".">0.7398*</td><td char=".">0.7558*</td><td char=".">0.7394*</td><td char=".">0.7400*</td></tr><tr><td>(-1.81)</td><td>(-1.68)</td><td>(-1.81)</td><td>(-1.66)</td></tr><tr><td><italic>Semester FE</italic></td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td></tr><tr><td><italic>Pseudo. R<sup>2</sup></italic></td><td char=".">0.0892</td><td char=".">0.0921</td><td char=".">0.0900</td><td char=".">0.0892</td></tr></tbody></table> </ephtml> </p> <ulist> <item>3 This table reports the odds ratios instead of coefficients. See Table 1 for the definition of variables. The robust z statistics are in parentheses.</item> <item>4 indicate statistical significance at the 1%, 5%, and 10% level, respectively.</item> </ulist> <p>Similarly, I exclude the instructors whose previous evaluation was poor and focus on the comparison between the instructors with prior good evaluation and those with prior medium evaluation. Panel B of Table 5 presents the regression results of the new sample. I also exclude any changes in current evaluations between plus or minus 5% and present the results in Panel C. The findings in Panels B and C are similar to those in Panel A. In conclusion, there is a greater probability that instructors whose prior evaluation was good will see their current evaluation decline, and the effect is more pronounced among the instructors who are associate/assistant professors, aged 45 years or under and female.</p> <hd id="AN0173779477-13">Discussion and conclusion</hd> <p>Views are mixed regarding the validity of SET, with evidence for and against its use in universities. Notably, opponents' arguments have dominated in the past decade (Wallace, Lewis, and Allen [<reflink idref="bib44" id="ref93">44</reflink>]; Heffernan [<reflink idref="bib20" id="ref94">20</reflink>]). Scholars continue to challenge the use of SET as an accurate measure of teaching quality in higher education, focusing on poor reliability, low effectiveness, disturbance stemming from non-teaching factors, and grade inflation. However, few studies have examined the effect of SET on the improvement of teaching quality in universities. This study is the first to examine whether SET improves teaching quality. Moreover, I adopt a new methodology to distinguish the improvement in teaching quality from the reversal effect. This study bolsters research into SET by showing that its weaknesses cause the failure to improve teaching quality.</p> <p>With a large pool of SET data collected at a top Chinese university over 11 consecutive semesters of the 2016–2021 period, this study empirically examines whether SET prompts instructors to improve the quality of their teaching. Given the many doubts about SET in the literature, this study proposes that there could be a reversal effect in evaluations. I adopt a new design to distinguish the improvement in teaching quality from the reversal effect. It shows that instructors whose prior evaluation was poor (good) are more likely to see their current evaluation improve (decrease). I provide evidence that supports the reversal effect. Moreover, I find that instructors' characteristics play moderating roles. Regardless of bad-to-good or good-to-bad, the reversal effects are more marked among associate/assistant professors, instructors aged 45 years or under and female instructors. Taken overall, a reversal effect emerges, and SET fails to improve teaching quality.</p> <p>My findings have three implications for SET in higher education. First, it is necessary to adopt other evaluation methods, such as supervisor and peer evaluation, alongside SET. Because of the reversal effect in SET, an increase in SET ranking does not mean teaching quality improvement. A more scientific and comprehensive system of evaluating teaching quality should involve assessments by supervisors, peers and students. Although supervisor and peer evaluation also have weaknesses, multiple measures would be more accurate in evaluations. Second, universities should act to improve the validity of SET. The shortcomings of SET, including poor reliability, low effectiveness, disturbance by non-teaching factors, and grade inflation, are responsible for the reversal effect. Instructors would be motivated to improve the quality of their teaching by capitalizing on student feedback if they recognized the validity of SET and valued its feedback. Finally, universities should consider how to reduce the extent of the reversal effect on instructors who are young, female and lack the title of professor. One possible way might be to inform the students of their bias, as they are probably unconscious of their discriminatory behaviors. Making students realize that non-teaching factors influence their evaluations may change their behaviors.</p> <p>Although my findings are robust, this study has limitations. The study uses SET data from a college at a particular university in China. The extent to which the findings can be generalized is unclear. Research on the relationship between SET and teaching quality improvement is scarce, and more studies with large-scale SET samples from other countries are needed to explore this issue.</p> <hd id="AN0173779477-14">Disclosure statement</hd> <p>No potential conflict of interest was reported by the authors.</p> <ref id="AN0173779477-15"> <title> References </title> <blist> <bibl id="bib1" idref="ref41" type="bt">1</bibl> <bibtext> Bavishi, A., M. R. Hebl, and J. M. Madera. 2010. " The Effect of Professor Ethnicity and Gender on Student Evaluations: Judged before Met." Journal of Diversity in Higher Education 3 (4): 245 – 256. doi: 10.1037/a0020763.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref52" type="bt">2</bibl> <bibtext> Berezvai, Z., G. D. Lukáts, and R. Molontay. 2021. " Can Professors Buy Better Evaluation with Lenient Grading? The Effect of Grade Inflation on Student Evaluation of Teaching." Assessment & Evaluation in Higher Education 46 (5): 793 – 808. doi: 10.1080/02602938.2020.1821866.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref1" type="bt">3</bibl> <bibtext> Blair, E., and K. V. Noel. 2014. " Improving Higher Education Practice through Student Evaluation Systems: Is the Student Voice Being Heard? " Assessment & Evaluation in Higher Education 39 (7): 879 – 894. doi: 10.1080/02602938.2013.875984.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref38" type="bt">4</bibl> <bibtext> Boring, A. 2017. " Gender Biases in Student Evaluations of Teaching." Journal of Public Economics 145 : 27 – 41. doi: 10.1016/j.jpubeco.2016.11.006.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref73" type="bt">5</bibl> <bibtext> Boring, A., K. Ottoboni, and P. Stark. 2016. Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness. Science Open Research.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref30" type="bt">6</bibl> <bibtext> Braga, M., M. Paccagnella, and M. Pellizzari. 2014. " Evaluating Students' Evaluations." Economics of Education Review 41 : 71 – 88. doi: 10.1016/j.econedurev.2014.04.002.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref28" type="bt">7</bibl> <bibtext> Carpenter, S. K., A. E. Witherby, and S. K. Tauber. 2020. " On Students' (Mis) Judgments of Learning and Teaching Effectiveness." Journal of Applied Research in Memory and Cognition 9 (2): 137 – 151. doi: 10.1016/j.jarmac.2019.12.009.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref29" type="bt">8</bibl> <bibtext> Carrell, S. E., and J. E. West. 2010. " Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors." Journal of Political Economy 118 (3): 409 – 432. doi: 10.1086/653808.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref51" type="bt">9</bibl> <bibtext> Cho, D., W. Baek, and J. Cho. 2015. " Why Do Good Performing Students Highly Rate Their Instructors? Evidence from a Natural Experiment." Economics of Education Review 49 : 172 – 179. doi: 10.1016/j.econedurev.2015.10.001.</bibtext> </blist> <blist> <bibtext> Clayson, D. 2022. " The Student Evaluation of Teaching and Likability: What the Evaluations Actually Measure." Assessment & Evaluation in Higher Education 47 (2): 313 – 326. doi: 10.1080/02602938.2021.1909702.</bibtext> </blist> <blist> <bibtext> Cook-Sather, A. 2006. " Sound, Presence, and Power: 'Student Voice' in Educational Research and Reform." Curriculum Inquiry 36 (4): 359 – 390. doi: 10.1111/j.1467-873X.2006.00363.x.</bibtext> </blist> <blist> <bibtext> Eagly, A., and S. J. Karau. 2002. " Role Congruity Theory of Prejudice toward Female Leaders." Psychological Review 109 (3): 573 – 598. doi: 10.1037/0033-295x.109.3.573.</bibtext> </blist> <blist> <bibtext> Fan, Y., L. Shepherd, D. Slavich, D. Waters, M. Stone, R. Abel, and E. Johnston. 2019. " Gender and Cultural Bias in Student Evaluations: Why Representation Matters." PloS One 14 (2): e0209749. doi: 10.1371/journal.pone.0209749.</bibtext> </blist> <blist> <bibtext> Feistauer, D., and T. Richter. 2017. " How Reliable Are Students' Evaluations of Teaching Quality? A Variance Components Approach." Assessment & Evaluation in Higher Education 42 (8): 1263 – 1279. doi: 10.1080/02602938.2016.1261083.</bibtext> </blist> <blist> <bibtext> Freng, S., and D. Webber. 2009. " Turning up the Heat on Online Teaching Evaluations: Does "Hotness." Teaching of Psychology 36 (3): 189 – 193. doi: 10.1080/00986280902959739.</bibtext> </blist> <blist> <bibtext> Galbraith, C., G. Merrill, and D. Kline. 2012. " Are Student Evaluations of Teaching Effectiveness Valid for Measuring Student Learning Outcomes in Business Related Classes? A Neural Network and Bayesian Analyses." Research in Higher Education 53 (3): 353 – 374. doi: 10.1007/s11162-011-9229-0.</bibtext> </blist> <blist> <bibtext> Ginns, P., M. Prosser, and S. Barrie. 2007. " Students' Perceptions of Teaching Quality in Higher Education: The Perspective of Currently Enrolled Students." Studies in Higher Education 32 (5): 603 – 615. doi: 10.1080/03075070701573773.</bibtext> </blist> <blist> <bibtext> Griffin, B. W. 2004. " Grading Leniency, Grade Discrepancy, and Student Ratings of Instruction." Contemporary Educational Psychology 29 (4): 410 – 425. doi: 10.1016/j.cedpsych.2003.11.001.</bibtext> </blist> <blist> <bibtext> Hammonds, F., G. J. Mariano, G. Ammons, and S. Chambers. 2017. " Student Evaluations of Teaching: Improving Teaching Quality in Higher Education." Perspectives: Policy and Practice in Higher Education 21 (1): 26 – 33.</bibtext> </blist> <blist> <bibtext> Heffernan, T. 2022. " Sexism, Racism, Prejudice, and Bias: A Literature Review and Synthesis of Research Surrounding Student Evaluations of Courses and Teaching." Assessment & Evaluation in Higher Education 47 (1): 144 – 154. doi: 10.1080/02602938.2021.1888075.</bibtext> </blist> <blist> <bibtext> Hellman, C. M. 1998. " Faculty Evaluation by Students: A Comparison between Full-Time and Adjunct Faculty." Journal of Applied Research in the Community College 6 (1): 45 – 50.</bibtext> </blist> <blist> <bibtext> Hessler, M., D. M. Pöpping, H. Hollstein, H. Ohlenburg, P. H. Arnemann, C. Massoth, L. M. Seidel, A. Zarbock, and M. Wenk. 2018. " Availability of Cookies during an Academic Course Session Affects Evaluation of Teaching." Medical Education 52 (10): 1064 – 1072. doi: 10.1111/medu.13627.</bibtext> </blist> <blist> <bibtext> Johnston, R. J. 1994. " Is There a Correlation between Department Research and Teaching Quality? " Environment and Planning A: Economy and Space 26 (10): 1491 – 1496. doi: 10.1068/a261491.</bibtext> </blist> <blist> <bibtext> Lakeman, R., R. Coutts, M. Hutchinson, D. Massey, D. Nasrawi, J. Fielden, and M. Lee. 2022. " Playing the SET Game: How Teachers View the Impact of Student Evaluation on the Experience of Teaching and Learning." Assessment & Evaluation in Higher Education. doi: 10.1080/02602938.2022.2126430.</bibtext> </blist> <blist> <bibtext> MacNell, L., A. Driscoll, and A. Hunt. 2015. " What's in a Name: Exposing Gender Bias in Student Ratings of Teaching." Innovative Higher Education 40 (4): 291 – 303. doi: 10.1007/s10755-014-9313-4.</bibtext> </blist> <blist> <bibtext> Marsh, H. W., and M. Bailey. 1993. " Multidimensional Students' Evaluations of Teaching Effectiveness: A Profile Analysis." The Journal of Higher Education 64 (1): 1 – 18.</bibtext> </blist> <blist> <bibtext> McPherson, M. A., R. T. Jewell, and M. Kim. 2009. " What Determines Student Evaluation Scores? A Random Effects Analysis of Undergraduate Economics Classes." Eastern Economic Journal 35 (1): 37 – 51. doi: 10.1057/palgrave.eej.9050042.</bibtext> </blist> <blist> <bibtext> Meshkani, Z., and A. F. Hossein. 2003. " The Effect of Contextual Factors on Results of Teaching Evaluation in College of Medicine, Tehran University of Medical Sciences." Journal of Medical Education 6 : 117 – 121.</bibtext> </blist> <blist> <bibtext> Otto, J., D. A. Sanford, and D. N. Ross. 2008. " Does Ratemyprofessor.com Really Rate My Professor? " Assessment & Evaluation in Higher Education 33 (4): 355 – 368. doi: 10.1080/02602930701293405.</bibtext> </blist> <blist> <bibtext> Palmer, S. 2012. " Student Evaluation of Teaching: Keeping in Touch with Reality." Quality in Higher Education 18 (3): 297 – 311. doi: 10.1080/13538322.2012.730336.</bibtext> </blist> <blist> <bibtext> Park, B., and J. Cho. 2022. " How Does Grade Inflation Affect Student Evaluation of Teaching? " Assessment & Evaluation in Higher Education. doi: 10.1080/02602938.2022.2126429.</bibtext> </blist> <blist> <bibtext> Ragan, J. F., and B. Walia. 2010. " Differences in Student Evaluations of Principles and Other Economic Courses and the Allocation of Faculty across Courses." The Journal of Economic Education 41 (4): 335 – 352. doi: 10.1080/00220485.2010.510389.</bibtext> </blist> <blist> <bibtext> Rantanen, P. 2013. " The Number of Feedbacks Needed for Reliable Evaluation: A Multilevel Analysis of the Reliability, Stability and Generalisability of Students' Evaluation of Teaching." Assessment & Evaluation in Higher Education 38 (2): 224 – 239. doi: 10.1080/02602938.2011.625471.</bibtext> </blist> <blist> <bibtext> Read, W. J., D. V. Rama, and K. Raghunandan. 2001. " The Relationship between Student Evaluations of Teaching and Faculty Evaluations." Journal of Education for Business 76 (4): 189 – 192. doi: 10.1080/08832320109601309.</bibtext> </blist> <blist> <bibtext> Richardson, J., J. Slater, and J. Wilson. 2007. " The National Student Survey: Development, Findings and Implications." Studies in Higher Education 32 (5): 557 – 580. doi: 10.1080/03075070701573757.</bibtext> </blist> <blist> <bibtext> Spooren, P. 2010. " On the Credibility of the Judge: A Cross-Classified Multilevel Analysis on Students' Evaluation of Teaching." Studies in Educational Evaluation 36 (4): 121 – 131. doi: 10.1016/j.stueduc.2011.02.001.</bibtext> </blist> <blist> <bibtext> Stonebraker, R. J., and G. S. Stone. 2015. " Too Old to Teach? The Effect of Age on College and University Professors." Research in Higher Education, 2015 56 (8): 793 – 812.</bibtext> </blist> <blist> <bibtext> Storage, D., Z. Horne, A. Cimpian, and S. J. Leslie. 2016. " The Frequency of 'Brilliant' and 'Genius' in Teaching Evaluations Predicts the Representation of Women and African Americans across Fields." PloS One 11 (3): e0150194–17. doi: 10.1371/journal.pone.0150194.</bibtext> </blist> <blist> <bibtext> Stroebe, W. 2016. " Why Good Teaching Evaluations May Reward Bad Teaching: On Grade Inflation and Other Unintended Consequences of Student Evaluations." Perspectives on Psychological Science : A Journal of the Association for Psychological Science 11 (6): 800 – 816. doi: 10.1177/1745691616650284.</bibtext> </blist> <blist> <bibtext> Stroebe, W. 2020. " Student Evaluations of Teaching Encourages Poor Teaching and Contributes to Grade Inflation: A Theoretical and Empirical Analysis." Basic and Applied Social Psychology 42 (4): 276 – 294. doi: 10.1080/01973533.2020.1756817.</bibtext> </blist> <blist> <bibtext> Ting, K. 2000. " A Multilevel Perspective on Student Ratings of Instruction: Lessons from the Chinese Experience." Research in Higher Education 141 (5): 637 – 661.</bibtext> </blist> <blist> <bibtext> Uttl, B., C. A. White, and D. W. Gonzalez. 2017. " Meta-Analysis of Faculty's Teaching Effectiveness: Student Evaluation of Teaching Ratings and Student Learning Are Not Related." Studies in Educational Evaluation 54 : 22 – 42. doi: 10.1016/j.stueduc.2016.08.007.</bibtext> </blist> <blist> <bibtext> Valencia, E. 2020. " Acquiescence, Instructor's Gender Bias and Validity of Student Evaluation of Teaching." Assessment & Evaluation in Higher Education 45 (4): 483 – 495. doi: 10.1080/02602938.2019.1666085.</bibtext> </blist> <blist> <bibtext> Wallace, S. L., A. K. Lewis, and M. D. Allen. 2019. " The State of the Literature on Student Evaluations of Teaching and an Exploratory Analysis of Written Comments: Who Benefits Most? " College Teaching 67 (1): 1 – 14. doi: 10.1080/87567555.2018.1483317.</bibtext> </blist> <blist> <bibtext> Worthington, A. 2002. " The Impact of Student Perceptions and Characteristics on Teaching Evaluations: A Case Study in Finance Education." Assessment & Evaluation in Higher Education 27 (1): 49 – 64. doi: 10.1080/02602930120105054.</bibtext> </blist> <blist> <bibtext> Zhao, J., and D. J. Gallant. 2012. " Student Evaluation of Instruction in Higher Education: Exploring Issues of Validity and Reliability." Assessment & Evaluation in Higher Education 37 (2): 227 – 235. doi: 10.1080/02602938.2010.523819.</bibtext> </blist> </ref> <aug> <p>By Yanyan Chen</p> <p>Reported by Author</p> </aug> <nolink nlid="nl1" bibid="bib40" firstref="ref2"></nolink> <nolink nlid="nl2" bibid="bib31" firstref="ref3"></nolink> <nolink nlid="nl3" bibid="bib44" firstref="ref4"></nolink> <nolink nlid="nl4" bibid="bib20" firstref="ref5"></nolink> <nolink nlid="nl5" bibid="bib30" firstref="ref6"></nolink> <nolink nlid="nl6" bibid="bib19" firstref="ref7"></nolink> <nolink nlid="nl7" bibid="bib46" firstref="ref10"></nolink> <nolink nlid="nl8" bibid="bib26" firstref="ref11"></nolink> <nolink nlid="nl9" bibid="bib21" firstref="ref12"></nolink> <nolink nlid="nl10" bibid="bib17" firstref="ref13"></nolink> <nolink nlid="nl11" bibid="bib35" firstref="ref14"></nolink> <nolink nlid="nl12" bibid="bib11" firstref="ref16"></nolink> <nolink nlid="nl13" bibid="bib14" firstref="ref20"></nolink> <nolink nlid="nl14" bibid="bib36" firstref="ref21"></nolink> <nolink nlid="nl15" bibid="bib33" firstref="ref22"></nolink> <nolink nlid="nl16" bibid="bib29" firstref="ref24"></nolink> <nolink nlid="nl17" bibid="bib10" firstref="ref25"></nolink> <nolink nlid="nl18" bibid="bib16" firstref="ref26"></nolink> <nolink nlid="nl19" bibid="bib42" firstref="ref27"></nolink> <nolink nlid="nl20" bibid="bib39" firstref="ref31"></nolink> <nolink nlid="nl21" bibid="bib34" firstref="ref34"></nolink> <nolink nlid="nl22" bibid="bib45" firstref="ref35"></nolink> <nolink nlid="nl23" bibid="bib32" firstref="ref36"></nolink> <nolink nlid="nl24" bibid="bib37" firstref="ref37"></nolink> <nolink nlid="nl25" bibid="bib13" firstref="ref39"></nolink> <nolink nlid="nl26" bibid="bib43" firstref="ref40"></nolink> <nolink nlid="nl27" bibid="bib38" firstref="ref42"></nolink> <nolink nlid="nl28" bibid="bib22" firstref="ref43"></nolink> <nolink nlid="nl29" bibid="bib15" firstref="ref47"></nolink> <nolink nlid="nl30" bibid="bib18" firstref="ref50"></nolink> <nolink nlid="nl31" bibid="bib24" firstref="ref55"></nolink> <nolink nlid="nl32" bibid="bib41" firstref="ref60"></nolink> <nolink nlid="nl33" bibid="bib23" firstref="ref61"></nolink> <nolink nlid="nl34" bibid="bib28" firstref="ref62"></nolink> <nolink nlid="nl35" bibid="bib27" firstref="ref63"></nolink> <nolink nlid="nl36" bibid="bib12" firstref="ref74"></nolink> <nolink nlid="nl37" bibid="bib25" firstref="ref75"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1402755
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Does Students' Evaluation of Teaching Improve Teaching Quality? Improvement versus the Reversal Effect
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Chen%2C+Yanyan%22">Chen, Yanyan</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-8892-2012">0000-0002-8892-2012</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Assessment+%26+Evaluation+in+Higher+Education%22"><i>Assessment & Evaluation in Higher Education</i></searchLink>. 2023 48(8):1195-1207.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 13
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research<br />Tests/Questionnaires
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Universities%22">Universities</searchLink><br /><searchLink fieldCode="DE" term="%22College+Faculty%22">College Faculty</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Evaluation+of+Teacher+Performance%22">Student Evaluation of Teacher Performance</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Effectiveness%22">Teacher Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Improvement%22">Teacher Improvement</searchLink><br /><searchLink fieldCode="DE" term="%22Shift+Studies%22">Shift Studies</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Response%22">Teacher Response</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Quality%22">Educational Quality</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Evaluation%22">Teacher Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Problems%22">Evaluation Problems</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22China%22">China</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/02602938.2023.2177252
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0260-2938<br />1469-297X
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Using students' assessments of teaching at a top university in China from 2016 to 2021, this study examines whether evaluation of teaching improves teaching quality. Given the many doubts about the validity of students' evaluation of teaching, this study adopts a methodology to distinguish teaching quality improvement from the reversal effect. It shows that instructors with a poor (good) ranking in the previous evaluation are more likely to receive an improved (decreased) ranking in the current evaluation. This reversal effect is more pronounced among the instructors who are associate/assistant professors, younger and female. This study provides evidence that supports the reversal effect rather than the improvement.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1402755
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1402755
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/02602938.2023.2177252
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 13
        StartPage: 1195
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Universities
        Type: general
      – SubjectFull: College Faculty
        Type: general
      – SubjectFull: Student Evaluation of Teacher Performance
        Type: general
      – SubjectFull: Teacher Effectiveness
        Type: general
      – SubjectFull: Teacher Improvement
        Type: general
      – SubjectFull: Shift Studies
        Type: general
      – SubjectFull: Teacher Response
        Type: general
      – SubjectFull: Educational Quality
        Type: general
      – SubjectFull: Teacher Evaluation
        Type: general
      – SubjectFull: Evaluation Problems
        Type: general
      – SubjectFull: China
        Type: general
    Titles:
      – TitleFull: Does Students' Evaluation of Teaching Improve Teaching Quality? Improvement versus the Reversal Effect
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Chen, Yanyan
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-print
              Value: 0260-2938
            – Type: issn-electronic
              Value: 1469-297X
          Numbering:
            – Type: volume
              Value: 48
            – Type: issue
              Value: 8
          Titles:
            – TitleFull: Assessment & Evaluation in Higher Education
              Type: main
ResultId 1