View in EDS HTML Full Text PDF Full Text

Forecasting the Student-Professor Matches That Result in Unusually Effective Teaching

Saved in:
Bibliographic Details
Title: Forecasting the Student-Professor Matches That Result in Unusually Effective Teaching
Language: English
Authors: Gross, Jennifer, Lakey, Brian, Lucas, Jessica L., LaCross, Ryan, Plotkowski, Andrea R., Winegard, Bo
Source: British Journal of Educational Psychology. Mar 2015 85(1):19-32.
Availability: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Peer Reviewed: Y
Page Count: 14
Publication Date: 2015
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Teacher Student Relationship, College Students, College Faculty, Lecture Method, Student Evaluation of Teacher Performance, Video Technology, Teaching Methods, Statistical Analysis, Student Attitudes, Teacher Characteristics, Teacher Effectiveness, Teacher Evaluation, Memory, Predictor Variables
DOI: 10.1111/bjep.12049
ISSN: 0007-0998
Abstract: Background: Two important influences on students' evaluations of teaching are relationship and professor effects. Relationship effects reflect unique matches between students and professors such that some professors are unusually effective for some students, but not for others. Professor effects reflect inter-rater agreement that some professors are more effective than others, on average across students. Aims: We attempted to forecast students' evaluations of live lectures from brief, video-recorded teaching trailers. Sample: Participants were 145 college students (74% female) enrolled in introductory psychology courses at a public university in the Great Lakes region of the United States. Methods: Students viewed trailers early in the semester and attended live lectures months later. Because subgroups of students viewed the same professors, statistical analyses could isolate professor and relationship effects. Results: Evaluations were influenced strongly by relationship and professor effects, and students' evaluations of live lectures could be forecasted from students' evaluations of teaching trailers. That is, we could forecast the individual students who would respond unusually well to a specific professor (relationship effects). We could also forecast which professors elicited better evaluations in live lectures, on average across students (professor effects). Professors who elicited unusually good evaluations in some students also elicited better memory for lectures in those students. Conclusions: It appears possible to forecast relationship and professor effects on teaching evaluations by presenting brief teaching trailers to students. Thus, it might be possible to develop online recommender systems to help match students and professors so that unusually effective teaching emerges.
Abstractor: As Provided
Number of References: 29
Entry Date: 2015
Accession Number: EJ1052142
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHgHKQxzBV-4sOEYbXfzX4_AAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDDvKFdjhHNVy8BkByAIBEICBmjnJPrYbKTZlN99pRyXPZuDC2E8awPuPOFuRMZqPnazE4amU7HHiUijhgsEhr5LpYxtHUc-7-7345sCAiozUE6dgF8GBJ6vVp_q5Y3PpKlZvj5REO2jw8f1W5-LlYBagqxu4O9ktqGwox25GfFGvW_2oCNo52aNUa06M-MhBkNO4Lm7MtLVzaos9dBS3b-Enf9gV1iQd8gBuo8M=
Text:
  Availability: 1
  Value: <anid>AN0101003818;6kx01mar.15;2018Jun29.11:54;v2.2.500</anid> <title id="AN0101003818-1">Forecasting the student-professor matches that result in unusually effective teaching. </title> <p>Background: Two important influences on students' evaluations of teaching are relationship and professor effects. Relationship effects reflect unique matches between students and professors such that some professors are unusually effective for some students, but not for others. Professor effects reflect inter‐rater agreement that some professors are more effective than others, on average across students. Aims: We attempted to forecast students' evaluations of live lectures from brief, video‐recorded teaching trailers. Sample: Participants were 145 college students (74% female) enrolled in introductory psychology courses at a public university in the Great Lakes region of the United States. Methods: Students viewed trailers early in the semester and attended live lectures months later. Because subgroups of students viewed the same professors, statistical analyses could isolate professor and relationship effects. Results: Evaluations were influenced strongly by relationship and professor effects, and students' evaluations of live lectures could be forecasted from students' evaluations of teaching trailers. That is, we could forecast the individual students who would respond unusually well to a specific professor (relationship effects). We could also forecast which professors elicited better evaluations in live lectures, on average across students (professor effects). Professors who elicited unusually good evaluations in some students also elicited better memory for lectures in those students. Conclusions: It appears possible to forecast relationship and professor effects on teaching evaluations by presenting brief teaching trailers to students. Thus, it might be possible to develop online recommender systems to help match students and professors so that unusually effective teaching emerges.</p> <p>Teaching Evaluations; SRM; G theory; Person Perception; RRT</p> <p>Nearly all colleges and universities in the United States use students' evaluations of teaching as part of tenure and promotion decisions. Many measures of students' evaluations have impressive validity. For example, professors' scores on students' evaluations correlate substantially with students' learning, at least when assessed in courses with standardized examinations and content (Cohen, [<reflink idref="bib6" id="ref1">6</reflink>] ; Feldman, [<reflink idref="bib10" id="ref2">10</reflink>] ; Marsh, [<reflink idref="bib24" id="ref3">24</reflink>] , [<reflink idref="bib25" id="ref4">25</reflink>] ). Furthermore, there is a reasonable agreement between current students, faculty, administrators, and alumni about which professors are most effective (Centra, [<reflink idref="bib5" id="ref5">5</reflink>] ; Feldman, [<reflink idref="bib11" id="ref6">11</reflink>] ; Marsh, [<reflink idref="bib24" id="ref7">24</reflink>] , [<reflink idref="bib25" id="ref8">25</reflink>] ). However, teaching evaluations have not yet been used to help match students and professors so that unusually effective teaching emerges. The goal of the current research was to test whether brief video trailers of professors' teaching can forecast students' evaluations of live lectures months later. We envision an online system similar in some ways to recommender systems used by Amazon.com and itunes that make individualized recommendations for music or book purchases. As applied to college teaching, a student would view and rate brief videos of professors' teaching and would then be given individualized feedback about which of these professors the student would find especially effective, based on the students' ratings as well as the ratings of other students. We believe such forecasting would be useful, regardless of whether the reader is persuaded of the construct validity of students' evaluations of teaching. If teaching evaluations reflect student learning, then forecasting evaluations should improve student learning by helping each student choose professors who are uniquely effective for the student. If teaching evaluations reflect only consumer satisfaction, then forecasting evaluations should lead to more satisfied students.</p> <p>When a student rates a professor's effectiveness, the rating reflects at least three distinct influences (Gross, Lakey, Edinger, Orehek, & Heffron, [<reflink idref="bib13" id="ref9">13</reflink>] ). Part of the student's rating reflects the objective effectiveness of the professor, as reflected in inter‐rater agreement among observers that some professors are more effective than others (professor effects). For example, raters might consistently give Professor Hendersen higher ratings than Professor Duren. Professors' effectiveness (as measured by student‐rater agreement) is the most widely studied aspect of student evaluations of teaching, and there is a good agreement about effectiveness across observers (Centra, [<reflink idref="bib5" id="ref10">5</reflink>] ; Feldman, [<reflink idref="bib11" id="ref11">11</reflink>] ; Marsh, [<reflink idref="bib24" id="ref12">24</reflink>] , [<reflink idref="bib25" id="ref13">25</reflink>] ). A second determinant of a student's rating is rater bias and occurs when some students characteristically rate the same professors more favourably than do other students, regardless of the actual characteristics of professors. For example, when Ellen and Will rate the same professors, Ellen gives higher scores than does Will, on average. A third determinant of a student's ratings are relationship effects (Kenny, [<reflink idref="bib14" id="ref14">14</reflink>] ). Relationship effects occur when a student rates a professor (<reflink idref="bib1" id="ref15">1</reflink>) more favourably than the student typically rates professors (rater bias) and (<reflink idref="bib2" id="ref16">2</reflink>) more favourably than the professor is typically rated by other students (professor effects). That is, in an ANOVA model, relationship effects are rater × professor interactions. For example, Ellen might rate Professor Hendersen more favourably than one would expect, given (<reflink idref="bib1" id="ref17">1</reflink>) Ellen's tendency to rate professors leniently (rater bias) and (<reflink idref="bib2" id="ref18">2</reflink>) Professor Hendersen's tendency to elicit favourable ratings from students on average. Thus, relationship effects reflect the extent to which a professor is unusually effective for a specific student.</p> <p>There is a good reason to expect relationship effects in college teaching as research using generalizability theory (Cronbach, Gleser, Nanda, & Rajaratnam, [<reflink idref="bib9" id="ref19">9</reflink>] ) and the social relations model (SRM; Kenny, [<reflink idref="bib14" id="ref20">14</reflink>] ; Kenny, Kashy, & Cook, [<reflink idref="bib16" id="ref21">16</reflink>] ) has identified relationship effects in a wide range of human judgments, including leadership (Livi, Kenny, Albright, & Pierro, [<reflink idref="bib21" id="ref22">21</reflink>] ), personality (Park, Kraus, & Ryan, [<reflink idref="bib27" id="ref23">27</reflink>] ), social support (Lakey & Orehek, [<reflink idref="bib20" id="ref24">20</reflink>] ), physician's cultural competency (Lucas, Lakey, Arnetz, & Arnetz, [<reflink idref="bib22" id="ref25">22</reflink>] ), psychotherapist qualities (Lakey, Cohen, & Neely, [<reflink idref="bib19" id="ref26">19</reflink>] ), family negativity (Cook, Kenny, & Goldstein, [<reflink idref="bib8" id="ref27">8</reflink>] ), and parent–child attachment (Cook, [<reflink idref="bib7" id="ref28">7</reflink>] ).</p> <p>Large relationship effects have also been observed in students' evaluations of college teaching and quiz performance (Gross et al., [<reflink idref="bib13" id="ref29">13</reflink>] ). In Study 1, undergraduate and graduate students rated their professors after completing full semester courses. In Study 2, a new sample of students rated live lectures. In Study 3, another group of students rated video‐recorded lectures. Large relationship effects on teaching evaluations were found in all studies. On average (median), relationship effects accounted for 52% of the variance in students' evaluations of professors' teaching. Moreover, Studies 2 and 3 included quizzes based on each lecture, and relationship effects on teaching effectiveness were linked to relationship effects on memory for lecture. That is, when a student rated a lecture as unusually effective, he or she also performed unusually well on the quiz. Gross et al. ([<reflink idref="bib13" id="ref30">13</reflink>] ) also estimated professor and rater effects. Consistent with the research on inter‐rater agreement on teaching evaluations (Centra, [<reflink idref="bib5" id="ref31">5</reflink>] ; Feldman, [<reflink idref="bib11" id="ref32">11</reflink>] ; Marsh, [<reflink idref="bib24" id="ref33">24</reflink>] , [<reflink idref="bib25" id="ref34">25</reflink>] ), professor effects accounted for 30% of the variance. Rater bias accounted for 17%. In short, personal tastes (relationship effects) account for the largest variance in students' ratings and forecast learning in the classroom.</p> <p>Forecasting which student will find which professor unusually effective requires methods that can isolate relationship effects from professor and rater effects, and predictive accuracy must be established separately for each. First, we describe the simpler cases of predicting perceived teaching effectiveness from professor and rater effects. To predict how students on average would respond to a specific professor, one could base prediction on professor effects. If the results of teaching evaluations were available for every professor, students should select the professor with the highest evaluations. If teaching evaluations derived from real courses were not publically available, prediction could be based on students' averaged ratings of ‘teaching trailers'. To predict how a given student would evaluate all professors on average (rater bias), one could take each student's average rating across all teaching trailers. Predicting rater bias is not likely to be useful in practice. Yet, as described in the discussion, understanding rater bias could be useful when using teaching evaluations for personnel decisions. For example, it would be helpful to know whether some majors were more generous raters than other majors. To forecast how the student Ellen would uniquely respond to Professor Hendersen (relationship effects), one would take Ellen's reaction to Hendersen's teaching trailer, with Hendersen's average score across all students (cf. professor effects) and Ellen's average rating across all trailers (cf. rater effects) removed.</p> <p>Research on other constructs suggests that it should be possible to forecast relational teaching evaluations. For example, Veenstra et al. ([<reflink idref="bib28" id="ref35">28</reflink>] ) attempted to forecast relational perceived support on the basis of brief conversations between support providers and recipients, as well as brief video interviews with each provider. First, each recipient viewed a video recording of an interview with each provider. Next, each recipient had a brief conversation with each provider. Recipients and providers continued to meet for several weeks (Study 1) or several months (Study 2). Veenstra et al. ([<reflink idref="bib28" id="ref36">28</reflink>] ) could forecast which provider a recipient would ultimately see as unusually supportive on the basis of a brief conversation at r ≈ .45, but not from the video interview. Similarly, Park et al. ([<reflink idref="bib27" id="ref37">27</reflink>] ) found that relationship effects on personality were somewhat stable over time, suggesting it should be possible to forecast them. Still, predictive accuracy for relationship effects might not be as strong as for professor and rater effects. Kenny's ([<reflink idref="bib15" id="ref38">15</reflink>] ) quantitative theory of person perception predicts that professor (i.e., target) effects should be predicted with very high accuracy (r ≈ .90). Similarly, rater effects should be predictable with excellent accuracy as Veenstra et al. ([<reflink idref="bib28" id="ref39">28</reflink>] ) forecasted rater effects in perceived social support with high accuracy (r ≈ .90).</p> <p>The goal of this study was to explore the viability of using video trailers of professors' teaching to forecast which students would respond unusually well to specific professors' teaching. Early in the semester, students viewed brief video trailers depicting professors' teaching. Students rated the effectiveness of each professor's teaching in the trailer, and students rated their own affect during the trailer. We included affect because students' affect has been strongly linked to teaching ratings (Fortunato & Mincy, [<reflink idref="bib12" id="ref40">12</reflink>] ; Gross et al., [<reflink idref="bib13" id="ref41">13</reflink>] ). These measures of perceived teaching effectiveness and experienced affect were used to forecast students' evaluations of live lectures later in the semester. Consistent with Gross et al. ([<reflink idref="bib13" id="ref42">13</reflink>] ), we hypothesized (<reflink idref="bib1" id="ref43">1</reflink>) significant relationship, rater, and professor effects on student evaluations of teaching and on students' affect experienced when watching the video trailers and live lectures and (<reflink idref="bib2" id="ref44">2</reflink>) significant correlations between relational teaching evaluations and relational memory for lectures. Consistent with Veenstra et al. ([<reflink idref="bib28" id="ref45">28</reflink>] ), we hypothesized that (<reflink idref="bib3" id="ref46">3</reflink>) relationship effects on evaluations and experienced affect in response to the trailers would forecast relationship effects on teaching evaluations during the live lecture.</p> <hd id="AN0101003818-2">Method</hd> <hd id="AN0101003818-3">Participants</hd> <p>One hundred and forty‐five college students (74% female; mean age = 19) from three sections of introductory psychology taught by JG completed all measures during the course of a semester. Isolating relationship effects requires a design in which students rate the same professors (Kenny, [<reflink idref="bib14" id="ref47">14</reflink>] ; Kenny et al., [<reflink idref="bib16" id="ref48">16</reflink>] ). Gathering complete data from students required their participation during seven classes in each of sections 1 (N = 47) and 2 (N = 50) and 10 classes in section 3 (N = 48). Still, 78% of students enrolled participated in all sessions. Students missed classes for a variety of reasons including a required retreat for an academic programme, illness, weather‐related travel difficulties, and off‐campus games for athletes. Our analytic procedures are intolerant to missing data, and so, students with incomplete data were excluded from the analyses.</p> <p>Guest lectures and teaching trailers were obtained from 10, tenure‐track professors (80% male; mean age = 44; range = 33–64) from a medium‐sized state university in the Great Lakes region of the United States. All professors were native English speakers. Three professors taught in the first section, three in the second section, and four in the third section. Students viewed the trailers only for the professors who gave guest lectures in the students' section. One professor taught in two sections. We retained the professor in both the sections to boost statistical power for professor effects, after determining that this professor's appearance in both the sections did not influence the results.</p> <hd id="AN0101003818-4">Procedure</hd> <p>Students were shown 6‐min video trailers of each professor. Students rated their own affect and the effectiveness of each professor's teaching in response to each trailer. Later in the semester (median = 8 weeks, range = 3–12), students heard a 40‐min live lecture by each professor. Students again evaluated teaching and rated their own affect. Students completed a quiz on each lecture during the next class period.</p> <hd id="AN0101003818-5">Teaching trailers</hd> <p>In the year preceding the study, each professor was video‐recorded teaching a representative 50‐min class. The authors independently viewed each recorded lecture and identified the key features of each professor's style (e.g., sarcastic humour, enthusiasm, quick pace, eye contact, confidence). In subsequent meetings, a consensus description of each professor's style was developed. Finally, video passages were identified that reflected the consensus description and were compiled into the final 6‐min trailers.</p> <hd id="AN0101003818-6">Measures</hd> <hd id="AN0101003818-7">Teaching evaluations</hd> <p>Students completed the widely used Students' Evaluations of Educational Quality (Marsh, [<reflink idref="bib23" id="ref49">23</reflink>] ), modified for use with teaching trailers (18 items) and live lectures (24 items). For example, the item ‘You found the course intellectually challenging and stimulating' was modified to read ‘You found the lecture intellectually challenging and stimulating.' For all measures, we calculated internal consistency separately for each of the effects of interest.[<reflink idref="bib1" id="ref50">1</reflink>] For the trailers, internal consistency reliability was.97 for rater,.99 for professor, and.93 for relationship effects. For live lectures, reliability was.98 for rater,.99 for professor, and.95 for relationship effects.</p> <hd id="AN0101003818-8">Affect</hd> <p>After each trailer and lecture, students completed the 20‐item, Positive and Negative Affectivity Schedule (Watson, Clark, & Tellegen, [<reflink idref="bib29" id="ref51">29</reflink>] ), perhaps the most widely used measure of affect in psychology. The measure includes two mostly independent subscales of positive and negative affect. Example items include ‘interested' and ‘excited' for positive affect and ‘distressed' and ‘nervous' for negative affect. For positive affect, reliability was.96 for rater,.99 for professor, and.89 for relationship effects (trailers), and.98 for rater,.99 for professor, and.90 for relationship effects (live lectures). For negative affect, reliability was.96 for rater,.62 for professor, and.90 for relationship effects (trailers), and.95 for rater,.93 for professor, and.79 for relationship effects (live lectures).</p> <hd id="AN0101003818-9">Quizzes</hd> <p>Students' memory for live lectures was assessed by 12‐item multiple‐choice quizzes administered during the next class period. Reliability was.91 for rater,.70 for professor, and.34 for relationship effects. Although there was much more random error in the relationship effect than desired, as described momentarily, the effect was still able to replicate the link between teaching evaluations and quiz performance reported by Gross et al. ([<reflink idref="bib13" id="ref52">13</reflink>] ).</p> <hd id="AN0101003818-10">Statistical analyses</hd> <p>First, we determined the extent to which there were rater, professor, and relationship effects for each of the study constructs. We analysed the data as a students × professors (nested within sections) × item design in VARCOMP within SPSS. Each factor was random. Students and professors were nested within sections, and each factor was crossed with items. Each section formed a level of the sections factor, each student formed a level of the raters factor, each professor formed a level of the professors factor, and each item aggregate formed a level of the items factor. We constructed two aggregates of odd and even items for each construct to reduce measurement error and the size of the design (Gross et al., [<reflink idref="bib13" id="ref53">13</reflink>] ). We analysed the study as a nested design to obtain better statistical power for professor effects. Analysing each section separately has the disadvantage of only 3 or 4 professors per study, whereas the nested design had 10 professors. We also analysed each section's data separately, and the results were very similar to those of the nested design. Analyses of each section's data are available by request. The design yielded nine effects: raters nested within section (i.e., raters:sections), professors:sections, items, sections, raters:sections × items, professors:sections × items, items × sections, raters:sections × professors:sections, and raters:sections × professors:sections × items. Relationship effects are reflected in the raters:sections × professors:sections effect. Effects involving items are typically viewed as measurement error and thus are not reported.</p> <p>Correlations between constructs were estimated by first calculating rater, professor, and relationship scores following the examples of Cook and Kenny ([<reflink idref="bib15" id="ref54">15</reflink>] ); Kwan, John, Kenny, Bond, and Robins ([<reflink idref="bib17" id="ref55">17</reflink>] ); and Kwan, John, Robins, and Kuang ([<reflink idref="bib18" id="ref56">18</reflink>] ). Professor scores were simply the mean score of each professor averaged across raters and items (N = 10). Rater scores were simply the mean score of each rater averaged across professors and items (N = 145). Relationship scores (N = 483) were calculated using the formula Rel<subs>ij</subs> = X<subs>ij</subs> − MRater<subs>i</subs> − MProfessor<subs>j</subs>, for which X<subs>ij</subs> indicates a score on a variable for rater i and professor j, MRater<subs>i</subs> is rater i's mean score across professors, and MProfessor<subs>j</subs> is professor j's mean score across raters. Relationship scores were averaged across items. We did not adjust for section in calculating these scores as there were no effects for section on the study variables. Once rater, professor, and relationship scores were calculated, we used conventional correlation and regression analysis. We used percentile bootstrapping with 1,000 resamples to estimate statistical significance for correlations based on professor and relationship scores as these scores violated the independence of observations assumption. We used parametric significant tests for rater scores.</p> <p>In our team's previous analyses (e.g., Gross et al., [<reflink idref="bib13" id="ref57">13</reflink>] ), we estimated multivariate generalizability correlations (Cronbach et al., [<reflink idref="bib9" id="ref58">9</reflink>] ) between constructs using the software Mgenova (Brennan, [<reflink idref="bib4" id="ref59">4</reflink>] ). Although appropriate for the study's design, Mgenova has two disadvantages. Statistical control is cumbersome, and significance tests must be bootstrapped by hand. Fortunately, the results of the analyses just described and the results from Mgenova yielded identical results.[<reflink idref="bib2" id="ref60">2</reflink>]</p> <hd id="AN0101003818-11">Results</hd> <p>First, we attempted to replicate Gross et al.'s ([<reflink idref="bib13" id="ref61">13</reflink>] ) findings of large relationship, professor, and rater effects on teaching evaluations and on affect, as well as relationship effects on quiz performance.</p> <p>As predicted, there were large relationship effects for teaching evaluations for both trailers and live lectures, with each accounting for about 40% of the variance (Table [NaN] ). That is, some professors elicited unusually favourable evaluations from some raters, more favourable than how the rater typically evaluated professors and more favourable than how the professor was typically evaluated by others. There were also large relationship effects for both positive and negative affect, with each accounting for more than 30% of the variance. That is, some professors elicited unusually favourable affect in some raters, but not others. There were also significant relationship effects on quiz scores. Some professors elicited unusually good memory for lectures in some students, but not others.</p> <p>Variance components, standard errors, and effect sizes for study variables</p> <p> <ephtml> <table><tr><th align="left" /><th align="left">Variance component</th><th align="left">Standard error</th><th align="left">Proportion of variance explained</th></tr><tr><td align="left">Teaching evaluations (trailer)</td></tr><tr><td align="left">Rater</td><td align="left">.105</td><td align="left">.022</td><td align="left">.182</td></tr><tr><td align="left">Professor</td><td align="left">.201</td><td align="left">.099</td><td align="left">.347</td></tr><tr><td align="left">Relationship</td><td align="left">.226</td><td align="left">.019</td><td align="left">.392</td></tr><tr><td align="left">Positive affect (trailer)</td></tr><tr><td align="left">Rater</td><td align="left">.318</td><td align="left">.051</td><td align="left">.378</td></tr><tr><td align="left">Professor</td><td align="left">.156</td><td align="left">.079</td><td align="left">.185</td></tr><tr><td align="left">Relationship</td><td align="left">.264</td><td align="left">.023</td><td align="left">.314</td></tr><tr><td align="left">Negative affect (trailer)</td></tr><tr><td align="left">Rater</td><td align="left">.091</td><td align="left">.018</td><td align="left">.306</td></tr><tr><td align="left">Professor</td><td align="left">.000</td><td align="left">.002</td><td align="left">.001</td></tr><tr><td align="left">Relationship</td><td align="left">.157</td><td align="left">.014</td><td align="left">.527</td></tr><tr><td align="left">Teaching evaluations (class)</td></tr><tr><td align="left">Rater</td><td align="left">.098</td><td align="left">.021</td><td align="left">.184</td></tr><tr><td align="left">Professor</td><td align="left">.191</td><td align="left">.093</td><td align="left">.359</td></tr><tr><td align="left">Relationship</td><td align="left">.215</td><td align="left">.018</td><td align="left">.404</td></tr><tr><td align="left">Positive affect (class)</td></tr><tr><td align="left">Rater</td><td align="left">.366</td><td align="left">.060</td><td align="left">.351</td></tr><tr><td align="left">Professor</td><td align="left">.181</td><td align="left">.092</td><td align="left">.174</td></tr><tr><td align="left">Relationship</td><td align="left">.369</td><td align="left">.032</td><td align="left">.354</td></tr><tr><td align="left">Negative affect (class)</td></tr><tr><td align="left">Rater</td><td align="left">.090</td><td align="left">.015</td><td align="left">.418</td></tr><tr><td align="left">Professor</td><td align="left">.003</td><td align="left">.003</td><td align="left">.014</td></tr><tr><td align="left">Relationship</td><td align="left">.070</td><td align="left">.007</td><td align="left">.324</td></tr><tr><td align="left">Quiz (class)</td></tr><tr><td align="left">Rater</td><td align="left">.009</td><td align="left">.002</td><td align="left">.148</td></tr><tr><td align="left">Professor</td><td align="left">.008</td><td align="left">.006</td><td align="left">.123</td></tr><tr><td align="left">Relationship</td><td align="left">.008</td><td align="left">.002</td><td align="left">.115</td></tr></table> </ephtml> </p> <ulist> <item>3 Note</item> <item>4 p < .05. The variance components for all variables for the section factor were zero and not significant.</item> </ulist> <p>There was a substantial professor effect (i.e., inter‐rater agreement) on teaching effectiveness for both trailers and live lectures, accounting for about 35% of the variance (Table [NaN] ). That is, raters agreed to a large extent that some professors were more effective than others. In addition, some professors consistently elicited more positive affect in raters than did other professors. There were no professor effects for negative affect or quiz scores.</p> <p>There were large rater effects for teaching evaluations for both trailers and live lectures, accounting for nearly 20% of the variance (Table [NaN] ). That is, some raters consistently gave professors high scores, and other raters consistently gave professors low scores. Similarly, there were large rater effects for positive and negative affect for both trailers and live lectures, accounting for more than 35% of the variance. Some raters consistently reported high positive or low negative affect across professors and time. Finally, there were significant rater effects for quizzes. Some students had consistently higher scores across quizzes than did other students.</p> <p>Our primary question was, could we forecast from teaching trailers the students who responded unusually well to specific professors' live lectures (i.e., relationship effects)? As predicted, relational teaching evaluations and positive affect in response to trailers significantly forecasted evaluations of live lectures (Table [NaN] ). That is, when a student rated a professor's trailer unusually favourably, or experienced unusually high positive affect, the student also rated the professor's live lecture unusually well. In multiple regression analyses, teaching evaluations and positive affect in response to the trailers forecasted 7% (R = .27) of the variance in live lectures. Positive affect uniquely forecasted ratings of live lectures (β = .18; p < .05; ΔR<sups>2</sups> = .02), but evaluations of trailers did not (β = .11; n.s.; ΔR<sups>2</sups> = .01). The small predictive accuracy of the unique predictors (compared with the full equation) shows that most of the predictive accuracy was shared between evaluations and positive affect. Negative affect during trailers did not forecast evaluations of live lectures.</p> <p>Correlations between constructs for relationship, professor, and rater effects</p> <p> <ephtml> <table><tr><th align="left" /><th align="left">Positive affect (trailer)</th><th align="left">Negative affect (trailer)</th><th align="left">Teaching evaluations (class)</th><th align="left">Positive affect (class)</th><th align="left">Negative affect (class)</th><th align="left">Quiz (class)</th></tr><tr><td align="left">Teaching evaluations (trailers)</td></tr><tr><td align="left">Rater</td><td align="left">.55</td><td align="left">−.03</td><td align="left">.60</td><td align="left">.39</td><td align="left">−.08</td><td align="left">.17</td></tr><tr><td align="left">Professor</td><td align="left">.98</td><td align="left">NC</td><td align="left">.86</td><td align="left">.74</td><td align="left">NC</td><td align="left">NC</td></tr><tr><td align="left">Relationship</td><td align="left">.72</td><td align="left">−.22</td><td align="left">.24</td><td align="left">.23</td><td align="left">−.09</td><td align="left">.08</td></tr><tr><td align="left">Positive affect (trailers)</td></tr><tr><td align="left">Rater</td><td align="left">−</td><td align="left">.34</td><td align="left">.36</td><td align="left">.76</td><td align="left">.16</td><td align="left">.14</td></tr><tr><td align="left">Professor</td><td align="left">−</td><td align="left">NC</td><td align="left">.87</td><td align="left">.76</td><td align="left">NC</td><td align="left">NC</td></tr><tr><td align="left">Relationship</td><td align="left">−</td><td align="left">−.15</td><td align="left">.26</td><td align="left">.30</td><td align="left">−.07</td><td align="left">.08</td></tr><tr><td align="left">Negative affect (trailers)</td></tr><tr><td align="left">Rater</td><td align="left" /><td align="left">−</td><td align="left">.00</td><td align="left">.34</td><td align="left">.34</td><td align="left">−.09</td></tr><tr><td align="left">Professor</td><td align="left" /><td align="left">−</td><td align="left">NC</td><td align="left">NC</td><td align="left">NC</td><td align="left">NC</td></tr><tr><td align="left">Relationship</td><td align="left" /><td align="left">−</td><td align="left">−.03</td><td align="left">−.03</td><td align="left">−.03</td><td align="left">−.08</td></tr><tr><td align="left">Teaching evaluations (class)</td></tr><tr><td align="left">Rater</td><td align="left" /><td align="left" /><td align="left">−</td><td align="left">.51</td><td align="left">−.13</td><td align="left">.19</td></tr><tr><td align="left">Professor</td><td align="left" /><td align="left" /><td align="left">−</td><td align="left">.94</td><td align="left">NC</td><td align="left">NC</td></tr><tr><td align="left">Relationship</td><td align="left" /><td align="left" /><td align="left">−</td><td align="left">.69</td><td align="left">−.22</td><td align="left">.16</td></tr><tr><td align="left">Positive affect (class)</td></tr><tr><td align="left">Rater</td><td align="left" /><td align="left" /><td align="left" /><td align="left">−</td><td align="left">.33</td><td align="left">.08</td></tr><tr><td align="left">Professor</td><td align="left" /><td align="left" /><td align="left" /><td align="left">−</td><td align="left">NC</td><td align="left">NC</td></tr><tr><td align="left">Relationship</td><td align="left" /><td align="left" /><td align="left" /><td align="left">−</td><td align="left">−.18</td><td align="left">.12</td></tr><tr><td align="left">Negative affect (class)</td></tr><tr><td align="left">Rater</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left">−</td><td align="left">−.09</td></tr><tr><td align="left">Professor</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left">−</td><td align="left">NC</td></tr><tr><td align="left">Relationship</td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left">−</td><td align="left">−.02</td></tr></table> </ephtml> </p> <ulist> <item>5 Note</item> <item>6 p < .05. N = 145 for rater correlations, N = 10 for professor correlations, and N = 483 for relationship correlations. NC, not calculated as at least one of the variance components was not significant.</item> </ulist> <p>For professor effects, teaching evaluations and positive affect in response to trailers predicted evaluations of live lectures with superb accuracy (Table [NaN] ). That is, the professors who elicited consensus favourable evaluations and positive affect in trailers also elicited consensus favourable evaluations of live lectures. There were no significant professor effects for negative affect for either trailers or live lectures, and so, negative affect was not used in prediction. Evaluations and positive affect in response to trailers were nearly perfectly correlated. That is, for trailers, the consensus about teaching quality was the same empirically as the consensus about elicited positive affect. Multiple regression indicated that evaluations and positive affect from trailers predicted 76% of the variance in evaluations of live lectures (R = .87). Given how highly intercorrelated the two predictors were, and the limited sample size of professors, we did not estimate which predictor had unique predictive power.</p> <p>Although not the primary focus of our research, we report predictive accuracy for rater effects for the sake of completeness. For rater effects, we could forecast evaluations of live lectures with excellent accuracy from evaluations and positive affect in response to trailers (Table [NaN] ). That is, raters who typically evaluated trailers favourably also evaluated live lectures favourably. Raters who typically responded to trailers with positive affect also typically evaluated live lectures favourably. Multiple regression analyses showed that evaluations of trailers predicted evaluations of live lectures significantly (β = .58; ΔR<sups>2</sups> = .24; t = 7.24; p < .05), but positive affect did not (β = .04; ΔR<sups>2</sups> = .00; t = 0.47; n.s.). The two constructs together forecasted 36% of the variance in teaching evaluations (R = .60). In contrast, negative affect in response to trailers did not forecast evaluations of live lectures. Raters who typically evaluated teaching trailers positively also had higher scores on the quizzes.</p> <p>Finally, there were a number of interesting cross‐sectional findings (Table [NaN] ). Perhaps most important, relationship effects on teaching evaluations and positive affect for live lectures were linked to relational quiz scores. That is, when a student saw a professor's lecture as unusually effective, or the professor elicited unusually high positive affect in a student, the student also scored especially well on the corresponding quiz. Other effects included that teaching evaluations were strongly linked to positive affect for each of the three effects for trailers and live lectures. For professor effects, the link between teaching evaluations and positive affect indicated that inter‐rater agreement about which professors were more effective overlapped substantially with inter‐rater agreement about which professors elicited more positive affect. For relationship effects, when a rater evaluated a professor unusually favourably, the professor also elicited unusual favourable affect. For rater effects, raters who typically responded to all professors with positive affect also typically evaluated all professors favourably. In contrast, negative affect was linked to poor evaluations only for relationship effects. When a professor elicited unusually high negative affect, the rater saw the professor as unusually ineffective, for both trailers and live lectures.</p> <hd id="AN0101003818-12">Discussion</hd> <p>We could forecast the professor that a student would see as unusually effective in a live lecture from the student's reactions to 6‐min video trailers of the professor (i.e., relationship effects). We could also forecast from trailers which professors would be seen as more effective (on average across students) than other professors in live lectures (i.e., professor effects). These findings support the possibility of developing online systems that would provide personalized recommendations that specific students take courses from specific professors.[<reflink idref="bib3" id="ref62">3</reflink>] This matching has the potential to improve teaching effectiveness and student satisfaction. It is important to forecast both relationship and professor effects, because each has a distinct link to student learning. In the present study and Gross et al.'s ([<reflink idref="bib13" id="ref63">13</reflink>] ) studies 2 and 3, relational memory for lecture was linked to relational student evaluations of teaching effectiveness. That is, when a student evaluated a professor's lecture as unusually effective, the student remembered an unusually large amount about the lecture. It is already well established that professors with higher teaching evaluations averaged across students (i.e., professor effects) elicit more student learning (Cohen, [<reflink idref="bib6" id="ref64">6</reflink>] ; Feldman, [<reflink idref="bib10" id="ref65">10</reflink>] ; Marsh, [<reflink idref="bib24" id="ref66">24</reflink>] , [<reflink idref="bib25" id="ref67">25</reflink>] ).</p> <p>That we could forecast relationship effects for teaching effectiveness is consistent with the research on forecasting relational perceived support. Relationship effects are especially large in perceived support, and thus, support interventions might be improved if support providers could be matched to support recipients such that unusually supportive relationships emerged. Veenstra et al. ([<reflink idref="bib28" id="ref68">28</reflink>] ) investigated the predictive accuracy of (<reflink idref="bib1" id="ref69">1</reflink>) very brief conversations between recipients and providers and (<reflink idref="bib2" id="ref70">2</reflink>) recipients' reactions to short video interviews of providers. In social support interventions, it would be more cost effective for recipients to view recorded interviews of providers rather than to have brief conversations with every potential provider. Veenstra et al. ([<reflink idref="bib28" id="ref71">28</reflink>] ) could forecast relational perceived support several months in advance from brief conversations, but not from video interviews.</p> <p>The accuracy of forecasting relational teaching evaluations was comparable to forecasting job performance from conscientiousness – one of the strongest predictors of job performance (Oh, Wang, & Mount, [<reflink idref="bib26" id="ref72">26</reflink>] ). Nonetheless, the accuracy of relational forecasting was small compared to forecasting based on professor and rater effects. Veenstra et al. ([<reflink idref="bib28" id="ref73">28</reflink>] ) also found that rater effects were more predictable than relationship effects. We suspect these differences in predictability result because professor and rater effects have stability as part of their operational definitions. By definition, professor effects are stable across raters and rater effects are stable across professors. In contrast, relationship effects are unstable across professors and raters.</p> <p>The very high predictive accuracy of professor effects was predicted by Kenny's ([<reflink idref="bib15" id="ref74">15</reflink>] ) quantitative theory of person perception (PERSON) and is consistent with Park et al.'s ([<reflink idref="bib27" id="ref75">27</reflink>] ) study of the longitudinal stability of target effects. According to PERSON, inter‐rater agreement reaches asymptote very quickly as more information about targets is presented. Agreement is driven by shared stereotypes initially (e.g., extroverted professors are better), but agreement based on targets' actions quickly dominates. Thus, it takes surprisingly few observations to achieve asymptotic agreement, and the video trailers apparently achieved this.</p> <p>Our findings are also consistent with ‘thin slices' research (Ambady & Rosenthal, [<reflink idref="bib1" id="ref76">1</reflink>] , [<reflink idref="bib2" id="ref77">2</reflink>] ; Babad, Avni‐Babad, & Rosenthal, [<reflink idref="bib3" id="ref78">3</reflink>] ), in which raters' evaluations of brief (<5 min) exposure to targets forecasted information derived from other sources (e.g., teaching evaluations). Thin slices research on teaching reflects professor effects, because professors' scores are averaged across raters and professors are the unit of analysis (Kenny, [<reflink idref="bib15" id="ref79">15</reflink>] ). Our ability to forecast professor effects was much more precise (r = .86) than obtained in the thin slices research, where the meta‐analytic estimate was r = .41 (Ambady & Rosenthal, [<reflink idref="bib1" id="ref80">1</reflink>] ). According to Kenny's ([<reflink idref="bib15" id="ref81">15</reflink>] ) PERSON model, the stronger predictive accuracy for professors in the current study likely reflected that the same raters viewed both trailers and lectures, whereas in most thin slices research, teaching evaluations and reactions to thin slices are provided by different raters. According to PERSON, the stability of professor effects is suppressed when raters have access to different information. The impressive predictive accuracy for professor scores in the current study might have also resulted from the great care taken in constructing the trailers to maximize the representativeness of professors' teaching.</p> <p>Our findings of strong and significant rater, professor, and relationship effects for teaching evaluations replicate previous studies (Gross et al., [<reflink idref="bib13" id="ref82">13</reflink>] ) and have implications for interpreting teaching evaluations in educational settings. Teaching evaluations are typically interpreted as though they reflect only professor effects (i.e., the extent to which some professors are better than others). Yet, only a portion of teaching evaluations reflects this effect. Other important influences are rater bias and relationship effects. The presence of strong rater and relationship effects opens the door for decision errors when teaching evaluations are treated as though they reflect only professor effects. For example, if the tendency of some students to rate all professors favourably (rater bias) is correlated with major, then professors in certain departments will have inflated teaching evaluations. Relationship effects could also bias teaching evaluations. If a professor is better at teaching advanced students than introductory students, he or she might have unusually low scores if assigned to teach many sections of introductory courses. The variance partitioning approach integral to generalizability theory (Cronbach et al., [<reflink idref="bib9" id="ref83">9</reflink>] ) and the SRM (Kenny, [<reflink idref="bib14" id="ref84">14</reflink>] ; Kenny et al., [<reflink idref="bib16" id="ref85">16</reflink>] ) informs personnel decisions by showing that professors' scores on teaching evaluations do not primarily reflect the simple case that some professors are better than others.</p> <p>The current research replicated previous findings (Fortunato & Mincy, [<reflink idref="bib12" id="ref86">12</reflink>] ; Gross et al., [<reflink idref="bib13" id="ref87">13</reflink>] ) that teaching evaluations are strongly linked to positive affect. That is, students rate professors as effective insofar as professors elicit positive affect. Yet, the psychological mechanisms differ for relationship, professor, and rater effects. Relationship effects reflect the unique match between specific professors and students. When a professor elicits unusually positive affect in a student, that student rates the professor as unusually effective. Rater effects reflect trait like personality processes in that the students who characteristically experience positive affect also characteristically rate professors as effective. Professor effects reflect the trait like personality characteristics of professors. Professors who consistently elicit positive affect in students also consistently elicit favourable teaching evaluations.</p> <p>We interpret findings for relationship effects on teaching evaluations, affect, and memory in terms of relational regulation theory (Lakey & Orehek, [<reflink idref="bib20" id="ref88">20</reflink>] ). According to this theory, social interaction is a key way by which people regulate their own affect and cognition, and this regulation is largely relational. From this perspective, some professors are unusually effective in regulating some students' positive affect and memory and are rewarded with high teaching evaluations. This does not mean, however, that eliciting positive affect is not an important part of teaching, as positive affect includes attentiveness and interest.</p> <p>We should note some limitations of the current study. First, we studied only psychology lectures and the results might not generalize to other fields. Second, we studied only 40‐min lectures and the results might not generalize to semester‐long courses. It will be important to determine whether responses to trailers will forecast teaching evaluations based on entire‐semester courses. This is especially important as teaching evaluations might be based, in part, on interactions with the professor outside of the classroom (e.g., office hours); the teaching trailers developed for the present study did not reflect extra‐classroom behaviour. Third, preparing carefully constructed trailers as in the present study might be too expensive to justify the cost‐effectiveness of forecasting students' responses to teaching. It will be important to determine the extent to which very small slices (Ambady & Rosenthal, [<reflink idref="bib1" id="ref89">1</reflink>] ) of professors' actions can forecast students' relational responses to specific professors. Fourth, professors were confounded with topic. For example, one professor spoke about psychopathology and another lectured on text comprehension. Thus, relational effects might reflect students' reactions to different topics as much as reactions to different professors. However, it would be difficult to disentangle student × professor interactions (i.e., relationship effects) from student × topic interactions. To do so, one would need a design in which professors and topics were fully crossed and each professor taught each topic. However, having students hear different faculty members give the same lecture would be sufficiently strange as to present another set of interpretive problems. Fifth, our statistical procedures require no missing data, and thus, the small numbers of students who missed a session were excluded from analyses. It is possible that students who missed sessions differed in some important way from students who attended all sessions. For example, students lower in conscientiousness might be under‐represented in the data and such students might not show the same findings as the full sample. Sixth, memory for lecture was assessed with multiple‐choice questions in the next class session. Responses to such questions might not generalize to performance on essay examinations or papers and might not last beyond a few days. Finally, the current results need replication.</p> <p>In conclusion, it appears possible to forecast from brief video trailers, the extent to which an individual student will respond unusually well to a specific professor, as well as how students on average will respond to different professors. These findings suggest the feasibility of developing online systems that would recommend that a student take courses from specific professors. Such an approach might improve student satisfaction and the effectiveness of instruction.</p> <ref id="AN0101003818-13"> <title>Footnotes</title> <blist> <bibl id="bib1" idref="ref15" type="bt">1</bibl> <bibtext>Internal consistency reliability was estimated using the following formulas derived from generalizability theory (Cronbach et al., 9) for which r = rater, p = professor, i = item, r × p = rater by professor (i.e., relationship effects), r × i = rater by item, p × i = professor by item, r × p × i = rater × professor × item, and ni = number of items: αr=σr2/(σr2+(σr×i2/ni)); αp=σp2/(σp2+(σp×i2/ni)) and αr×p=σr×p2/(σr×p2+(σr×p×i2/ni)). These generalizability coefficients are interpreted in the conventional manner. For example, αr is essentially Cronbach's alpha (Cronbach et al., 9) and indicates the expected squared correlation between participants' scores on the items administered and participants' scores on all possible, similar items. αr is interpreted similarly except that the unit of observation is each student–professor dyad. </bibtext> </blist> <blist> <bibl id="bib2" idref="ref16" type="bt">2</bibl> <bibtext>We estimated the correspondence between the results of the two approaches by intraclass correlation for absolute agreement. The correlations produced by the two approaches were the dependent variables. Approach was one factor (Mgenova vs. the current approach), and variable pair was the second factor (e.g., positive affect and teaching evaluation). This design indicates the extent to which the two approaches yielded the same results. The correspondence between the results obtained by the two approaches was.98. </bibtext> </blist> <blist> <bibl id="bib3" idref="ref46" type="bt">3</bibl> <bibtext>In recommending from which professor a student should take a class, one would use a linear equation that forecasted the student's evaluation of each professor from the student's average score, each professor's average score, and the relationship score for each student–professor dyad. Typically, recommendations would be made separately for each student. In this case, the student's average score can be dropped from the equation because the score would be the same for predicting reactions to each professor. When a professor's standardized score is zero, prediction would be based entirely on the relationship score. When the relationship standardized score is zero, prediction would be based entirely on the professor score. </bibtext> </blist> </ref> <ref id="AN0101003818-14"> <title>References</title> <blist> <bibtext>Ambady, N., & Rosenthal, R. ( 1992 ). Thin slices of expressive behavior as predictors of interpersonal consequences: A meta‐analysis. Psychological Bulletin, 111, 256 – 274. doi: 10.1037/0033‐2909.111.2.256 </bibtext> </blist> <blist> <bibtext>Ambady, N., & Rosenthal, R. ( 1993 ). Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of Personality and Social Psychology, 64, 431 – 441. </bibtext> </blist> <blist> <bibtext>Babad, E., Avni‐Babad, D., & Rosenthal, R. ( 2004 ). Prediction of students' evaluations from brief instances of professors' nonverbal behavior in defined instructional situations. Social Psychology of Education, 7, 3 – 33. doi: 10.1023/B:SPOE.0000010672.97522.c5 </bibtext> </blist> <blist> <bibl id="bib4" idref="ref59" type="bt">4</bibl> <bibtext>Brennan, R. L. ( 2001 ). Generalizability theory. New York, NY : Springer. </bibtext> </blist> <blist> <bibl id="bib5" idref="ref5" type="bt">5</bibl> <bibtext>Centra, J. A. ( 1975 ). Colleagues as raters of classroom instruction. Journal of Higher Education, 46, 327 – 337. doi: 10.2307/1980806 </bibtext> </blist> <blist> <bibl id="bib6" idref="ref1" type="bt">6</bibl> <bibtext>Cohen, P. A. ( 1981 ). Student ratings of instruction and student achievement: A meta‐analysis of multisection validity studies. Review of Educational Research, 51, 281 – 309. doi: 10.3102/00346543051003281 </bibtext> </blist> <blist> <bibl id="bib7" idref="ref28" type="bt">7</bibl> <bibtext>Cook, W. L. ( 2000 ). Understanding attachment security in family context. Journal of Personality and Social Psychology, 78, 285 – 294. doi: 10.1037/0022‐3514.78.2.285 </bibtext> </blist> <blist> <bibl id="bib8" idref="ref27" type="bt">8</bibl> <bibtext>Cook, W. L., Kenny, D. A., & Goldstein, M. J. ( 1991 ). Parental affective style risk and the family system: A social relations model analysis. Journal of Abnormal Psychology, 100, 492 – 501. doi: 10.1037/0021‐843X.100.4.492 </bibtext> </blist> <blist> <bibl id="bib9" idref="ref19" type="bt">9</bibl> <bibtext>Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. ( 1972 ). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York, NY : John Wiley & Sons. </bibtext> </blist> <blist> <bibl id="bib10" idref="ref2" type="bt">10</bibl> <bibtext>Feldman, K. A. ( 1989a ). The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies. Research in Higher Education, 30, 583 – 645. doi: 10.1007/BF00992392 </bibtext> </blist> <blist> <bibl id="bib11" idref="ref6" type="bt">11</bibl> <bibtext>Feldman, K. A. ( 1989b ). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers. Research in Higher Education, 30, 137 – 194. doi: 10.1007/BF00992716 </bibtext> </blist> <blist> <bibl id="bib12" idref="ref40" type="bt">12</bibl> <bibtext>Fortunato, V. J., & Mincy, M. D. ( 2003 ). The interactive effects of dispositional affectivity, sex, and a positive mood induction on student evaluations of teachers. Journal of Applied Social Psychology, 33, 1945 – 1972. doi: 10.1111/j.1559‐1816.2003.tb02088.x </bibtext> </blist> <blist> <bibl id="bib13" idref="ref9" type="bt">13</bibl> <bibtext>Gross, J. A., Lakey, B., Edinger, K., Orehek, E., & Heffron, D. ( 2009 ). Person perception in the college classroom: Accounting for tastes in students' evaluations of teaching effectiveness. Journal of Applied Social Psychology, 39, 1609 – 1638. doi: 10.1111/j.1559‐1816.2009.00497.x </bibtext> </blist> <blist> <bibl id="bib14" idref="ref14" type="bt">14</bibl> <bibtext>Kenny, D. ( 1994 ). Interpersonal perception: A social relations analysis. New York, NY : Guilford. </bibtext> </blist> <blist> <bibl id="bib15" idref="ref38" type="bt">15</bibl> <bibtext>Kenny, D. A. ( 2004 ). PERSON: A general model of interpersonal perception. Personality and Social Psychology Review, 8, 265 – 280. doi: 10.1207/s15327957pspr0803_3 </bibtext> </blist> <blist> <bibl id="bib16" idref="ref21" type="bt">16</bibl> <bibtext>Kenny, D. A., Kashy, D. A., & Cook, W. L. ( 2006 ). Analysis of dyadic data. New York, NY : Guilford Press. </bibtext> </blist> <blist> <bibl id="bib17" idref="ref55" type="bt">17</bibl> <bibtext>Kwan, V. S. Y., John, O. P., Kenny, D. A., Bond, M. H., & Robins, R. W. ( 2004 ). Reconceptualizing individual differences in self‐enhancement bias: An interpersonal approach. Psychological Review, 111, 94 – 110. doi: 10.1037/0033‐295X.111.1.94 </bibtext> </blist> <blist> <bibl id="bib18" idref="ref56" type="bt">18</bibl> <bibtext>Kwan, V. S. Y., John, O. P., Robins, R. W., & Kuang, L. L. ( 2008 ). Conceptualizing and assessing self‐enhancement bias: A componential approach. Journal of Personality and Social Psychology, 94, 1062 – 1077. doi: 10.1037/0022‐3514.94.6.1062 </bibtext> </blist> <blist> <bibl id="bib19" idref="ref26" type="bt">19</bibl> <bibtext>Lakey, B., Cohen, J. L., & Neely, L. C. ( 2008 ). Perceived support and relational influences on psychotherapy process constructs. Journal of Counseling Psychology, 55, 209 – 220. doi: 10.1037/0022‐0167.55.2.209 </bibtext> </blist> <blist> <bibl id="bib20" idref="ref24" type="bt">20</bibl> <bibtext>Lakey, B., & Orehek, E. ( 2011 ). Relational Regulation Theory: A new approach to explain the link between perceived support and mental health. Psychological Review, 118, 482 – 495. doi: 10.1037/a0023477 </bibtext> </blist> <blist> <bibl id="bib21" idref="ref22" type="bt">21</bibl> <bibtext>Livi, S., Kenny, D. A., Albright, L., & Pierro, A. ( 2008 ). A social relations analysis of leadership. The Leadership Quarterly, 19, 235 – 248. doi: 10.1016/j.leaqua.2008.01.003 </bibtext> </blist> <blist> <bibl id="bib22" idref="ref25" type="bt">22</bibl> <bibtext>Lucas, T., Lakey, B., Arnetz, J. E., & Arnetz, B. B. ( 2010 ). Do cultural competency ratings reflect characteristics of providers or perceivers? Initial demonstration of a generalizability theory approach. Psychology, Health & Medicine, 15, 445 – 453. doi: 10.1080/13548506.2010.482141 </bibtext> </blist> <blist> <bibl id="bib23" idref="ref49" type="bt">23</bibl> <bibtext>Marsh, H. M. ( 1982 ). SEEQ: A reliable, valid, and useful instrument for collecting students' evaluations of university teaching. British Journal of Educational Psychology, 52, 77 – 95. </bibtext> </blist> <blist> <bibl id="bib24" idref="ref3" type="bt">24</bibl> <bibtext>Marsh, H. M. ( 1984 ). Students' evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76, 707 – 754. doi: 10.1037/0022‐0663.76.5.707 </bibtext> </blist> <blist> <bibl id="bib25" idref="ref4" type="bt">25</bibl> <bibtext>Marsh, H. W. ( 2007 ). Students' evaluations of university teaching: A multidimensional perspective. In R. P. Perry & J. C. Smart (Ed.), The scholarship of teaching and learning in higher education: An evidence‐based perspective (pp. 319 – 384 ). New York, NY : Springer. </bibtext> </blist> <blist> <bibl id="bib26" idref="ref72" type="bt">26</bibl> <bibtext>Oh, I., Wang, G., & Mount, M. K. ( 2011 ). Validity of observer ratings of the five‐factor model of personality traits: A meta‐analysis. Journal of Applied Psychology, 96, 762 – 773. doi: 10.1037/a0021832 </bibtext> </blist> <blist> <bibl id="bib27" idref="ref23" type="bt">27</bibl> <bibtext>Park, B., Kraus, S., & Ryan, C. S. ( 1997 ). Longitudinal changes in consensus as a function of acquaintance and agreement in liking. Journal of Personality and Social Psychology, 72, 604 – 616. doi: 10.1037/0022‐3514.72.3.604 </bibtext> </blist> <blist> <bibl id="bib28" idref="ref35" type="bt">28</bibl> <bibtext>Veenstra, A., Lakey, B., Cohen, J. L., Neely, L. C., Orehek, E., Barry, R., & Abeare, C. ( 2011 ). Forecasting the specific providers that recipients will perceive as unusually supportive. Personal Relationships, 18, 677 – 696. doi: 10.1111/j.1475‐6811.2010.01340.x </bibtext> </blist> <blist> <bibl id="bib29" idref="ref51" type="bt">29</bibl> <bibtext>Watson, D., Clark, L., & Tellegen, A. ( 1988 ). Development and validation of brief measures of positive and negative affect: The PANAS Scales. Journal of Personality and Social Psychology, 54, 1063 – 1070. doi: 10.1037/0022‐3514.54.6.1063 </bibtext> </blist> </ref> <aug> <p>By Jennifer Gross; Brian Lakey; Jessica L. Lucas; Ryan LaCross; Andrea R. Plotkowski and Bo Winegard</p> </aug>
Header DbId: eric
DbLabel: ERIC
An: EJ1052142
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Forecasting the Student-Professor Matches That Result in Unusually Effective Teaching
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Gross%2C+Jennifer%22">Gross, Jennifer</searchLink><br /><searchLink fieldCode="AR" term="%22Lakey%2C+Brian%22">Lakey, Brian</searchLink><br /><searchLink fieldCode="AR" term="%22Lucas%2C+Jessica+L%2E%22">Lucas, Jessica L.</searchLink><br /><searchLink fieldCode="AR" term="%22LaCross%2C+Ryan%22">LaCross, Ryan</searchLink><br /><searchLink fieldCode="AR" term="%22Plotkowski%2C+Andrea+R%2E%22">Plotkowski, Andrea R.</searchLink><br /><searchLink fieldCode="AR" term="%22Winegard%2C+Bo%22">Winegard, Bo</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22British+Journal+of+Educational+Psychology%22"><i>British Journal of Educational Psychology</i></searchLink>. Mar 2015 85(1):19-32.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 14
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2015
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Teacher+Student+Relationship%22">Teacher Student Relationship</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22College+Faculty%22">College Faculty</searchLink><br /><searchLink fieldCode="DE" term="%22Lecture+Method%22">Lecture Method</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Evaluation+of+Teacher+Performance%22">Student Evaluation of Teacher Performance</searchLink><br /><searchLink fieldCode="DE" term="%22Video+Technology%22">Video Technology</searchLink><br /><searchLink fieldCode="DE" term="%22Teaching+Methods%22">Teaching Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Analysis%22">Statistical Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Attitudes%22">Student Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Characteristics%22">Teacher Characteristics</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Effectiveness%22">Teacher Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Evaluation%22">Teacher Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Memory%22">Memory</searchLink><br /><searchLink fieldCode="DE" term="%22Predictor+Variables%22">Predictor Variables</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/bjep.12049
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0007-0998
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Two important influences on students' evaluations of teaching are relationship and professor effects. Relationship effects reflect unique matches between students and professors such that some professors are unusually effective for some students, but not for others. Professor effects reflect inter-rater agreement that some professors are more effective than others, on average across students. Aims: We attempted to forecast students' evaluations of live lectures from brief, video-recorded teaching trailers. Sample: Participants were 145 college students (74% female) enrolled in introductory psychology courses at a public university in the Great Lakes region of the United States. Methods: Students viewed trailers early in the semester and attended live lectures months later. Because subgroups of students viewed the same professors, statistical analyses could isolate professor and relationship effects. Results: Evaluations were influenced strongly by relationship and professor effects, and students' evaluations of live lectures could be forecasted from students' evaluations of teaching trailers. That is, we could forecast the individual students who would respond unusually well to a specific professor (relationship effects). We could also forecast which professors elicited better evaluations in live lectures, on average across students (professor effects). Professors who elicited unusually good evaluations in some students also elicited better memory for lectures in those students. Conclusions: It appears possible to forecast relationship and professor effects on teaching evaluations by presenting brief teaching trailers to students. Thus, it might be possible to develop online recommender systems to help match students and professors so that unusually effective teaching emerges.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 29
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2015
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1052142
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1052142
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/bjep.12049
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 14
        StartPage: 19
    Subjects:
      – SubjectFull: Teacher Student Relationship
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: College Faculty
        Type: general
      – SubjectFull: Lecture Method
        Type: general
      – SubjectFull: Student Evaluation of Teacher Performance
        Type: general
      – SubjectFull: Video Technology
        Type: general
      – SubjectFull: Teaching Methods
        Type: general
      – SubjectFull: Statistical Analysis
        Type: general
      – SubjectFull: Student Attitudes
        Type: general
      – SubjectFull: Teacher Characteristics
        Type: general
      – SubjectFull: Teacher Effectiveness
        Type: general
      – SubjectFull: Teacher Evaluation
        Type: general
      – SubjectFull: Memory
        Type: general
      – SubjectFull: Predictor Variables
        Type: general
    Titles:
      – TitleFull: Forecasting the Student-Professor Matches That Result in Unusually Effective Teaching
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Gross, Jennifer
      – PersonEntity:
          Name:
            NameFull: Lakey, Brian
      – PersonEntity:
          Name:
            NameFull: Lucas, Jessica L.
      – PersonEntity:
          Name:
            NameFull: LaCross, Ryan
      – PersonEntity:
          Name:
            NameFull: Plotkowski, Andrea R.
      – PersonEntity:
          Name:
            NameFull: Winegard, Bo
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 16
              M: 03
              Type: published
              Y: 2015
          Identifiers:
            – Type: issn-print
              Value: 0007-0998
          Numbering:
            – Type: volume
              Value: 85
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: British Journal of Educational Psychology
              Type: main
ResultId 1