Impact of a Checklist on Principal-Teacher Feedback Conferences Following Classroom Observations. REL 2018-285
Saved in:
| Title: | Impact of a Checklist on Principal-Teacher Feedback Conferences Following Classroom Observations. REL 2018-285 |
|---|---|
| Language: | English |
| Authors: | Mihaly, Kata, Schwartz, Heather L., Opper, Isaac M., Grimm, Geoffrey, Rodriguez, Luis, Mariano, Louis T., Regional Educational Laboratory Southwest (ED), National Center for Education Evaluation and Regional Assistance (ED), SEDL |
| Source: | Regional Educational Laboratory Southwest. 2018. |
| Availability: | Regional Educational Laboratory Southwest. Available from: Institute of Education Sciences. 555 New Jersey Avenue NW, Washington, DC 20208. Tel: 800-872-5327; Web site: http://ies.ed.gov/ncee/edlabs/ |
| Peer Reviewed: | Y |
| Page Count: | 72 |
| Publication Date: | 2018 |
| Contract Number: | EDIES12C0012 |
| Document Type: | Reports - Research Numerical/Quantitative Data |
| Education Level: | Elementary Secondary Education |
| Descriptors: | Check Lists, Feedback (Response), Conferences (Gatherings), Teacher Administrator Relationship, Observation, Principals, Public School Teachers, Elementary Secondary Education, Academic Achievement, Grades (Scholastic), Faculty Development, Educational Quality, Regression (Statistics), Statistical Significance, Statistical Analysis |
| Geographic Terms: | New Mexico |
| Abstract: | Most states' teacher evaluation systems have changed substantially in the past decade. New evaluation systems typically require school leaders to observe teachers' classrooms two to three times a school year instead of once (Doherty & Jacobs, 2015). The feedback that school leaders provide to teachers after these observations is a key but understudied step in the teacher evaluation cycle. The feedback and subsequent professional development are intended to help teachers change their instructional practices and improve student achievement (Correnti & Rowan, 2007; DeNisi & Sonesh, 2011; Taylor & Tyler, 2012). However, little is known about the feedback that school leaders provide to teachers following classroom observations or about how to train leaders to make that feedback more effective. This study examined the impact of disseminating a detailed checklist intended to structure an effective feedback conference between a school leader and a teacher following a classroom observation. The feedback conference checklist is a modified version of one created by the Carnegie Foundation for the Advancement of Teaching (Tang & Chow, 2007). The checklist, along with short testimonial videos, was a low-cost, low-intensity intervention provided to a randomly selected half of 339 participating New Mexico principals in fall 2015 by the study team. These principals' schools constituted the treatment group. Principals in the treatment group schools received an email with an attachment containing a guide and a 24-item feedback conference checklist, plus a hyperlink to a three-minute testimonial video featuring a principal. Principals were encouraged to distribute the checklist to other school leaders and to use the checklist in all their feedback conferences in the 2015/16 school year. Principals were also asked to distribute the checklist to all their teachers in order to promote greater teacher participation in the feedback conference. The study team also emailed the same checklist plus a hyperlink to a three-minute testimonial video featuring a teacher to up to 10 randomly sampled teachers in each treatment group school. The other half of the principals in the study schools formed the control group. Each of the control group principals received a two-page principal guide as an email attachment in fall 2015. The two-page guide reprised the five tips about feedback included in the summer 2015 New Mexico Public Education Department--sponsored professional development for principals and informed principals about the study. In addition, the study team sent up to 10 randomly sampled teachers in each control group school a two-page teacher guide summarizing the teacher evaluation system (Skandera, 2013) and teachers' right to receive post-observation feedback. All principals and teachers in both the treatment group and the control group who consented to be in the study were asked to complete an online survey (one for principals, another for teachers) in spring 2015 and again in spring 2016. The main outcomes of the study were principals' and teachers' reports of the impacts of the checklist and testimonial video on the perceived quality of feedback conferences following formal classroom observations; principals' recommendations for and teachers' take-up of professional development; and the quality of teachers' subsequent instructional practices as measured by principals' formal classroom observation scores and teachers' self-reported scores. Additional exploratory outcomes included the impact of the checklist on student achievement (school-average math and English language arts scores on the Partnership for Assessment of Readiness for College and Careers assessment) and school report card grades (reported as an A, B, C, D, or F of multiple measures of a school's student achievement) compiled annually by the New Mexico Public Education Department. The study also documented how many recipients reported using the checklist and what they thought about it. The checklist had few clear impacts on the quality of feedback, professional development outcomes, instructional practice, or student achievement. There were two exceptions: teachers who received the checklist reported that their principals were less likely to dominate the feedback conferences, and they reported that they were more likely to follow their principal's professional development recommendations. Use of the checklist in the treatment group was moderate: 77 percent of principals surveyed who received the checklist reported viewing it, and 58 percent said they used it with one or more teachers. At the same time, 29 percent of control group principals (who were not emailed the checklist) reported that they had seen the checklist, and 10 percent reported using it with one or more teachers. The relatively moderate use of the checklist by treatment group principals, combined with the reports by some control group school leaders that they were using it, implies that the estimated impacts of using the checklist would be larger than the estimated impacts of receiving it. Though distribution of the feedback conference checklist to principals and teachers had a few modest impacts, this study indicates that distributing the checklist is unlikely by itself to substantially alter feedback conferences, teachers' classroom practices, or student achievement, at least during the first school year in which the checklist is used. This study suggests that only a fraction of school leaders are likely to use the checklist if it is distributed in the low-cost manner followed in this study. But the checklist may also have failed to help principals overcome common barriers to effective feedback, such as providing critical comments to teachers or recommending appropriate professional development. The study results raise the possibility that additional (or different) investments might be necessary to improve school leaders' feedback conferences with teachers--for example, pairing training with written guidance. |
| Abstractor: | ERIC |
| Number of References: | 44 |
| IES Funded: | Yes |
| IES Publication: | https://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=457 |
| Entry Date: | 2018 |
| Accession Number: | ED580182 |
| Database: | ERIC |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED580182 Name: ERIC Full Text Category: fullText Text: Full Text from ERIC |
|---|---|
| Header | DbId: eric DbLabel: ERIC An: ED580182 AccessLevel: 3 PubType: Report PubTypeId: report PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Impact of a Checklist on Principal-Teacher Feedback Conferences Following Classroom Observations. REL 2018-285 – Name: Language Label: Language Group: Lang Data: English – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Mihaly%2C+Kata%22">Mihaly, Kata</searchLink><br /><searchLink fieldCode="AR" term="%22Schwartz%2C+Heather+L%2E%22">Schwartz, Heather L.</searchLink><br /><searchLink fieldCode="AR" term="%22Opper%2C+Isaac+M%2E%22">Opper, Isaac M.</searchLink><br /><searchLink fieldCode="AR" term="%22Grimm%2C+Geoffrey%22">Grimm, Geoffrey</searchLink><br /><searchLink fieldCode="AR" term="%22Rodriguez%2C+Luis%22">Rodriguez, Luis</searchLink><br /><searchLink fieldCode="AR" term="%22Mariano%2C+Louis+T%2E%22">Mariano, Louis T.</searchLink><br /><searchLink fieldCode="AR" term="%22Regional+Educational+Laboratory+Southwest+%28ED%29%22">Regional Educational Laboratory Southwest (ED)</searchLink><br /><searchLink fieldCode="AR" term="%22National+Center+for+Education+Evaluation+and+Regional+Assistance+%28ED%29%22">National Center for Education Evaluation and Regional Assistance (ED)</searchLink><br /><searchLink fieldCode="AR" term="%22SEDL%22">SEDL</searchLink> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="SO" term="%22Regional+Educational+Laboratory+Southwest%22"><i>Regional Educational Laboratory Southwest</i></searchLink>. 2018. – Name: Avail Label: Availability Group: Avail Data: Regional Educational Laboratory Southwest. Available from: Institute of Education Sciences. 555 New Jersey Avenue NW, Washington, DC 20208. Tel: 800-872-5327; Web site: http://ies.ed.gov/ncee/edlabs/ – Name: PeerReviewed Label: Peer Reviewed Group: SrcInfo Data: Y – Name: Pages Label: Page Count Group: Src Data: 72 – Name: DatePubCY Label: Publication Date Group: Date Data: 2018 – Name: NumberContract Label: Contract Number Group: NumCntrct Data: EDIES12C0012 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Reports - Research<br />Numerical/Quantitative Data – Name: Audience Label: Education Level Group: Audnce Data: <searchLink fieldCode="EL" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink> – Name: Subject Label: Descriptors Group: Su Data: <searchLink fieldCode="DE" term="%22Check+Lists%22">Check Lists</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Conferences+%28Gatherings%29%22">Conferences (Gatherings)</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Administrator+Relationship%22">Teacher Administrator Relationship</searchLink><br /><searchLink fieldCode="DE" term="%22Observation%22">Observation</searchLink><br /><searchLink fieldCode="DE" term="%22Principals%22">Principals</searchLink><br /><searchLink fieldCode="DE" term="%22Public+School+Teachers%22">Public School Teachers</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Achievement%22">Academic Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Grades+%28Scholastic%29%22">Grades (Scholastic)</searchLink><br /><searchLink fieldCode="DE" term="%22Faculty+Development%22">Faculty Development</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Quality%22">Educational Quality</searchLink><br /><searchLink fieldCode="DE" term="%22Regression+%28Statistics%29%22">Regression (Statistics)</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Significance%22">Statistical Significance</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Analysis%22">Statistical Analysis</searchLink> – Name: Subject Label: Geographic Terms Group: Su Data: <searchLink fieldCode="DE" term="%22New+Mexico%22">New Mexico</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Most states' teacher evaluation systems have changed substantially in the past decade. New evaluation systems typically require school leaders to observe teachers' classrooms two to three times a school year instead of once (Doherty & Jacobs, 2015). The feedback that school leaders provide to teachers after these observations is a key but understudied step in the teacher evaluation cycle. The feedback and subsequent professional development are intended to help teachers change their instructional practices and improve student achievement (Correnti & Rowan, 2007; DeNisi & Sonesh, 2011; Taylor & Tyler, 2012). However, little is known about the feedback that school leaders provide to teachers following classroom observations or about how to train leaders to make that feedback more effective. This study examined the impact of disseminating a detailed checklist intended to structure an effective feedback conference between a school leader and a teacher following a classroom observation. The feedback conference checklist is a modified version of one created by the Carnegie Foundation for the Advancement of Teaching (Tang & Chow, 2007). The checklist, along with short testimonial videos, was a low-cost, low-intensity intervention provided to a randomly selected half of 339 participating New Mexico principals in fall 2015 by the study team. These principals' schools constituted the treatment group. Principals in the treatment group schools received an email with an attachment containing a guide and a 24-item feedback conference checklist, plus a hyperlink to a three-minute testimonial video featuring a principal. Principals were encouraged to distribute the checklist to other school leaders and to use the checklist in all their feedback conferences in the 2015/16 school year. Principals were also asked to distribute the checklist to all their teachers in order to promote greater teacher participation in the feedback conference. The study team also emailed the same checklist plus a hyperlink to a three-minute testimonial video featuring a teacher to up to 10 randomly sampled teachers in each treatment group school. The other half of the principals in the study schools formed the control group. Each of the control group principals received a two-page principal guide as an email attachment in fall 2015. The two-page guide reprised the five tips about feedback included in the summer 2015 New Mexico Public Education Department--sponsored professional development for principals and informed principals about the study. In addition, the study team sent up to 10 randomly sampled teachers in each control group school a two-page teacher guide summarizing the teacher evaluation system (Skandera, 2013) and teachers' right to receive post-observation feedback. All principals and teachers in both the treatment group and the control group who consented to be in the study were asked to complete an online survey (one for principals, another for teachers) in spring 2015 and again in spring 2016. The main outcomes of the study were principals' and teachers' reports of the impacts of the checklist and testimonial video on the perceived quality of feedback conferences following formal classroom observations; principals' recommendations for and teachers' take-up of professional development; and the quality of teachers' subsequent instructional practices as measured by principals' formal classroom observation scores and teachers' self-reported scores. Additional exploratory outcomes included the impact of the checklist on student achievement (school-average math and English language arts scores on the Partnership for Assessment of Readiness for College and Careers assessment) and school report card grades (reported as an A, B, C, D, or F of multiple measures of a school's student achievement) compiled annually by the New Mexico Public Education Department. The study also documented how many recipients reported using the checklist and what they thought about it. The checklist had few clear impacts on the quality of feedback, professional development outcomes, instructional practice, or student achievement. There were two exceptions: teachers who received the checklist reported that their principals were less likely to dominate the feedback conferences, and they reported that they were more likely to follow their principal's professional development recommendations. Use of the checklist in the treatment group was moderate: 77 percent of principals surveyed who received the checklist reported viewing it, and 58 percent said they used it with one or more teachers. At the same time, 29 percent of control group principals (who were not emailed the checklist) reported that they had seen the checklist, and 10 percent reported using it with one or more teachers. The relatively moderate use of the checklist by treatment group principals, combined with the reports by some control group school leaders that they were using it, implies that the estimated impacts of using the checklist would be larger than the estimated impacts of receiving it. Though distribution of the feedback conference checklist to principals and teachers had a few modest impacts, this study indicates that distributing the checklist is unlikely by itself to substantially alter feedback conferences, teachers' classroom practices, or student achievement, at least during the first school year in which the checklist is used. This study suggests that only a fraction of school leaders are likely to use the checklist if it is distributed in the low-cost manner followed in this study. But the checklist may also have failed to help principals overcome common barriers to effective feedback, such as providing critical comments to teachers or recommending appropriate professional development. The study results raise the possibility that additional (or different) investments might be necessary to improve school leaders' feedback conferences with teachers--for example, pairing training with written guidance. – Name: AbstractInfo Label: Abstractor Group: Ab Data: ERIC – Name: Ref Label: Number of References Group: RefInfo Data: 44 – Name: CodeSource Label: IES Funded Group: SrcInfo Data: Yes – Name: CodeSource Label: IES Publication Group: SrcInfo Data: <link linkTarget="URL" linkTerm="https://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=457" linkWindow="_blank">https://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=457</link> – Name: DateEntry Label: Entry Date Group: Date Data: 2018 – Name: AN Label: Accession Number Group: ID Data: ED580182 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED580182 |
| RecordInfo | BibRecord: BibEntity: Languages: – Text: English PhysicalDescription: Pagination: PageCount: 72 Subjects: – SubjectFull: Check Lists Type: general – SubjectFull: Feedback (Response) Type: general – SubjectFull: Conferences (Gatherings) Type: general – SubjectFull: Teacher Administrator Relationship Type: general – SubjectFull: Observation Type: general – SubjectFull: Principals Type: general – SubjectFull: Public School Teachers Type: general – SubjectFull: Elementary Secondary Education Type: general – SubjectFull: Academic Achievement Type: general – SubjectFull: Grades (Scholastic) Type: general – SubjectFull: Faculty Development Type: general – SubjectFull: Educational Quality Type: general – SubjectFull: Regression (Statistics) Type: general – SubjectFull: Statistical Significance Type: general – SubjectFull: Statistical Analysis Type: general – SubjectFull: New Mexico Type: general Titles: – TitleFull: Impact of a Checklist on Principal-Teacher Feedback Conferences Following Classroom Observations. REL 2018-285 Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Regional Educational Laboratory Southwest (ED) – PersonEntity: Name: NameFull: National Center for Education Evaluation and Regional Assistance (ED) – PersonEntity: Name: NameFull: SEDL – PersonEntity: Name: NameFull: Mihaly, Kata – PersonEntity: Name: NameFull: Schwartz, Heather L. – PersonEntity: Name: NameFull: Opper, Isaac M. – PersonEntity: Name: NameFull: Grimm, Geoffrey – PersonEntity: Name: NameFull: Rodriguez, Luis – PersonEntity: Name: NameFull: Mariano, Louis T. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 01 Type: published Y: 2018 Titles: – TitleFull: Regional Educational Laboratory Southwest Type: main |
| ResultId | 1 |
