View in EDS

The Reliability of Classroom Observations and Student Surveys in Non-Research Settings: Evidence from Argentina. EdWorkingPaper No. 25-1322

Saved in:
Bibliographic Details
Title: The Reliability of Classroom Observations and Student Surveys in Non-Research Settings: Evidence from Argentina. EdWorkingPaper No. 25-1322
Language: English
Authors: Alejandro J. Ganimian, Andrew D. Ho, Alejandra Campos Quintero, Annenberg Institute for School Reform at Brown University
Source: Annenberg Institute for School Reform at Brown University. 2025.
Availability: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
Peer Reviewed: N
Page Count: 44
Publication Date: 2025
Sponsoring Agency: Inter-American Development Bank (IDB)
Document Type: Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Foreign Countries, Clinical Supervision (of Teachers), Student Evaluation of Teacher Performance, Teacher Effectiveness, Evaluation Methods, Student Surveys, Classroom Observation Techniques, Observation, Response Rates (Questionnaires), Research Problems, Feedback (Response), Preservice Teachers
Geographic Terms: Argentina (Buenos Aires)
Abstract: There is a growing consensus on the need to measure teaching effectiveness using multiple instruments. Yet, guidance on how to achieve reliable ratings derives largely from formal research in high-income countries. We study the reliability of classroom observations and student surveys conducted by practitioners in a middle-income country. Both instruments can achieve relatively high reliability (0.6-0.8 on a 0-1 scale) when averaged across raters and occasions, but the reliability of observations varies widely (from 0.4 to 0.8) based mostly on how raters are assigned to teachers. We use Generalizability Theory to estimate how reliability improves by increasing the number of times teachers are observed or the number of respondents to surveys. We recommend that practitioners design their teacher feedback systems based on analyses of their own data, instead of assuming that instruments and rubrics will generate scores with the same reliability as research settings.
Abstractor: As Provided
Entry Date: 2026
Accession Number: ED678303
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED678303
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED678303
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Reliability of Classroom Observations and Student Surveys in Non-Research Settings: Evidence from Argentina. EdWorkingPaper No. 25-1322
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Alejandro+J%2E+Ganimian%22">Alejandro J. Ganimian</searchLink><br /><searchLink fieldCode="AR" term="%22Andrew+D%2E+Ho%22">Andrew D. Ho</searchLink><br /><searchLink fieldCode="AR" term="%22Alejandra+Campos+Quintero%22">Alejandra Campos Quintero</searchLink><br /><searchLink fieldCode="AR" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22">Annenberg Institute for School Reform at Brown University</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22"><i>Annenberg Institute for School Reform at Brown University</i></searchLink>. 2025.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 44
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Inter-American Development Bank (IDB)
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Clinical+Supervision+%28of+Teachers%29%22">Clinical Supervision (of Teachers)</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Evaluation+of+Teacher+Performance%22">Student Evaluation of Teacher Performance</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Effectiveness%22">Teacher Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Surveys%22">Student Surveys</searchLink><br /><searchLink fieldCode="DE" term="%22Classroom+Observation+Techniques%22">Classroom Observation Techniques</searchLink><br /><searchLink fieldCode="DE" term="%22Observation%22">Observation</searchLink><br /><searchLink fieldCode="DE" term="%22Response+Rates+%28Questionnaires%29%22">Response Rates (Questionnaires)</searchLink><br /><searchLink fieldCode="DE" term="%22Research+Problems%22">Research Problems</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Preservice+Teachers%22">Preservice Teachers</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Argentina+%28Buenos+Aires%29%22">Argentina (Buenos Aires)</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: There is a growing consensus on the need to measure teaching effectiveness using multiple instruments. Yet, guidance on how to achieve reliable ratings derives largely from formal research in high-income countries. We study the reliability of classroom observations and student surveys conducted by practitioners in a middle-income country. Both instruments can achieve relatively high reliability (0.6-0.8 on a 0-1 scale) when averaged across raters and occasions, but the reliability of observations varies widely (from 0.4 to 0.8) based mostly on how raters are assigned to teachers. We use Generalizability Theory to estimate how reliability improves by increasing the number of times teachers are observed or the number of respondents to surveys. We recommend that practitioners design their teacher feedback systems based on analyses of their own data, instead of assuming that instruments and rubrics will generate scores with the same reliability as research settings.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED678303
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED678303
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 44
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Clinical Supervision (of Teachers)
        Type: general
      – SubjectFull: Student Evaluation of Teacher Performance
        Type: general
      – SubjectFull: Teacher Effectiveness
        Type: general
      – SubjectFull: Evaluation Methods
        Type: general
      – SubjectFull: Student Surveys
        Type: general
      – SubjectFull: Classroom Observation Techniques
        Type: general
      – SubjectFull: Observation
        Type: general
      – SubjectFull: Response Rates (Questionnaires)
        Type: general
      – SubjectFull: Research Problems
        Type: general
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Preservice Teachers
        Type: general
      – SubjectFull: Argentina (Buenos Aires)
        Type: general
    Titles:
      – TitleFull: The Reliability of Classroom Observations and Student Surveys in Non-Research Settings: Evidence from Argentina. EdWorkingPaper No. 25-1322
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Annenberg Institute for School Reform at Brown University
      – PersonEntity:
          Name:
            NameFull: Alejandro J. Ganimian
      – PersonEntity:
          Name:
            NameFull: Andrew D. Ho
      – PersonEntity:
          Name:
            NameFull: Alejandra Campos Quintero
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 11
              Type: published
              Y: 2025
          Titles:
            – TitleFull: Annenberg Institute for School Reform at Brown University
              Type: main
ResultId 1