View in EDS

Is a Picture Worth 51 Million Words? A Text Analysis of Public User Reviews of Schools. Technical Report

Saved in:
Bibliographic Details
Title: Is a Picture Worth 51 Million Words? A Text Analysis of Public User Reviews of Schools. Technical Report
Language: English
Authors: Douglas Harris, Jamie Carroll, Debbie Kim, Nicholas Mattei, Olivia Carr, National Center for Research on Education Access and Choice (REACH)
Source: National Center for Research on Education Access and Choice. 2024.
Availability: National Center for Research on Education Access and Choice. 1555 Poydras Street Suite 700, New Orleans, LA 70112. Tel: 870-540-6576; e-mail: info@reachcentered.org; Web site: https://reachcentered.org/
Peer Reviewed: Y
Page Count: 99
Publication Date: 2024
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305C180025
Document Type: Reports - Research
Education Level: Elementary Secondary Education
Descriptors: Elementary Secondary Education, School Effectiveness, Parents, Teachers, Students, Principals, Administrator Attitudes, Parent Attitudes, Student Attitudes, Teacher Attitudes, Public Schools, Charter Schools, Private Schools, School Choice, Public Opinion, Natural Language Processing, Evaluation Problems
Abstract: Massive online user review platforms, with their star ratings and text reviews, are reshaping the information available for consumer and public service decisions. We study the leading K-12 schooling platform, GreatSchools, applying machine learning (Natural Language Processing, NLP) to 600,000 reviews that encompass the vast majority of the nation's traditional public, charter, and private schools (84,009 schools in total), supplemented with qualitative analysis of a subsample of reviews. Encompassing more than fifty million words of text, our initial analysis pre-specified eight broad topics and 27 sub-topics and coded review words into these categories. We find that parents write the vast majority of reviews and tend to write more about School Staff and School Culture than students. More generally, text reviews vary in important ways across user types (parents, students, teachers, principals), school sector (traditional public, charter, and private schools), grade level, and demographics. The partial correlation between topics and star ratings also differs across user types and sectors. Taken together, these results suggest that user reviews are less useful than they appear and less useful than with other kinds of products. Our analysis points to design features that might improve their usefulness. The variation in content and value of the reviews also has methodological implications as it shows how NLP can complement qualitative research methods with such large volumes of text.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2024
Accession Number: ED661838
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED661838
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED661838
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Is a Picture Worth 51 Million Words? A Text Analysis of Public User Reviews of Schools. Technical Report
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Douglas+Harris%22">Douglas Harris</searchLink><br /><searchLink fieldCode="AR" term="%22Jamie+Carroll%22">Jamie Carroll</searchLink><br /><searchLink fieldCode="AR" term="%22Debbie+Kim%22">Debbie Kim</searchLink><br /><searchLink fieldCode="AR" term="%22Nicholas+Mattei%22">Nicholas Mattei</searchLink><br /><searchLink fieldCode="AR" term="%22Olivia+Carr%22">Olivia Carr</searchLink><br /><searchLink fieldCode="AR" term="%22National+Center+for+Research+on+Education+Access+and+Choice+%28REACH%29%22">National Center for Research on Education Access and Choice (REACH)</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22National+Center+for+Research+on+Education+Access+and+Choice%22"><i>National Center for Research on Education Access and Choice</i></searchLink>. 2024.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: National Center for Research on Education Access and Choice. 1555 Poydras Street Suite 700, New Orleans, LA 70112. Tel: 870-540-6576; e-mail: info@reachcentered.org; Web site: https://reachcentered.org/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 99
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305C180025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink><br /><searchLink fieldCode="DE" term="%22School+Effectiveness%22">School Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Parents%22">Parents</searchLink><br /><searchLink fieldCode="DE" term="%22Teachers%22">Teachers</searchLink><br /><searchLink fieldCode="DE" term="%22Students%22">Students</searchLink><br /><searchLink fieldCode="DE" term="%22Principals%22">Principals</searchLink><br /><searchLink fieldCode="DE" term="%22Administrator+Attitudes%22">Administrator Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Parent+Attitudes%22">Parent Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Attitudes%22">Student Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Attitudes%22">Teacher Attitudes</searchLink><br /><searchLink fieldCode="DE" term="%22Public+Schools%22">Public Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Charter+Schools%22">Charter Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Private+Schools%22">Private Schools</searchLink><br /><searchLink fieldCode="DE" term="%22School+Choice%22">School Choice</searchLink><br /><searchLink fieldCode="DE" term="%22Public+Opinion%22">Public Opinion</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Problems%22">Evaluation Problems</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Massive online user review platforms, with their star ratings and text reviews, are reshaping the information available for consumer and public service decisions. We study the leading K-12 schooling platform, GreatSchools, applying machine learning (Natural Language Processing, NLP) to 600,000 reviews that encompass the vast majority of the nation's traditional public, charter, and private schools (84,009 schools in total), supplemented with qualitative analysis of a subsample of reviews. Encompassing more than fifty million words of text, our initial analysis pre-specified eight broad topics and 27 sub-topics and coded review words into these categories. We find that parents write the vast majority of reviews and tend to write more about School Staff and School Culture than students. More generally, text reviews vary in important ways across user types (parents, students, teachers, principals), school sector (traditional public, charter, and private schools), grade level, and demographics. The partial correlation between topics and star ratings also differs across user types and sectors. Taken together, these results suggest that user reviews are less useful than they appear and less useful than with other kinds of products. Our analysis points to design features that might improve their usefulness. The variation in content and value of the reviews also has methodological implications as it shows how NLP can complement qualitative research methods with such large volumes of text.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED661838
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED661838
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 99
    Subjects:
      – SubjectFull: Elementary Secondary Education
        Type: general
      – SubjectFull: School Effectiveness
        Type: general
      – SubjectFull: Parents
        Type: general
      – SubjectFull: Teachers
        Type: general
      – SubjectFull: Students
        Type: general
      – SubjectFull: Principals
        Type: general
      – SubjectFull: Administrator Attitudes
        Type: general
      – SubjectFull: Parent Attitudes
        Type: general
      – SubjectFull: Student Attitudes
        Type: general
      – SubjectFull: Teacher Attitudes
        Type: general
      – SubjectFull: Public Schools
        Type: general
      – SubjectFull: Charter Schools
        Type: general
      – SubjectFull: Private Schools
        Type: general
      – SubjectFull: School Choice
        Type: general
      – SubjectFull: Public Opinion
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Evaluation Problems
        Type: general
    Titles:
      – TitleFull: Is a Picture Worth 51 Million Words? A Text Analysis of Public User Reviews of Schools. Technical Report
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: National Center for Research on Education Access and Choice (REACH)
      – PersonEntity:
          Name:
            NameFull: Douglas Harris
      – PersonEntity:
          Name:
            NameFull: Jamie Carroll
      – PersonEntity:
          Name:
            NameFull: Debbie Kim
      – PersonEntity:
          Name:
            NameFull: Nicholas Mattei
      – PersonEntity:
          Name:
            NameFull: Olivia Carr
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 08
              M: 11
              Type: published
              Y: 2024
          Titles:
            – TitleFull: National Center for Research on Education Access and Choice
              Type: main
ResultId 1