View in EDS

Online Reviews Are Leading Indicators of Changes in K-12 School Attributes

Saved in:
Bibliographic Details
Title: Online Reviews Are Leading Indicators of Changes in K-12 School Attributes
Language: English
Authors: Linsen Li (ORCID 0000-0002-8285-0089), Aron Culotta (ORCID 0000-0003-2660-7575), Douglas N. Harris (ORCID 0000-0003-3605-7132), Nicholas Mattei (ORCID 0000-0002-3569-4335)
Source: Grantee Submission. 2023.
Peer Reviewed: Y
Page Count: 11
Publication Date: 2023
Sponsoring Agency: Institute of Education Sciences (ED)
National Science Foundation (NSF), Division of Information and Intelligent Systems (IIS)
Contract Number: R305C180025
RI2007955
III2107505
RI2134857
Document Type: Speeches/Meeting Papers
Reports - Research
Education Level: Elementary Education
Junior High Schools
Middle Schools
Secondary Education
Descriptors: Elementary Schools, Middle Schools, Secondary Schools, Educational Change, Institutional Characteristics, Evaluation, Computer Mediated Communication, Language Usage, Discourse Analysis, Change Agents, Internet, Electronic Publishing
DOI: 10.1145/3543507.3583531
Abstract: School rating websites are increasingly used by parents to assess the quality and fit of U.S. K-12 schools for their children. These online reviews often contain detailed descriptions of a school's strengths and weaknesses, which both reflect and inform perceptions of a school. Existing work on these text reviews has focused on finding words or themes that underlie these perceptions, but has stopped short of using the textual reviews as leading indicators of school performance. In this paper, we investigate to what extent the language used in online reviews of a school is predictive of changes in the attributes of that school, such as its socio-economic makeup and student test scores. Using over 300K reviews of 70K U.S. schools from a popular ratings website, we apply language processing models to predict whether schools will significantly increase or decrease in an attribute of interest over a future time horizon. We find that using the text improves predictive performance significantly over a baseline model that does not include text but only the historical time-series of the indicators themselves, suggesting that the review text carries predictive power. A qualitative analysis of the most predictive terms and phrases used in the text reviews indicates a number of topics that serve as leading indicators, such as diversity, changes in school leadership, a focus on testing, and school safety.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2024
Accession Number: ED652691
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED652691
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED652691
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Online Reviews Are Leading Indicators of Changes in K-12 School Attributes
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Linsen+Li%22">Linsen Li</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-8285-0089">0000-0002-8285-0089</externalLink>)<br /><searchLink fieldCode="AR" term="%22Aron+Culotta%22">Aron Culotta</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-2660-7575">0000-0003-2660-7575</externalLink>)<br /><searchLink fieldCode="AR" term="%22Douglas+N%2E+Harris%22">Douglas N. Harris</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3605-7132">0000-0003-3605-7132</externalLink>)<br /><searchLink fieldCode="AR" term="%22Nicholas+Mattei%22">Nicholas Mattei</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-3569-4335">0000-0002-3569-4335</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Grantee+Submission%22"><i>Grantee Submission</i></searchLink>. 2023.
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 11
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)<br />National Science Foundation (NSF), Division of Information and Intelligent Systems (IIS)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305C180025<br />RI2007955<br />III2107505<br />RI2134857
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Elementary+Schools%22">Elementary Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Secondary+Schools%22">Secondary Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Change%22">Educational Change</searchLink><br /><searchLink fieldCode="DE" term="%22Institutional+Characteristics%22">Institutional Characteristics</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation%22">Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Mediated+Communication%22">Computer Mediated Communication</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Usage%22">Language Usage</searchLink><br /><searchLink fieldCode="DE" term="%22Discourse+Analysis%22">Discourse Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Change+Agents%22">Change Agents</searchLink><br /><searchLink fieldCode="DE" term="%22Internet%22">Internet</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+Publishing%22">Electronic Publishing</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1145/3543507.3583531
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: School rating websites are increasingly used by parents to assess the quality and fit of U.S. K-12 schools for their children. These online reviews often contain detailed descriptions of a school's strengths and weaknesses, which both reflect and inform perceptions of a school. Existing work on these text reviews has focused on finding words or themes that underlie these perceptions, but has stopped short of using the textual reviews as leading indicators of school performance. In this paper, we investigate to what extent the language used in online reviews of a school is predictive of changes in the attributes of that school, such as its socio-economic makeup and student test scores. Using over 300K reviews of 70K U.S. schools from a popular ratings website, we apply language processing models to predict whether schools will significantly increase or decrease in an attribute of interest over a future time horizon. We find that using the text improves predictive performance significantly over a baseline model that does not include text but only the historical time-series of the indicators themselves, suggesting that the review text carries predictive power. A qualitative analysis of the most predictive terms and phrases used in the text reviews indicates a number of topics that serve as leading indicators, such as diversity, changes in school leadership, a focus on testing, and school safety.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED652691
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED652691
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1145/3543507.3583531
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
    Subjects:
      – SubjectFull: Elementary Schools
        Type: general
      – SubjectFull: Middle Schools
        Type: general
      – SubjectFull: Secondary Schools
        Type: general
      – SubjectFull: Educational Change
        Type: general
      – SubjectFull: Institutional Characteristics
        Type: general
      – SubjectFull: Evaluation
        Type: general
      – SubjectFull: Computer Mediated Communication
        Type: general
      – SubjectFull: Language Usage
        Type: general
      – SubjectFull: Discourse Analysis
        Type: general
      – SubjectFull: Change Agents
        Type: general
      – SubjectFull: Internet
        Type: general
      – SubjectFull: Electronic Publishing
        Type: general
    Titles:
      – TitleFull: Online Reviews Are Leading Indicators of Changes in K-12 School Attributes
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Linsen Li
      – PersonEntity:
          Name:
            NameFull: Aron Culotta
      – PersonEntity:
          Name:
            NameFull: Douglas N. Harris
      – PersonEntity:
          Name:
            NameFull: Nicholas Mattei
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2023
          Titles:
            – TitleFull: Grantee Submission
              Type: main
ResultId 1