View in EDS

From Sentence-Corrections to Deeper Dialogue: Qualitative Insights from LLM and Teacher Feedback on Student Writing. EdWorkingPaper No. 25-1193

Saved in:
Bibliographic Details
Title: From Sentence-Corrections to Deeper Dialogue: Qualitative Insights from LLM and Teacher Feedback on Student Writing. EdWorkingPaper No. 25-1193
Language: English
Authors: Christopher Mah, Mei Tan, Lena Phalen, Alexa Sparks, Dorottya Demszky, Annenberg Institute for School Reform at Brown University
Source: Annenberg Institute for School Reform at Brown University. 2025.
Availability: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
Peer Reviewed: N
Page Count: 33
Publication Date: 2025
Document Type: Reports - Research
Descriptors: Artificial Intelligence, Technology Uses in Education, Natural Language Processing, Writing Evaluation, Feedback (Response), Sentences, Error Correction, Revision (Written Composition), Writing Teachers, Writing Skills, Positive Reinforcement
Abstract: Effective writing feedback is a powerful tool for enhancing student learning, encouraging revision, and increasing motivation and agency. Yet, teachers face many challenges that prevent them from consistently providing effective writing feedback. Recent advances in generative artificial intelligence (AI) have led educators and researchers to experiment with AI tools powered by large language models (LLMs) to provide writing feedback, but research in this area has yielded mixed results. In this study, we used qualitative methods to compare LLM writing feedback and expert teacher (n = 12) feedback. Using a framework of dialogic writing feedback as our analytic lens, we highlight differences in LLM and teacher feedback along three dimensions: cognitive, social, and structural. We observed that LLMs primarily enacted corrective feedback at the sentence level and positioned students as novices requiring remediation. By contrast, we observed that teachers enacted more dialogic feedback, offering feedback at multiple levels and employing tactics that positioned students as agentic writers. Our findings support previous research describing limitations of LLM-based writing feedback. More importantly, our study contributes to the growing base of research by identifying specific feedback practices unique to highly skilled teachers that LLMs did not exhibit. These findings have implications for improving the quality of LLM feedback and shifting teachers' practice to foreground the types of writing feedback that best promote independent thinking and writing skills students will need in the age of generative AI. [Funding for this report was received from the Stanford Institute for Human-Centered AI and Stanford Accelerator for Learning.]
Abstractor: As Provided
Entry Date: 2025
Accession Number: ED674108
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED674108
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED674108
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: From Sentence-Corrections to Deeper Dialogue: Qualitative Insights from LLM and Teacher Feedback on Student Writing. EdWorkingPaper No. 25-1193
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Christopher+Mah%22">Christopher Mah</searchLink><br /><searchLink fieldCode="AR" term="%22Mei+Tan%22">Mei Tan</searchLink><br /><searchLink fieldCode="AR" term="%22Lena+Phalen%22">Lena Phalen</searchLink><br /><searchLink fieldCode="AR" term="%22Alexa+Sparks%22">Alexa Sparks</searchLink><br /><searchLink fieldCode="AR" term="%22Dorottya+Demszky%22">Dorottya Demszky</searchLink><br /><searchLink fieldCode="AR" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22">Annenberg Institute for School Reform at Brown University</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22"><i>Annenberg Institute for School Reform at Brown University</i></searchLink>. 2025.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 33
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Evaluation%22">Writing Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Sentences%22">Sentences</searchLink><br /><searchLink fieldCode="DE" term="%22Error+Correction%22">Error Correction</searchLink><br /><searchLink fieldCode="DE" term="%22Revision+%28Written+Composition%29%22">Revision (Written Composition)</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Teachers%22">Writing Teachers</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Skills%22">Writing Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Positive+Reinforcement%22">Positive Reinforcement</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Effective writing feedback is a powerful tool for enhancing student learning, encouraging revision, and increasing motivation and agency. Yet, teachers face many challenges that prevent them from consistently providing effective writing feedback. Recent advances in generative artificial intelligence (AI) have led educators and researchers to experiment with AI tools powered by large language models (LLMs) to provide writing feedback, but research in this area has yielded mixed results. In this study, we used qualitative methods to compare LLM writing feedback and expert teacher (n = 12) feedback. Using a framework of dialogic writing feedback as our analytic lens, we highlight differences in LLM and teacher feedback along three dimensions: cognitive, social, and structural. We observed that LLMs primarily enacted corrective feedback at the sentence level and positioned students as novices requiring remediation. By contrast, we observed that teachers enacted more dialogic feedback, offering feedback at multiple levels and employing tactics that positioned students as agentic writers. Our findings support previous research describing limitations of LLM-based writing feedback. More importantly, our study contributes to the growing base of research by identifying specific feedback practices unique to highly skilled teachers that LLMs did not exhibit. These findings have implications for improving the quality of LLM feedback and shifting teachers' practice to foreground the types of writing feedback that best promote independent thinking and writing skills students will need in the age of generative AI. [Funding for this report was received from the Stanford Institute for Human-Centered AI and Stanford Accelerator for Learning.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED674108
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED674108
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 33
    Subjects:
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Writing Evaluation
        Type: general
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Sentences
        Type: general
      – SubjectFull: Error Correction
        Type: general
      – SubjectFull: Revision (Written Composition)
        Type: general
      – SubjectFull: Writing Teachers
        Type: general
      – SubjectFull: Writing Skills
        Type: general
      – SubjectFull: Positive Reinforcement
        Type: general
    Titles:
      – TitleFull: From Sentence-Corrections to Deeper Dialogue: Qualitative Insights from LLM and Teacher Feedback on Student Writing. EdWorkingPaper No. 25-1193
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Annenberg Institute for School Reform at Brown University
      – PersonEntity:
          Name:
            NameFull: Christopher Mah
      – PersonEntity:
          Name:
            NameFull: Mei Tan
      – PersonEntity:
          Name:
            NameFull: Lena Phalen
      – PersonEntity:
          Name:
            NameFull: Alexa Sparks
      – PersonEntity:
          Name:
            NameFull: Dorottya Demszky
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 05
              Type: published
              Y: 2025
          Titles:
            – TitleFull: Annenberg Institute for School Reform at Brown University
              Type: main
ResultId 1