View in EDS

Standards-aligned annotations reveal organizational patterns in argumentative essays at scale.

Saved in:

Bibliographic Details
Title:	Standards-aligned annotations reveal organizational patterns in argumentative essays at scale.
Authors:	Burkhardt, Amy¹ (AUTHOR) amy.burkhardt@cambiumassessment.com, Han, Suhwa¹ (AUTHOR), Woolf, Sherri¹ (AUTHOR), Boykin, Allison¹ (AUTHOR), Rijmen, Frank¹ (AUTHOR), Lottridge, Susan¹ (AUTHOR)
Source:	Frontiers in Education. 2025, p1-13. 13p.
Subject Terms:	Educational standards, Academic discourse, Scoring rubrics, Inter-observer reliability, Annotations, Essays, Latent class analysis (Statistics)
Abstract:	While scoring rubrics are widely used to evaluate student writing, they often fail to provide actionable feedback. Delivering such feedback—especially in an automated, scalable manner—requires the standardized detection of finer-grained information within a student's essay. Achieving this level of detail demands the same rigor in development and training as creating a high-quality rubric. To this end, we describe the development of annotation guidelines aligned with state standards for detecting these elements, outline the annotator training process, and report strong inter-rater agreement results from a large-scale annotation effort involving nearly 20,000 essays. To further validate this approach, we connect annotations to broader patterns in student writing using Latent Class Analysis (LCA). Through this analysis, we identify distinct writing patterns from these fine-grained annotations and demonstrate their meaningful associations with overall rubric scores. Our findings show promise for how fine-grained analysis of argumentative essays can support students, at scale, in becoming more effective argumentative essay writers. [ABSTRACT FROM AUTHOR]
	Copyright of Frontiers in Education is the property of Frontiers Media S.A. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Education Research Complete

Description
Abstract:	While scoring rubrics are widely used to evaluate student writing, they often fail to provide actionable feedback. Delivering such feedback—especially in an automated, scalable manner—requires the standardized detection of finer-grained information within a student's essay. Achieving this level of detail demands the same rigor in development and training as creating a high-quality rubric. To this end, we describe the development of annotation guidelines aligned with state standards for detecting these elements, outline the annotator training process, and report strong inter-rater agreement results from a large-scale annotation effort involving nearly 20,000 essays. To further validate this approach, we connect annotations to broader patterns in student writing using Latent Class Analysis (LCA). Through this analysis, we identify distinct writing patterns from these fine-grained annotations and demonstrate their meaningful associations with overall rubric scores. Our findings show promise for how fine-grained analysis of argumentative essays can support students, at scale, in becoming more effective argumentative essay writers. [ABSTRACT FROM AUTHOR]
ISSN:	2504284X
DOI:	10.3389/feduc.2025.1569529