Standards-aligned annotations reveal organizational patterns in argumentative essays at scale.
Saved in:
| Title: | Standards-aligned annotations reveal organizational patterns in argumentative essays at scale. |
|---|---|
| Authors: | Burkhardt, Amy1 (AUTHOR) amy.burkhardt@cambiumassessment.com, Han, Suhwa1 (AUTHOR), Woolf, Sherri1 (AUTHOR), Boykin, Allison1 (AUTHOR), Rijmen, Frank1 (AUTHOR), Lottridge, Susan1 (AUTHOR) |
| Source: | Frontiers in Education. 2025, p1-13. 13p. |
| Subject Terms: | *Educational standards, *Academic discourse, *Scoring rubrics, *Inter-observer reliability, Annotations, Essays, Latent class analysis (Statistics) |
| Abstract: | While scoring rubrics are widely used to evaluate student writing, they often fail to provide actionable feedback. Delivering such feedback—especially in an automated, scalable manner—requires the standardized detection of finer-grained information within a student's essay. Achieving this level of detail demands the same rigor in development and training as creating a high-quality rubric. To this end, we describe the development of annotation guidelines aligned with state standards for detecting these elements, outline the annotator training process, and report strong inter-rater agreement results from a large-scale annotation effort involving nearly 20,000 essays. To further validate this approach, we connect annotations to broader patterns in student writing using Latent Class Analysis (LCA). Through this analysis, we identify distinct writing patterns from these fine-grained annotations and demonstrate their meaningful associations with overall rubric scores. Our findings show promise for how fine-grained analysis of argumentative essays can support students, at scale, in becoming more effective argumentative essay writers. [ABSTRACT FROM AUTHOR] |
| Copyright of Frontiers in Education is the property of Frontiers Media S.A. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 186393693 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Standards-aligned annotations reveal organizational patterns in argumentative essays at scale. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Burkhardt%2C+Amy%22">Burkhardt, Amy</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> amy.burkhardt@cambiumassessment.com</i><br /><searchLink fieldCode="AR" term="%22Han%2C+Suhwa%22">Han, Suhwa</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Woolf%2C+Sherri%22">Woolf, Sherri</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Boykin%2C+Allison%22">Boykin, Allison</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Rijmen%2C+Frank%22">Rijmen, Frank</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Lottridge%2C+Susan%22">Lottridge, Susan</searchLink><relatesTo>1</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Frontiers+in+Education%22">Frontiers in Education</searchLink>. 2025, p1-13. 13p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Educational+standards%22">Educational standards</searchLink><br />*<searchLink fieldCode="DE" term="%22Academic+discourse%22">Academic discourse</searchLink><br />*<searchLink fieldCode="DE" term="%22Scoring+rubrics%22">Scoring rubrics</searchLink><br />*<searchLink fieldCode="DE" term="%22Inter-observer+reliability%22">Inter-observer reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Annotations%22">Annotations</searchLink><br /><searchLink fieldCode="DE" term="%22Essays%22">Essays</searchLink><br /><searchLink fieldCode="DE" term="%22Latent+class+analysis+%28Statistics%29%22">Latent class analysis (Statistics)</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: While scoring rubrics are widely used to evaluate student writing, they often fail to provide actionable feedback. Delivering such feedback—especially in an automated, scalable manner—requires the standardized detection of finer-grained information within a student's essay. Achieving this level of detail demands the same rigor in development and training as creating a high-quality rubric. To this end, we describe the development of annotation guidelines aligned with state standards for detecting these elements, outline the annotator training process, and report strong inter-rater agreement results from a large-scale annotation effort involving nearly 20,000 essays. To further validate this approach, we connect annotations to broader patterns in student writing using Latent Class Analysis (LCA). Through this analysis, we identify distinct writing patterns from these fine-grained annotations and demonstrate their meaningful associations with overall rubric scores. Our findings show promise for how fine-grained analysis of argumentative essays can support students, at scale, in becoming more effective argumentative essay writers. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Frontiers in Education is the property of Frontiers Media S.A. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=186393693 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.3389/feduc.2025.1569529 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 13 StartPage: 1 Subjects: – SubjectFull: Educational standards Type: general – SubjectFull: Academic discourse Type: general – SubjectFull: Scoring rubrics Type: general – SubjectFull: Inter-observer reliability Type: general – SubjectFull: Annotations Type: general – SubjectFull: Essays Type: general – SubjectFull: Latent class analysis (Statistics) Type: general Titles: – TitleFull: Standards-aligned annotations reveal organizational patterns in argumentative essays at scale. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Burkhardt, Amy – PersonEntity: Name: NameFull: Han, Suhwa – PersonEntity: Name: NameFull: Woolf, Sherri – PersonEntity: Name: NameFull: Boykin, Allison – PersonEntity: Name: NameFull: Rijmen, Frank – PersonEntity: Name: NameFull: Lottridge, Susan IsPartOfRelationships: – BibEntity: Dates: – D: 03 M: 07 Text: 2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 2504284X Titles: – TitleFull: Frontiers in Education Type: main |
| ResultId | 1 |
