Automated Feedback on Fluency and Complexity Measures for L2 Academic Paragraphs: Student Perspectives and Impacts on Human-rater Scores.
Saved in:
| Title: | Automated Feedback on Fluency and Complexity Measures for L2 Academic Paragraphs: Student Perspectives and Impacts on Human-rater Scores. |
|---|---|
| Authors: | MACFARLANE, EMILY1 macfarlane.emily.b4@tohoku.ac.jp, SPRING, RYAN1 spring.ryan.edward.c4@tohoku.ac.jp |
| Source: | Technology in Language Teaching & Learning. 2024, Vol. 6 Issue 3, p1-18. 18p. |
| Subject Terms: | *Student attitudes, *Cognitive load, *Writing education, *Academic discourse, *Psychological feedback, *Vocabulary |
| Abstract: | While complexity, accuracy, and fluency (CAF) measures are known to correlate with L2 writers' scores, less is known about the effectiveness of providing automated feedback based on these measures for improving writing performance. Furthermore, it remains unclear if improvements in specific CAF measures correspond to improvements in human-rater scores. Finally, the trade-off hypothesis predicts that all three components of CAF cannot be improved at once, so providing multiple CAF measures to students at the same time might cause cognitive overload and overwhelm students, reducing their ability to uptake the feedback. To examine these issues, a simple paragraph feedback tool was developed to provide input on number of words, vocabulary variety, supporting detail markers, and sentence length. The tool was implemented repeatedly in L2 academic paragraph writing lessons with 124 students. Improvement was assessed through complexity, accuracy, and fluency (CAF) measures and human-rater scores, while also gathering student opinions. Results showed significant improvements in number of words, sentence length, and supporting detail marker usage from pre- to post-treatment with improvements in number of words and vocabulary variety found to be the strongest predictors of improvements in human-rater scores. Students reported finding vocabulary variety feedback most helpful, while supporting detail marker feedback was perceived as most confusing. Notably, students reported very low cognitive load overall. This article also discusses pedagogical implications of these findings for L2 writing instruction and automated feedback implementation. [ABSTRACT FROM AUTHOR] |
| Copyright of Technology in Language Teaching & Learning is the property of Castledown Publishers and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 183259575 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Automated Feedback on Fluency and Complexity Measures for L2 Academic Paragraphs: Student Perspectives and Impacts on Human-rater Scores. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22MACFARLANE%2C+EMILY%22">MACFARLANE, EMILY</searchLink><relatesTo>1</relatesTo><i> macfarlane.emily.b4@tohoku.ac.jp</i><br /><searchLink fieldCode="AR" term="%22SPRING%2C+RYAN%22">SPRING, RYAN</searchLink><relatesTo>1</relatesTo><i> spring.ryan.edward.c4@tohoku.ac.jp</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Technology+in+Language+Teaching+%26+Learning%22">Technology in Language Teaching & Learning</searchLink>. 2024, Vol. 6 Issue 3, p1-18. 18p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Student+attitudes%22">Student attitudes</searchLink><br />*<searchLink fieldCode="DE" term="%22Cognitive+load%22">Cognitive load</searchLink><br />*<searchLink fieldCode="DE" term="%22Writing+education%22">Writing education</searchLink><br />*<searchLink fieldCode="DE" term="%22Academic+discourse%22">Academic discourse</searchLink><br />*<searchLink fieldCode="DE" term="%22Psychological+feedback%22">Psychological feedback</searchLink><br />*<searchLink fieldCode="DE" term="%22Vocabulary%22">Vocabulary</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: While complexity, accuracy, and fluency (CAF) measures are known to correlate with L2 writers' scores, less is known about the effectiveness of providing automated feedback based on these measures for improving writing performance. Furthermore, it remains unclear if improvements in specific CAF measures correspond to improvements in human-rater scores. Finally, the trade-off hypothesis predicts that all three components of CAF cannot be improved at once, so providing multiple CAF measures to students at the same time might cause cognitive overload and overwhelm students, reducing their ability to uptake the feedback. To examine these issues, a simple paragraph feedback tool was developed to provide input on number of words, vocabulary variety, supporting detail markers, and sentence length. The tool was implemented repeatedly in L2 academic paragraph writing lessons with 124 students. Improvement was assessed through complexity, accuracy, and fluency (CAF) measures and human-rater scores, while also gathering student opinions. Results showed significant improvements in number of words, sentence length, and supporting detail marker usage from pre- to post-treatment with improvements in number of words and vocabulary variety found to be the strongest predictors of improvements in human-rater scores. Students reported finding vocabulary variety feedback most helpful, while supporting detail marker feedback was perceived as most confusing. Notably, students reported very low cognitive load overall. This article also discusses pedagogical implications of these findings for L2 writing instruction and automated feedback implementation. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Technology in Language Teaching & Learning is the property of Castledown Publishers and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=183259575 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.29140/tltl.v6n3.1536 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 18 StartPage: 1 Subjects: – SubjectFull: Student attitudes Type: general – SubjectFull: Cognitive load Type: general – SubjectFull: Writing education Type: general – SubjectFull: Academic discourse Type: general – SubjectFull: Psychological feedback Type: general – SubjectFull: Vocabulary Type: general Titles: – TitleFull: Automated Feedback on Fluency and Complexity Measures for L2 Academic Paragraphs: Student Perspectives and Impacts on Human-rater Scores. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: MACFARLANE, EMILY – PersonEntity: Name: NameFull: SPRING, RYAN IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 10 Text: 2024 Type: published Y: 2024 Identifiers: – Type: issn-print Value: 26521687 Numbering: – Type: volume Value: 6 – Type: issue Value: 3 Titles: – TitleFull: Technology in Language Teaching & Learning Type: main |
| ResultId | 1 |
