Evaluating the applicability of the transformer-based grammatical error correction system for assessing language accuracy.
Saved in:
| Title: | Evaluating the applicability of the transformer-based grammatical error correction system for assessing language accuracy. |
|---|---|
| Authors: | Hwang, Haerim1 haerimhwang@cuhk.edu.hk |
| Source: | Language Testing in Asia. 12/4/2025, Vol. 15 Issue 1, p1-11. 11p. |
| Subject Terms: | *Fluency (Language learning), *Sentences (Grammar), *Mixed methods research, Linguistic errors, Korean language |
| Abstract: | The current study focuses on the use of Grammatical Error Correction (GEC) technology for assessing language accuracy, which has received relatively less attention than complexity and fluency in the context of automated assessment. Adopting a technology-enhanced approach to language assessment, rather than a technology-driven approach, we critically assessed the suitability of the state-of-the-art GEC system for assessing language accuracy in Korean, an understudied language in this regard. We analyzed how reliable this system is quantitatively and what types of error can be generated by this system qualitatively. Also, we used out-of-domain, inclusive data from heritage speakers of Korean, which has never been considered in the development of GEC. Our accuracy analyses show that the system achieves a fairly high accuracy in differentiating between correct and incorrect sentences on our data (F0.5 = 0.819). However, the system exhibits a tendency to make unnecessary corrections, such as inserting topics/adverbials or correcting particles, while failing to correct ungrammatical ones in some cases. These findings from our mixed-method analyses suggest that language evaluators should recognize the potential for inaccurate assessments when using a GEC system, as its output may be incorrect at this moment, thus highlighting the critical need for digital language assessment literacy. [ABSTRACT FROM AUTHOR] |
| Copyright of Language Testing in Asia is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 190940115 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Evaluating the applicability of the transformer-based grammatical error correction system for assessing language accuracy. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Hwang%2C+Haerim%22">Hwang, Haerim</searchLink><relatesTo>1</relatesTo><i> haerimhwang@cuhk.edu.hk</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Language+Testing+in+Asia%22">Language Testing in Asia</searchLink>. 12/4/2025, Vol. 15 Issue 1, p1-11. 11p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Fluency+%28Language+learning%29%22">Fluency (Language learning)</searchLink><br />*<searchLink fieldCode="DE" term="%22Sentences+%28Grammar%29%22">Sentences (Grammar)</searchLink><br />*<searchLink fieldCode="DE" term="%22Mixed+methods+research%22">Mixed methods research</searchLink><br /><searchLink fieldCode="DE" term="%22Linguistic+errors%22">Linguistic errors</searchLink><br /><searchLink fieldCode="DE" term="%22Korean+language%22">Korean language</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: The current study focuses on the use of Grammatical Error Correction (GEC) technology for assessing language accuracy, which has received relatively less attention than complexity and fluency in the context of automated assessment. Adopting a technology-enhanced approach to language assessment, rather than a technology-driven approach, we critically assessed the suitability of the state-of-the-art GEC system for assessing language accuracy in Korean, an understudied language in this regard. We analyzed how reliable this system is quantitatively and what types of error can be generated by this system qualitatively. Also, we used out-of-domain, inclusive data from heritage speakers of Korean, which has never been considered in the development of GEC. Our accuracy analyses show that the system achieves a fairly high accuracy in differentiating between correct and incorrect sentences on our data (F0.5 = 0.819). However, the system exhibits a tendency to make unnecessary corrections, such as inserting topics/adverbials or correcting particles, while failing to correct ungrammatical ones in some cases. These findings from our mixed-method analyses suggest that language evaluators should recognize the potential for inaccurate assessments when using a GEC system, as its output may be incorrect at this moment, thus highlighting the critical need for digital language assessment literacy. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Language Testing in Asia is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=190940115 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1186/s40468-025-00415-3 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 11 StartPage: 1 Subjects: – SubjectFull: Fluency (Language learning) Type: general – SubjectFull: Sentences (Grammar) Type: general – SubjectFull: Mixed methods research Type: general – SubjectFull: Linguistic errors Type: general – SubjectFull: Korean language Type: general Titles: – TitleFull: Evaluating the applicability of the transformer-based grammatical error correction system for assessing language accuracy. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Hwang, Haerim IsPartOfRelationships: – BibEntity: Dates: – D: 04 M: 12 Text: 12/4/2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 22290443 Numbering: – Type: volume Value: 15 – Type: issue Value: 1 Titles: – TitleFull: Language Testing in Asia Type: main |
| ResultId | 1 |
