Generative AI and CEFR Levels: Evaluating the Accuracy of Text Generation with ChatGPT-4o through Textual Features
Saved in:
| Title: | Generative AI and CEFR Levels: Evaluating the Accuracy of Text Generation with ChatGPT-4o through Textual Features |
|---|---|
| Language: | English |
| Authors: | Satoru Uchida (ORCID |
| Source: | Vocabulary Learning and Instruction. 2025 14(1). |
| Availability: | Castledown Publishers. Ground Level, 470 St Kilda Road, Melbourne, 3004, Australia. Tel: +61-3-7003-8355; e-mail: contact@castledown.com; Web site: https://www.castledown.com/journals/vli |
| Peer Reviewed: | Y |
| Page Count: | 13 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Vocabulary Development, Language Proficiency, Rating Scales, Guidelines, Standards, Artificial Intelligence, Computer Software, Textbooks, Technology Integration, English (Second Language), Second Language Learning, Second Language Instruction, Computational Linguistics, Text Structure, Comparative Analysis, Material Development |
| ISSN: | 2981-9954 |
| Abstract: | Since its emergence, generative AI has significantly impacted various fields, including English language education. Numerous academic studies have investigated its capabilities in grammar correction, writing evaluation, and dynamics of user interaction. However, there have been insufficient investigations into whether texts generated by such AI align appropriately with CEFR proficiency levels. This study addresses this gap by exploring the applicability of generative AI to CEFR standards. Multiple texts were generated using ChatGPT-4o with specified CEFR levels and analyzed using a vocabulary level analyzer (CVLA) to evaluate text features. The findings revealed discrepancies between AI-generated texts and textbook standards, significant divergences between levels below B1 and above B2, and a noticeable topic bias. Although AI-generated texts seem to differ by level, they require careful evaluation before being applied to CEFR-based education. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1466280 |
| Database: | ERIC |
| Abstract: | Since its emergence, generative AI has significantly impacted various fields, including English language education. Numerous academic studies have investigated its capabilities in grammar correction, writing evaluation, and dynamics of user interaction. However, there have been insufficient investigations into whether texts generated by such AI align appropriately with CEFR proficiency levels. This study addresses this gap by exploring the applicability of generative AI to CEFR standards. Multiple texts were generated using ChatGPT-4o with specified CEFR levels and analyzed using a vocabulary level analyzer (CVLA) to evaluate text features. The findings revealed discrepancies between AI-generated texts and textbook standards, significant divergences between levels below B1 and above B2, and a noticeable topic bias. Although AI-generated texts seem to differ by level, they require careful evaluation before being applied to CEFR-based education. |
|---|---|
| ISSN: | 2981-9954 |
