View in EDS

Generative AI and CEFR Levels: Evaluating the Accuracy of Text Generation with ChatGPT-4o through Textual Features

Saved in:
Bibliographic Details
Title: Generative AI and CEFR Levels: Evaluating the Accuracy of Text Generation with ChatGPT-4o through Textual Features
Language: English
Authors: Satoru Uchida (ORCID 0000-0002-7779-0085)
Source: Vocabulary Learning and Instruction. 2025 14(1).
Availability: Castledown Publishers. Ground Level, 470 St Kilda Road, Melbourne, 3004, Australia. Tel: +61-3-7003-8355; e-mail: contact@castledown.com; Web site: https://www.castledown.com/journals/vli
Peer Reviewed: Y
Page Count: 13
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Descriptors: Vocabulary Development, Language Proficiency, Rating Scales, Guidelines, Standards, Artificial Intelligence, Computer Software, Textbooks, Technology Integration, English (Second Language), Second Language Learning, Second Language Instruction, Computational Linguistics, Text Structure, Comparative Analysis, Material Development
ISSN: 2981-9954
Abstract: Since its emergence, generative AI has significantly impacted various fields, including English language education. Numerous academic studies have investigated its capabilities in grammar correction, writing evaluation, and dynamics of user interaction. However, there have been insufficient investigations into whether texts generated by such AI align appropriately with CEFR proficiency levels. This study addresses this gap by exploring the applicability of generative AI to CEFR standards. Multiple texts were generated using ChatGPT-4o with specified CEFR levels and analyzed using a vocabulary level analyzer (CVLA) to evaluate text features. The findings revealed discrepancies between AI-generated texts and textbook standards, significant divergences between levels below B1 and above B2, and a noticeable topic bias. Although AI-generated texts seem to differ by level, they require careful evaluation before being applied to CEFR-based education.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1466280
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1466280
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1466280
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Generative AI and CEFR Levels: Evaluating the Accuracy of Text Generation with ChatGPT-4o through Textual Features
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Satoru+Uchida%22">Satoru Uchida</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-7779-0085">0000-0002-7779-0085</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Vocabulary+Learning+and+Instruction%22"><i>Vocabulary Learning and Instruction</i></searchLink>. 2025 14(1).
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Castledown Publishers. Ground Level, 470 St Kilda Road, Melbourne, 3004, Australia. Tel: +61-3-7003-8355; e-mail: contact@castledown.com; Web site: https://www.castledown.com/journals/vli
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 13
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Vocabulary+Development%22">Vocabulary Development</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Proficiency%22">Language Proficiency</searchLink><br /><searchLink fieldCode="DE" term="%22Rating+Scales%22">Rating Scales</searchLink><br /><searchLink fieldCode="DE" term="%22Guidelines%22">Guidelines</searchLink><br /><searchLink fieldCode="DE" term="%22Standards%22">Standards</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software%22">Computer Software</searchLink><br /><searchLink fieldCode="DE" term="%22Textbooks%22">Textbooks</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Integration%22">Technology Integration</searchLink><br /><searchLink fieldCode="DE" term="%22English+%28Second+Language%29%22">English (Second Language)</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Learning%22">Second Language Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Instruction%22">Second Language Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Computational+Linguistics%22">Computational Linguistics</searchLink><br /><searchLink fieldCode="DE" term="%22Text+Structure%22">Text Structure</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Material+Development%22">Material Development</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2981-9954
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Since its emergence, generative AI has significantly impacted various fields, including English language education. Numerous academic studies have investigated its capabilities in grammar correction, writing evaluation, and dynamics of user interaction. However, there have been insufficient investigations into whether texts generated by such AI align appropriately with CEFR proficiency levels. This study addresses this gap by exploring the applicability of generative AI to CEFR standards. Multiple texts were generated using ChatGPT-4o with specified CEFR levels and analyzed using a vocabulary level analyzer (CVLA) to evaluate text features. The findings revealed discrepancies between AI-generated texts and textbook standards, significant divergences between levels below B1 and above B2, and a noticeable topic bias. Although AI-generated texts seem to differ by level, they require careful evaluation before being applied to CEFR-based education.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1466280
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1466280
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 13
    Subjects:
      – SubjectFull: Vocabulary Development
        Type: general
      – SubjectFull: Language Proficiency
        Type: general
      – SubjectFull: Rating Scales
        Type: general
      – SubjectFull: Guidelines
        Type: general
      – SubjectFull: Standards
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Computer Software
        Type: general
      – SubjectFull: Textbooks
        Type: general
      – SubjectFull: Technology Integration
        Type: general
      – SubjectFull: English (Second Language)
        Type: general
      – SubjectFull: Second Language Learning
        Type: general
      – SubjectFull: Second Language Instruction
        Type: general
      – SubjectFull: Computational Linguistics
        Type: general
      – SubjectFull: Text Structure
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Material Development
        Type: general
    Titles:
      – TitleFull: Generative AI and CEFR Levels: Evaluating the Accuracy of Text Generation with ChatGPT-4o through Textual Features
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Satoru Uchida
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-electronic
              Value: 2981-9954
          Numbering:
            – Type: volume
              Value: 14
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Vocabulary Learning and Instruction
              Type: main
ResultId 1