GenAI Models as Keyword Rankers: A Learner-Centred Case Study for L2 Spanish
Saved in:
| Title: | GenAI Models as Keyword Rankers: A Learner-Centred Case Study for L2 Spanish |
|---|---|
| Language: | English |
| Authors: | Jasper Degraeuwe (ORCID |
| Source: | The EUROCALL Review. 2025 32(2):63-75. |
| Availability: | European Association for Computer-Assisted Language Learning (EUROCALL). EUROCALL Headquarters, School of Modern Languages, University of Ulster, Cromore Road, Coleraine BT52 1SA, Northern Ireland, UK. Tel: +34-67-943-1283; Web site: http://www.eurocall-languages.org/ |
| Peer Reviewed: | Y |
| Page Count: | 13 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Second Language Learning, Spanish, Word Lists, Word Frequency, Vocabulary Development, Artificial Intelligence, Intellectual Disciplines, Indo European Languages, Computer Uses in Education, Educational Technology, Units of Study |
| ISSN: | 1695-2618 |
| Abstract: | Frequency-based word lists form an important part of general-purpose vocabulary learning courses aimed at beginner and (lower-)intermediate learners of a foreign/second language (L2). For advanced learners and/or specific purposes, however, relying exclusively on these general word lists will be unlikely to lead to an adequate selection of vocabulary. As research in this latter area remains scarce (especially for languages other than English), the present study aims to fill (part of) the gap by investigating the use of Generative Artificial Intelligence (GenAI) models to automatically rank vocabulary items based on how typical they are of a given topic, focusing on Spanish as the target language. I compile a dataset containing four domain-specific subsets of 200 vocabulary items (for the topics economics, health, law, and migration) and analyse how well GenAI-based rankings of these vocabulary items (using zero-shot prompting) correlate with gold standard human rankings (provided by L2 learners). As the evaluation baseline, I use the rankings obtained by means of the Kullback-Leibler divergence (i.e., a statistical "keyness" measure based on word frequencies). With a top average Spearman's ? and Kendall's weighted [tau] of 0.73, this first-of-its-kind study demonstrates that the tested GenAI models (Gemma, Llama, and Mistral) outperform the baseline by a large margin, showing great potential for use in the real-life creation of domain-specific vocabulary lists for L2 learning purposes. [Note: The page range (63-74) shown on the PDF is incorrect. The correct page range is 63-75.] |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1494351 |
| Database: | ERIC |
| Abstract: | Frequency-based word lists form an important part of general-purpose vocabulary learning courses aimed at beginner and (lower-)intermediate learners of a foreign/second language (L2). For advanced learners and/or specific purposes, however, relying exclusively on these general word lists will be unlikely to lead to an adequate selection of vocabulary. As research in this latter area remains scarce (especially for languages other than English), the present study aims to fill (part of) the gap by investigating the use of Generative Artificial Intelligence (GenAI) models to automatically rank vocabulary items based on how typical they are of a given topic, focusing on Spanish as the target language. I compile a dataset containing four domain-specific subsets of 200 vocabulary items (for the topics economics, health, law, and migration) and analyse how well GenAI-based rankings of these vocabulary items (using zero-shot prompting) correlate with gold standard human rankings (provided by L2 learners). As the evaluation baseline, I use the rankings obtained by means of the Kullback-Leibler divergence (i.e., a statistical "keyness" measure based on word frequencies). With a top average Spearman's ? and Kendall's weighted [tau] of 0.73, this first-of-its-kind study demonstrates that the tested GenAI models (Gemma, Llama, and Mistral) outperform the baseline by a large margin, showing great potential for use in the real-life creation of domain-specific vocabulary lists for L2 learning purposes. [Note: The page range (63-74) shown on the PDF is incorrect. The correct page range is 63-75.] |
|---|---|
| ISSN: | 1695-2618 |
