View in EDS

The possibility of improving automated calculation of measures of lexical richness for EFL writing: A comparison of the LCA, NLTK and SpaCy tools.

Saved in:
Bibliographic Details
Title: The possibility of improving automated calculation of measures of lexical richness for EFL writing: A comparison of the LCA, NLTK and SpaCy tools.
Authors: Spring, Ryan1 spring.ryan.edward.c4@tohoku.ac.jp, Johnson, Matthew2
Source: System. Jun2022, Vol. 106, pN.PAG-N.PAG. 1p.
Subject Terms: *Lexical access, *Statistical correlation, Natural language processing, Linguistics, Sophistication (The English word)
Abstract: Automatically calculating measures of lexical richness is important for L2 learning because they can be used for assessment of productive abilities and general linguistic ability. One popular tool for doing so is the Lexical Complexity Analyzer (LCA), but more advanced tools for parsing have become available since its creation. This paper compares a modified version of the LCA code run with NLTK and SpaCy, two popular natural language processing toolkits, and the online version of the LCA to calculate 26 measures of lexical richness. We show how similarly they calculate the measures and how well each of the three tools' calculations correlate with EFL writer's human-rated scores and TOEFL® ITP scores. We found that six of the measures suggested to be associated with higher oral proficiency by Lu (2012) were also highly correlated with higher human-rated scores and TOEFL® ITP scores in our data set. However, the modifications to our code that utilize a different list to determine word sophistication and allow be and have verbs to be treated as lexical verbs caused four measures which Lu (2012) found to be unassociated with proficiency to be correlated with both human-rated scores and TOEFL® ITP scores, particularly when run with SpaCy. • SpaCy and NLTK based tools were compared with the LCA. • The three tools performed similarly, but SpaCy provided measures most correlated to human-rating and TOEFL® ITP scores. • Code created for SpaCy and NLTK based tools modified definitions of word sophistication and lexical verbs. • Measures that showed high correlation to human-rating and TOEFL(R) ITP scores include: NDW, NDW-ER50, CVS1, CTTR, SVV1, and LS1. [ABSTRACT FROM AUTHOR]
Copyright of System is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
Description
Abstract:Automatically calculating measures of lexical richness is important for L2 learning because they can be used for assessment of productive abilities and general linguistic ability. One popular tool for doing so is the Lexical Complexity Analyzer (LCA), but more advanced tools for parsing have become available since its creation. This paper compares a modified version of the LCA code run with NLTK and SpaCy, two popular natural language processing toolkits, and the online version of the LCA to calculate 26 measures of lexical richness. We show how similarly they calculate the measures and how well each of the three tools' calculations correlate with EFL writer's human-rated scores and TOEFL® ITP scores. We found that six of the measures suggested to be associated with higher oral proficiency by Lu (2012) were also highly correlated with higher human-rated scores and TOEFL® ITP scores in our data set. However, the modifications to our code that utilize a different list to determine word sophistication and allow be and have verbs to be treated as lexical verbs caused four measures which Lu (2012) found to be unassociated with proficiency to be correlated with both human-rated scores and TOEFL® ITP scores, particularly when run with SpaCy. • SpaCy and NLTK based tools were compared with the LCA. • The three tools performed similarly, but SpaCy provided measures most correlated to human-rating and TOEFL® ITP scores. • Code created for SpaCy and NLTK based tools modified definitions of word sophistication and lexical verbs. • Measures that showed high correlation to human-rating and TOEFL(R) ITP scores include: NDW, NDW-ER50, CVS1, CTTR, SVV1, and LS1. [ABSTRACT FROM AUTHOR]
ISSN:0346251X
DOI:10.1016/j.system.2022.102770