The possibility of improving automated calculation of measures of lexical richness for EFL writing: A comparison of the LCA, NLTK and SpaCy tools.
Saved in:
| Title: | The possibility of improving automated calculation of measures of lexical richness for EFL writing: A comparison of the LCA, NLTK and SpaCy tools. |
|---|---|
| Authors: | Spring, Ryan1 spring.ryan.edward.c4@tohoku.ac.jp, Johnson, Matthew2 |
| Source: | System. Jun2022, Vol. 106, pN.PAG-N.PAG. 1p. |
| Subject Terms: | *Lexical access, *Statistical correlation, Natural language processing, Linguistics, Sophistication (The English word) |
| Abstract: | Automatically calculating measures of lexical richness is important for L2 learning because they can be used for assessment of productive abilities and general linguistic ability. One popular tool for doing so is the Lexical Complexity Analyzer (LCA), but more advanced tools for parsing have become available since its creation. This paper compares a modified version of the LCA code run with NLTK and SpaCy, two popular natural language processing toolkits, and the online version of the LCA to calculate 26 measures of lexical richness. We show how similarly they calculate the measures and how well each of the three tools' calculations correlate with EFL writer's human-rated scores and TOEFL® ITP scores. We found that six of the measures suggested to be associated with higher oral proficiency by Lu (2012) were also highly correlated with higher human-rated scores and TOEFL® ITP scores in our data set. However, the modifications to our code that utilize a different list to determine word sophistication and allow be and have verbs to be treated as lexical verbs caused four measures which Lu (2012) found to be unassociated with proficiency to be correlated with both human-rated scores and TOEFL® ITP scores, particularly when run with SpaCy. • SpaCy and NLTK based tools were compared with the LCA. • The three tools performed similarly, but SpaCy provided measures most correlated to human-rating and TOEFL® ITP scores. • Code created for SpaCy and NLTK based tools modified definitions of word sophistication and lexical verbs. • Measures that showed high correlation to human-rating and TOEFL(R) ITP scores include: NDW, NDW-ER50, CVS1, CTTR, SVV1, and LS1. [ABSTRACT FROM AUTHOR] |
| Copyright of System is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 156520149 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: The possibility of improving automated calculation of measures of lexical richness for EFL writing: A comparison of the LCA, NLTK and SpaCy tools. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Spring%2C+Ryan%22">Spring, Ryan</searchLink><relatesTo>1</relatesTo><i> spring.ryan.edward.c4@tohoku.ac.jp</i><br /><searchLink fieldCode="AR" term="%22Johnson%2C+Matthew%22">Johnson, Matthew</searchLink><relatesTo>2</relatesTo> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22System%22">System</searchLink>. Jun2022, Vol. 106, pN.PAG-N.PAG. 1p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Lexical+access%22">Lexical access</searchLink><br />*<searchLink fieldCode="DE" term="%22Statistical+correlation%22">Statistical correlation</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+language+processing%22">Natural language processing</searchLink><br /><searchLink fieldCode="DE" term="%22Linguistics%22">Linguistics</searchLink><br /><searchLink fieldCode="DE" term="%22Sophistication+%28The+English+word%29%22">Sophistication (The English word)</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Automatically calculating measures of lexical richness is important for L2 learning because they can be used for assessment of productive abilities and general linguistic ability. One popular tool for doing so is the Lexical Complexity Analyzer (LCA), but more advanced tools for parsing have become available since its creation. This paper compares a modified version of the LCA code run with NLTK and SpaCy, two popular natural language processing toolkits, and the online version of the LCA to calculate 26 measures of lexical richness. We show how similarly they calculate the measures and how well each of the three tools' calculations correlate with EFL writer's human-rated scores and TOEFL® ITP scores. We found that six of the measures suggested to be associated with higher oral proficiency by Lu (2012) were also highly correlated with higher human-rated scores and TOEFL® ITP scores in our data set. However, the modifications to our code that utilize a different list to determine word sophistication and allow be and have verbs to be treated as lexical verbs caused four measures which Lu (2012) found to be unassociated with proficiency to be correlated with both human-rated scores and TOEFL® ITP scores, particularly when run with SpaCy. • SpaCy and NLTK based tools were compared with the LCA. • The three tools performed similarly, but SpaCy provided measures most correlated to human-rating and TOEFL® ITP scores. • Code created for SpaCy and NLTK based tools modified definitions of word sophistication and lexical verbs. • Measures that showed high correlation to human-rating and TOEFL(R) ITP scores include: NDW, NDW-ER50, CVS1, CTTR, SVV1, and LS1. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of System is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=156520149 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1016/j.system.2022.102770 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 1 StartPage: N.PAG Subjects: – SubjectFull: Lexical access Type: general – SubjectFull: Statistical correlation Type: general – SubjectFull: Natural language processing Type: general – SubjectFull: Linguistics Type: general – SubjectFull: Sophistication (The English word) Type: general Titles: – TitleFull: The possibility of improving automated calculation of measures of lexical richness for EFL writing: A comparison of the LCA, NLTK and SpaCy tools. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Spring, Ryan – PersonEntity: Name: NameFull: Johnson, Matthew IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 06 Text: Jun2022 Type: published Y: 2022 Identifiers: – Type: issn-print Value: 0346251X Numbering: – Type: volume Value: 106 Titles: – TitleFull: System Type: main |
| ResultId | 1 |
