Automated Pipeline for Multi-Lingual Automated Essay Scoring with ReaderBench
Saved in:
| Title: | Automated Pipeline for Multi-Lingual Automated Essay Scoring with ReaderBench |
|---|---|
| Language: | English |
| Authors: | Stefan Ruseti (ORCID |
| Source: | International Journal of Artificial Intelligence in Education. 2024 34(4):1460-1481. |
| Availability: | Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ |
| Peer Reviewed: | Y |
| Page Count: | 22 |
| Publication Date: | 2024 |
| Sponsoring Agency: | Institute of Education Sciences (ED) |
| Contract Number: | R305A180261 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Computer Assisted Testing, Scoring, Automation, Essays, Natural Language Processing, Artificial Intelligence, English, Portuguese, French, Algorithms, Computer Interfaces, Prediction |
| DOI: | 10.1007/s40593-024-00402-4 |
| ISSN: | 1560-4292 1560-4306 |
| Abstract: | Automated Essay Scoring (AES) is a well-studied problem in Natural Language Processing applied in education. Solutions vary from handcrafted linguistic features to large Transformer-based models, implying a significant effort in feature extraction and model implementation. We introduce a novel Automated Machine Learning (AutoML) pipeline integrated into the ReaderBench platform designed to simplify the process of training AES models by automating both feature extraction and architecture tuning for any multilingual dataset uploaded by the user. The dataset must contain a list of texts, each with potentially multiple annotations, either scores or labels. The platform includes traditional ML models relying on linguistic features and a hybrid approach combining Transformer-based architectures with the previous features. Our method was evaluated on three publicly available datasets in three different languages (English, Portuguese, and French) and compared with the best currently published results on these datasets. Our automated approach achieved comparable results to state-of-the-art models on two datasets, while it obtained the best performance on the third corpus in Portuguese. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2024 |
| Accession Number: | EJ1453656 |
| Database: | ERIC |
| Abstract: | Automated Essay Scoring (AES) is a well-studied problem in Natural Language Processing applied in education. Solutions vary from handcrafted linguistic features to large Transformer-based models, implying a significant effort in feature extraction and model implementation. We introduce a novel Automated Machine Learning (AutoML) pipeline integrated into the ReaderBench platform designed to simplify the process of training AES models by automating both feature extraction and architecture tuning for any multilingual dataset uploaded by the user. The dataset must contain a list of texts, each with potentially multiple annotations, either scores or labels. The platform includes traditional ML models relying on linguistic features and a hybrid approach combining Transformer-based architectures with the previous features. Our method was evaluated on three publicly available datasets in three different languages (English, Portuguese, and French) and compared with the best currently published results on these datasets. Our automated approach achieved comparable results to state-of-the-art models on two datasets, while it obtained the best performance on the third corpus in Portuguese. |
|---|---|
| ISSN: | 1560-4292 1560-4306 |
| DOI: | 10.1007/s40593-024-00402-4 |
