View in EDS

Automated Pipeline for Multi-Lingual Automated Essay Scoring with ReaderBench

Saved in:
Bibliographic Details
Title: Automated Pipeline for Multi-Lingual Automated Essay Scoring with ReaderBench
Language: English
Authors: Stefan Ruseti (ORCID 0000-0002-0380-6814), Ionut Paraschiv, Mihai Dascalu (ORCID 0000-0002-4815-9227), Danielle S. McNamara
Source: International Journal of Artificial Intelligence in Education. 2024 34(4):1460-1481.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 22
Publication Date: 2024
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305A180261
Document Type: Journal Articles
Reports - Research
Descriptors: Computer Assisted Testing, Scoring, Automation, Essays, Natural Language Processing, Artificial Intelligence, English, Portuguese, French, Algorithms, Computer Interfaces, Prediction
DOI: 10.1007/s40593-024-00402-4
ISSN: 1560-4292
1560-4306
Abstract: Automated Essay Scoring (AES) is a well-studied problem in Natural Language Processing applied in education. Solutions vary from handcrafted linguistic features to large Transformer-based models, implying a significant effort in feature extraction and model implementation. We introduce a novel Automated Machine Learning (AutoML) pipeline integrated into the ReaderBench platform designed to simplify the process of training AES models by automating both feature extraction and architecture tuning for any multilingual dataset uploaded by the user. The dataset must contain a list of texts, each with potentially multiple annotations, either scores or labels. The platform includes traditional ML models relying on linguistic features and a hybrid approach combining Transformer-based architectures with the previous features. Our method was evaluated on three publicly available datasets in three different languages (English, Portuguese, and French) and compared with the best currently published results on these datasets. Our automated approach achieved comparable results to state-of-the-art models on two datasets, while it obtained the best performance on the third corpus in Portuguese.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2024
Accession Number: EJ1453656
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1453656
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Automated Pipeline for Multi-Lingual Automated Essay Scoring with ReaderBench
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Stefan+Ruseti%22">Stefan Ruseti</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-0380-6814">0000-0002-0380-6814</externalLink>)<br /><searchLink fieldCode="AR" term="%22Ionut+Paraschiv%22">Ionut Paraschiv</searchLink><br /><searchLink fieldCode="AR" term="%22Mihai+Dascalu%22">Mihai Dascalu</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-4815-9227">0000-0002-4815-9227</externalLink>)<br /><searchLink fieldCode="AR" term="%22Danielle+S%2E+McNamara%22">Danielle S. McNamara</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Artificial+Intelligence+in+Education%22"><i>International Journal of Artificial Intelligence in Education</i></searchLink>. 2024 34(4):1460-1481.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 22
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305A180261
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring%22">Scoring</searchLink><br /><searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Essays%22">Essays</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22English%22">English</searchLink><br /><searchLink fieldCode="DE" term="%22Portuguese%22">Portuguese</searchLink><br /><searchLink fieldCode="DE" term="%22French%22">French</searchLink><br /><searchLink fieldCode="DE" term="%22Algorithms%22">Algorithms</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Interfaces%22">Computer Interfaces</searchLink><br /><searchLink fieldCode="DE" term="%22Prediction%22">Prediction</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s40593-024-00402-4
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1560-4292<br />1560-4306
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Automated Essay Scoring (AES) is a well-studied problem in Natural Language Processing applied in education. Solutions vary from handcrafted linguistic features to large Transformer-based models, implying a significant effort in feature extraction and model implementation. We introduce a novel Automated Machine Learning (AutoML) pipeline integrated into the ReaderBench platform designed to simplify the process of training AES models by automating both feature extraction and architecture tuning for any multilingual dataset uploaded by the user. The dataset must contain a list of texts, each with potentially multiple annotations, either scores or labels. The platform includes traditional ML models relying on linguistic features and a hybrid approach combining Transformer-based architectures with the previous features. Our method was evaluated on three publicly available datasets in three different languages (English, Portuguese, and French) and compared with the best currently published results on these datasets. Our automated approach achieved comparable results to state-of-the-art models on two datasets, while it obtained the best performance on the third corpus in Portuguese.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1453656
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1453656
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s40593-024-00402-4
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 22
        StartPage: 1460
    Subjects:
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Scoring
        Type: general
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Essays
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: English
        Type: general
      – SubjectFull: Portuguese
        Type: general
      – SubjectFull: French
        Type: general
      – SubjectFull: Algorithms
        Type: general
      – SubjectFull: Computer Interfaces
        Type: general
      – SubjectFull: Prediction
        Type: general
    Titles:
      – TitleFull: Automated Pipeline for Multi-Lingual Automated Essay Scoring with ReaderBench
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Stefan Ruseti
      – PersonEntity:
          Name:
            NameFull: Ionut Paraschiv
      – PersonEntity:
          Name:
            NameFull: Mihai Dascalu
      – PersonEntity:
          Name:
            NameFull: Danielle S. McNamara
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1560-4292
            – Type: issn-electronic
              Value: 1560-4306
          Numbering:
            – Type: volume
              Value: 34
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: International Journal of Artificial Intelligence in Education
              Type: main
ResultId 1