View in EDS

Paraphrase Generation and Supervised Learning for Improved Automatic Short Answer Grading

Saved in:
Bibliographic Details
Title: Paraphrase Generation and Supervised Learning for Improved Automatic Short Answer Grading
Language: English
Authors: Leila Ouahrani (ORCID 0000-0002-4415-5915), Djamal Bennouar (ORCID 0000-0002-0105-4604)
Source: International Journal of Artificial Intelligence in Education. 2024 34(4):1627-1670.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 44
Publication Date: 2024
Document Type: Journal Articles
Reports - Descriptive
Descriptors: Grading, Automation, Answer Keys, Tests, Sentence Structure, Arabic, English, Learning Management Systems, Technology Uses in Education
DOI: 10.1007/s40593-023-00391-w
ISSN: 1560-4292
1560-4306
Abstract: We consider the reference-based approach for Automatic Short Answer Grading (ASAG) that involves scoring a textual constructed student answer comparing to a teacher-provided reference answer. The reference answer does not cover the variety of student answers as it contains only specific examples of correct answers. Considering other language variants of the reference answer can handle variability in student responses and improve scoring accuracy. Alternative reference answers may be possible, but manually creating them is expensive and time-consuming. In this paper, we consider two issues: First, we need to automatically generate various reference answers that can handle the diversity of student answers. Second, we should provide an accurate grading model that improves sentence similarity computation using multiple reference answers. Therefore, our proposed approach to solve both problems highlights two components. First, we provide a sequence-to-sequence deep learning model that targets generating plausible paraphrased reference answers conditioned on the provided reference answer. Secondly, we propose a supervised grading model based on sentence embedding features. The grading model enriches features to improve accuracy considering multiple reference answers. Experiments are conducted both in Arabic and English. They show that the paraphrase generator produces accurate paraphrases. Using multiple reference answers, the proposed grading model achieves a Root Mean Square Error of 0,6955, a Pearson correlation of 88,92% for the Arabic dataset, an RMSE of 0,7790, and a Pearson correlation of 73,50% for the English dataset. While fine-tuning pre-trained transformers on the English dataset provided state-of-the-art performance (RMSE: 0.7620), our approach yields comparable results. Simple to construct, load, and embed into the Learning Management System question engine with low computational complexity, the proposed approach can be easily integrated into the Learning Management System to support the assessment of short answers.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1453602
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1453602
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Paraphrase Generation and Supervised Learning for Improved Automatic Short Answer Grading
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Leila+Ouahrani%22">Leila Ouahrani</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-4415-5915">0000-0002-4415-5915</externalLink>)<br /><searchLink fieldCode="AR" term="%22Djamal+Bennouar%22">Djamal Bennouar</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-0105-4604">0000-0002-0105-4604</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Journal+of+Artificial+Intelligence+in+Education%22"><i>International Journal of Artificial Intelligence in Education</i></searchLink>. 2024 34(4):1627-1670.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 44
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Descriptive
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Grading%22">Grading</searchLink><br /><searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Answer+Keys%22">Answer Keys</searchLink><br /><searchLink fieldCode="DE" term="%22Tests%22">Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Sentence+Structure%22">Sentence Structure</searchLink><br /><searchLink fieldCode="DE" term="%22Arabic%22">Arabic</searchLink><br /><searchLink fieldCode="DE" term="%22English%22">English</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Management+Systems%22">Learning Management Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s40593-023-00391-w
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1560-4292<br />1560-4306
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: We consider the reference-based approach for Automatic Short Answer Grading (ASAG) that involves scoring a textual constructed student answer comparing to a teacher-provided reference answer. The reference answer does not cover the variety of student answers as it contains only specific examples of correct answers. Considering other language variants of the reference answer can handle variability in student responses and improve scoring accuracy. Alternative reference answers may be possible, but manually creating them is expensive and time-consuming. In this paper, we consider two issues: First, we need to automatically generate various reference answers that can handle the diversity of student answers. Second, we should provide an accurate grading model that improves sentence similarity computation using multiple reference answers. Therefore, our proposed approach to solve both problems highlights two components. First, we provide a sequence-to-sequence deep learning model that targets generating plausible paraphrased reference answers conditioned on the provided reference answer. Secondly, we propose a supervised grading model based on sentence embedding features. The grading model enriches features to improve accuracy considering multiple reference answers. Experiments are conducted both in Arabic and English. They show that the paraphrase generator produces accurate paraphrases. Using multiple reference answers, the proposed grading model achieves a Root Mean Square Error of 0,6955, a Pearson correlation of 88,92% for the Arabic dataset, an RMSE of 0,7790, and a Pearson correlation of 73,50% for the English dataset. While fine-tuning pre-trained transformers on the English dataset provided state-of-the-art performance (RMSE: 0.7620), our approach yields comparable results. Simple to construct, load, and embed into the Learning Management System question engine with low computational complexity, the proposed approach can be easily integrated into the Learning Management System to support the assessment of short answers.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1453602
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1453602
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s40593-023-00391-w
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 44
        StartPage: 1627
    Subjects:
      – SubjectFull: Grading
        Type: general
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Answer Keys
        Type: general
      – SubjectFull: Tests
        Type: general
      – SubjectFull: Sentence Structure
        Type: general
      – SubjectFull: Arabic
        Type: general
      – SubjectFull: English
        Type: general
      – SubjectFull: Learning Management Systems
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
    Titles:
      – TitleFull: Paraphrase Generation and Supervised Learning for Improved Automatic Short Answer Grading
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Leila Ouahrani
      – PersonEntity:
          Name:
            NameFull: Djamal Bennouar
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1560-4292
            – Type: issn-electronic
              Value: 1560-4306
          Numbering:
            – Type: volume
              Value: 34
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: International Journal of Artificial Intelligence in Education
              Type: main
ResultId 1