View in EDS

Automated Paraphrase Quality Assessment Using Recurrent Neural Networks and Language Models

Saved in:
Bibliographic Details
Title: Automated Paraphrase Quality Assessment Using Recurrent Neural Networks and Language Models
Language: English
Authors: Nicula, Bogdan, Dascalu, Mihai, Newton, Natalie, Orcutt, Ellen, McNamara, Danielle S.
Source: Grantee Submission. 2021.
Peer Reviewed: Y
Page Count: 9
Publication Date: 2021
Sponsoring Agency: Institute of Education Sciences (ED)
Office of Naval Research (ONR) (DOD)
Contract Number: R305A190063
R305A190050
N000141712300
N000141912424
Document Type: Speeches/Meeting Papers
Reports - Research
Education Level: Elementary Education
Descriptors: Phrase Structure, Networks, Semantics, Feedback (Response), Syntax, Computational Linguistics, Language Usage, Models, Teaching Methods, Classification, Artificial Intelligence, Linguistic Input, Intelligent Tutoring Systems, Natural Language Processing, Literacy Education, Elementary School Students
DOI: 10.1007/978-3-030-80421-3_36
Abstract: The ability to automatically assess the quality of paraphrases can be very useful for facilitating literacy skills and providing timely feedback to learners. Our aim is twofold: a) to automatically evaluate the quality of paraphrases across four dimensions: lexical similarity, syntactic similarity, semantic similarity and paraphrase quality, and b) to assess how well models trained for this task generalize. The task is modeled as a classification problem and three different methods are explored: (a) manual feature extraction combined with an Extra Trees model, (b) GloVe embeddings and a Siamese neural network, and (c) using a pre-trained BERT model fine-tuned on our task. Starting from a dataset of 1998 paraphrases from the User Language Paraphrase Corpus (ULPC), we explore how the three models trained on the ULPC dataset generalize when applied on a separate, small paraphrase corpus based on children inputs. The best out-of-the-box generalization performance is obtained by the Extra Trees model with at least 75% average F1-scores for the three similarity dimensions. We also show that the Siamese neural network and BERT models can obtain an improvement of at least 5% after fine-tuning across all dimensions.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2023
Accession Number: ED628430
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED628430
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED628430
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Automated Paraphrase Quality Assessment Using Recurrent Neural Networks and Language Models
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Nicula%2C+Bogdan%22">Nicula, Bogdan</searchLink><br /><searchLink fieldCode="AR" term="%22Dascalu%2C+Mihai%22">Dascalu, Mihai</searchLink><br /><searchLink fieldCode="AR" term="%22Newton%2C+Natalie%22">Newton, Natalie</searchLink><br /><searchLink fieldCode="AR" term="%22Orcutt%2C+Ellen%22">Orcutt, Ellen</searchLink><br /><searchLink fieldCode="AR" term="%22McNamara%2C+Danielle+S%2E%22">McNamara, Danielle S.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Grantee+Submission%22"><i>Grantee Submission</i></searchLink>. 2021.
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 9
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2021
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)<br />Office of Naval Research (ONR) (DOD)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305A190063<br />R305A190050<br />N000141712300<br />N000141912424
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Phrase+Structure%22">Phrase Structure</searchLink><br /><searchLink fieldCode="DE" term="%22Networks%22">Networks</searchLink><br /><searchLink fieldCode="DE" term="%22Semantics%22">Semantics</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Syntax%22">Syntax</searchLink><br /><searchLink fieldCode="DE" term="%22Computational+Linguistics%22">Computational Linguistics</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Usage%22">Language Usage</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Teaching+Methods%22">Teaching Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Classification%22">Classification</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Linguistic+Input%22">Linguistic Input</searchLink><br /><searchLink fieldCode="DE" term="%22Intelligent+Tutoring+Systems%22">Intelligent Tutoring Systems</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Literacy+Education%22">Literacy Education</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/978-3-030-80421-3_36
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The ability to automatically assess the quality of paraphrases can be very useful for facilitating literacy skills and providing timely feedback to learners. Our aim is twofold: a) to automatically evaluate the quality of paraphrases across four dimensions: lexical similarity, syntactic similarity, semantic similarity and paraphrase quality, and b) to assess how well models trained for this task generalize. The task is modeled as a classification problem and three different methods are explored: (a) manual feature extraction combined with an Extra Trees model, (b) GloVe embeddings and a Siamese neural network, and (c) using a pre-trained BERT model fine-tuned on our task. Starting from a dataset of 1998 paraphrases from the User Language Paraphrase Corpus (ULPC), we explore how the three models trained on the ULPC dataset generalize when applied on a separate, small paraphrase corpus based on children inputs. The best out-of-the-box generalization performance is obtained by the Extra Trees model with at least 75% average F1-scores for the three similarity dimensions. We also show that the Siamese neural network and BERT models can obtain an improvement of at least 5% after fine-tuning across all dimensions.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED628430
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED628430
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/978-3-030-80421-3_36
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 9
    Subjects:
      – SubjectFull: Phrase Structure
        Type: general
      – SubjectFull: Networks
        Type: general
      – SubjectFull: Semantics
        Type: general
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Syntax
        Type: general
      – SubjectFull: Computational Linguistics
        Type: general
      – SubjectFull: Language Usage
        Type: general
      – SubjectFull: Models
        Type: general
      – SubjectFull: Teaching Methods
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Linguistic Input
        Type: general
      – SubjectFull: Intelligent Tutoring Systems
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Literacy Education
        Type: general
      – SubjectFull: Elementary School Students
        Type: general
    Titles:
      – TitleFull: Automated Paraphrase Quality Assessment Using Recurrent Neural Networks and Language Models
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Nicula, Bogdan
      – PersonEntity:
          Name:
            NameFull: Dascalu, Mihai
      – PersonEntity:
          Name:
            NameFull: Newton, Natalie
      – PersonEntity:
          Name:
            NameFull: Orcutt, Ellen
      – PersonEntity:
          Name:
            NameFull: McNamara, Danielle S.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 09
              M: 07
              Type: published
              Y: 2021
          Titles:
            – TitleFull: Grantee Submission
              Type: main
ResultId 1