View in EDS

Towards diverse and contextually anchored paraphrase modeling: A dataset and baselines for Finnish.

Saved in:
Bibliographic Details
Title: Towards diverse and contextually anchored paraphrase modeling: A dataset and baselines for Finnish.
Authors: Kanerva, Jenna1 (AUTHOR) jmnybl@utu.fi, Ginter, Filip1 (AUTHOR), Chang, Li-Hsin1 (AUTHOR), Rastas, Iiro1 (AUTHOR), Skantsi, Valtteri1 (AUTHOR), Kilpeläinen, Jemina1 (AUTHOR), Kupari, Hanna-Mari1 (AUTHOR), Piirto, Aurora1 (AUTHOR), Saarni, Jenna1 (AUTHOR), Sevón, Maija1 (AUTHOR), Tarkka, Otto1 (AUTHOR)
Source: Natural Language Engineering. Mar2024, Vol. 30 Issue 2, p319-353. 35p.
Subjects: Paraphrase, Natural languages, Models & modelmaking
Abstract: In this paper, we study natural language paraphrasing from both corpus creation and modeling points of view. We focus in particular on the methodology that allows the extraction of challenging examples of paraphrase pairs in their natural textual context, leading to a dataset potentially more suitable for evaluating the models' ability to represent meaning, especially in document context, when compared with those gathered using various sentence-level heuristics. To this end, we introduce the Turku Paraphrase Corpus, the first large-scale, fully manually annotated corpus of paraphrases in Finnish. The corpus contains 104,645 manually labeled paraphrase pairs, of which 98% are verified to be true paraphrases, either universally or within their present context. In order to control the diversity of the paraphrase pairs and avoid certain biases easily introduced in automatic candidate extraction, the paraphrases are manually collected from different paraphrase-rich text sources. This allows us to create a challenging dataset including longer and more lexically diverse paraphrases than can be expected from those collected through heuristics. In addition to quality, this also allows us to keep the original document context for each pair, making it possible to study paraphrasing in context. To our knowledge, this is the first paraphrase corpus which provides the original document context for the annotated pairs. We also study several paraphrase models trained and evaluated on the new data. Our initial paraphrase classification experiments indicate a challenging nature of the dataset when classifying using the detailed labeling scheme used in the corpus annotation, the accuracy substantially lacking behind human performance. However, when evaluating the models on a large scale paraphrase retrieval task on almost 400M candidate sentences, the results are highly encouraging, 29–53% of the pairs being ranked in the top 10 depending on the paraphrase type. The Turku Paraphrase Corpus is available at github.com/TurkuNLP/Turku-paraphrase-corpus as well as through the popular HuggingFace datasets under the CC-BY-SA license. [ABSTRACT FROM AUTHOR]
Copyright of Natural Language Engineering is the property of Cambridge University Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 176361461
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Towards diverse and contextually anchored paraphrase modeling: A dataset and baselines for Finnish.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Kanerva%2C+Jenna%22">Kanerva, Jenna</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> jmnybl@utu.fi</i><br /><searchLink fieldCode="AR" term="%22Ginter%2C+Filip%22">Ginter, Filip</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Chang%2C+Li-Hsin%22">Chang, Li-Hsin</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Rastas%2C+Iiro%22">Rastas, Iiro</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Skantsi%2C+Valtteri%22">Skantsi, Valtteri</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Kilpeläinen%2C+Jemina%22">Kilpeläinen, Jemina</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Kupari%2C+Hanna-Mari%22">Kupari, Hanna-Mari</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Piirto%2C+Aurora%22">Piirto, Aurora</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Saarni%2C+Jenna%22">Saarni, Jenna</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Sevón%2C+Maija%22">Sevón, Maija</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Tarkka%2C+Otto%22">Tarkka, Otto</searchLink><relatesTo>1</relatesTo> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Natural+Language+Engineering%22">Natural Language Engineering</searchLink>. Mar2024, Vol. 30 Issue 2, p319-353. 35p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Paraphrase%22">Paraphrase</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+languages%22">Natural languages</searchLink><br /><searchLink fieldCode="DE" term="%22Models+%26+modelmaking%22">Models & modelmaking</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In this paper, we study natural language paraphrasing from both corpus creation and modeling points of view. We focus in particular on the methodology that allows the extraction of challenging examples of paraphrase pairs in their natural textual context, leading to a dataset potentially more suitable for evaluating the models' ability to represent meaning, especially in document context, when compared with those gathered using various sentence-level heuristics. To this end, we introduce the Turku Paraphrase Corpus, the first large-scale, fully manually annotated corpus of paraphrases in Finnish. The corpus contains 104,645 manually labeled paraphrase pairs, of which 98% are verified to be true paraphrases, either universally or within their present context. In order to control the diversity of the paraphrase pairs and avoid certain biases easily introduced in automatic candidate extraction, the paraphrases are manually collected from different paraphrase-rich text sources. This allows us to create a challenging dataset including longer and more lexically diverse paraphrases than can be expected from those collected through heuristics. In addition to quality, this also allows us to keep the original document context for each pair, making it possible to study paraphrasing in context. To our knowledge, this is the first paraphrase corpus which provides the original document context for the annotated pairs. We also study several paraphrase models trained and evaluated on the new data. Our initial paraphrase classification experiments indicate a challenging nature of the dataset when classifying using the detailed labeling scheme used in the corpus annotation, the accuracy substantially lacking behind human performance. However, when evaluating the models on a large scale paraphrase retrieval task on almost 400M candidate sentences, the results are highly encouraging, 29–53% of the pairs being ranked in the top 10 depending on the paraphrase type. The Turku Paraphrase Corpus is available at github.com/TurkuNLP/Turku-paraphrase-corpus as well as through the popular HuggingFace datasets under the CC-BY-SA license. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Natural Language Engineering is the property of Cambridge University Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=176361461
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1017/S1351324923000086
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 35
        StartPage: 319
    Subjects:
      – SubjectFull: Paraphrase
        Type: general
      – SubjectFull: Natural languages
        Type: general
      – SubjectFull: Models & modelmaking
        Type: general
    Titles:
      – TitleFull: Towards diverse and contextually anchored paraphrase modeling: A dataset and baselines for Finnish.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Kanerva, Jenna
      – PersonEntity:
          Name:
            NameFull: Ginter, Filip
      – PersonEntity:
          Name:
            NameFull: Chang, Li-Hsin
      – PersonEntity:
          Name:
            NameFull: Rastas, Iiro
      – PersonEntity:
          Name:
            NameFull: Skantsi, Valtteri
      – PersonEntity:
          Name:
            NameFull: Kilpeläinen, Jemina
      – PersonEntity:
          Name:
            NameFull: Kupari, Hanna-Mari
      – PersonEntity:
          Name:
            NameFull: Piirto, Aurora
      – PersonEntity:
          Name:
            NameFull: Saarni, Jenna
      – PersonEntity:
          Name:
            NameFull: Sevón, Maija
      – PersonEntity:
          Name:
            NameFull: Tarkka, Otto
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Text: Mar2024
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 13513249
          Numbering:
            – Type: volume
              Value: 30
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Natural Language Engineering
              Type: main
ResultId 1