View in EDS

BLANC: Implementing the Rand index for coreference evaluation.

Saved in:
Bibliographic Details
Title: BLANC: Implementing the Rand index for coreference evaluation.
Authors: RECASENS, M.1, HOVY, E.2
Source: Natural Language Engineering. Oct2011, Vol. 17 Issue 4, p485-510. 26p.
Subjects: Computational linguistics, Cluster analysis (Statistics), Noun phrases, Natural language processing, Decision making, Algorithms, Errors
Abstract: This paper addresses the current state of coreference resolution evaluation, in which different measures (notably, MUC, B3, CEAF, and ACE-value) are applied in different studies. None of them is fully adequate, and their measures are not commensurate. We enumerate the desiderata for a coreference scoring measure, discuss the strong and weak points of the existing measures, and propose the BiLateral Assessment of Noun-Phrase Coreference, a variation of the Rand index created to suit the coreference task. The BiLateral Assessment of Noun-Phrase Coreference rewards both coreference and non-coreference links by averaging the F-scores of the two types, does not ignore singletons – the main problem with the MUC score – and does not inflate the score in their presence – a problem with the B3 and CEAF scores. In addition, its fine granularity is consistent over the whole range of scores and affords better discrimination between systems. [ABSTRACT FROM PUBLISHER]
Copyright of Natural Language Engineering is the property of Cambridge University Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 65466144
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: BLANC: Implementing the Rand index for coreference evaluation.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22RECASENS%2C+M%2E%22">RECASENS, M.</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22HOVY%2C+E%2E%22">HOVY, E.</searchLink><relatesTo>2</relatesTo>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Natural+Language+Engineering%22">Natural Language Engineering</searchLink>. Oct2011, Vol. 17 Issue 4, p485-510. 26p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Computational+linguistics%22">Computational linguistics</searchLink><br /><searchLink fieldCode="DE" term="%22Cluster+analysis+%28Statistics%29%22">Cluster analysis (Statistics)</searchLink><br /><searchLink fieldCode="DE" term="%22Noun+phrases%22">Noun phrases</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+language+processing%22">Natural language processing</searchLink><br /><searchLink fieldCode="DE" term="%22Decision+making%22">Decision making</searchLink><br /><searchLink fieldCode="DE" term="%22Algorithms%22">Algorithms</searchLink><br /><searchLink fieldCode="DE" term="%22Errors%22">Errors</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This paper addresses the current state of coreference resolution evaluation, in which different measures (notably, MUC, B3, CEAF, and ACE-value) are applied in different studies. None of them is fully adequate, and their measures are not commensurate. We enumerate the desiderata for a coreference scoring measure, discuss the strong and weak points of the existing measures, and propose the BiLateral Assessment of Noun-Phrase Coreference, a variation of the Rand index created to suit the coreference task. The BiLateral Assessment of Noun-Phrase Coreference rewards both coreference and non-coreference links by averaging the F-scores of the two types, does not ignore singletons – the main problem with the MUC score – and does not inflate the score in their presence – a problem with the B3 and CEAF scores. In addition, its fine granularity is consistent over the whole range of scores and affords better discrimination between systems. [ABSTRACT FROM PUBLISHER]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Natural Language Engineering is the property of Cambridge University Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=65466144
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1017/S135132491000029X
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 26
        StartPage: 485
    Subjects:
      – SubjectFull: Computational linguistics
        Type: general
      – SubjectFull: Cluster analysis (Statistics)
        Type: general
      – SubjectFull: Noun phrases
        Type: general
      – SubjectFull: Natural language processing
        Type: general
      – SubjectFull: Decision making
        Type: general
      – SubjectFull: Algorithms
        Type: general
      – SubjectFull: Errors
        Type: general
    Titles:
      – TitleFull: BLANC: Implementing the Rand index for coreference evaluation.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: RECASENS, M.
      – PersonEntity:
          Name:
            NameFull: HOVY, E.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Text: Oct2011
              Type: published
              Y: 2011
          Identifiers:
            – Type: issn-print
              Value: 13513249
          Numbering:
            – Type: volume
              Value: 17
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Natural Language Engineering
              Type: main
ResultId 1