BLANC: Implementing the Rand index for coreference evaluation.
Saved in:
| Title: | BLANC: Implementing the Rand index for coreference evaluation. |
|---|---|
| Authors: | RECASENS, M.1, HOVY, E.2 |
| Source: | Natural Language Engineering. Oct2011, Vol. 17 Issue 4, p485-510. 26p. |
| Subjects: | Computational linguistics, Cluster analysis (Statistics), Noun phrases, Natural language processing, Decision making, Algorithms, Errors |
| Abstract: | This paper addresses the current state of coreference resolution evaluation, in which different measures (notably, MUC, B3, CEAF, and ACE-value) are applied in different studies. None of them is fully adequate, and their measures are not commensurate. We enumerate the desiderata for a coreference scoring measure, discuss the strong and weak points of the existing measures, and propose the BiLateral Assessment of Noun-Phrase Coreference, a variation of the Rand index created to suit the coreference task. The BiLateral Assessment of Noun-Phrase Coreference rewards both coreference and non-coreference links by averaging the F-scores of the two types, does not ignore singletons – the main problem with the MUC score – and does not inflate the score in their presence – a problem with the B3 and CEAF scores. In addition, its fine granularity is consistent over the whole range of scores and affords better discrimination between systems. [ABSTRACT FROM PUBLISHER] |
| Copyright of Natural Language Engineering is the property of Cambridge University Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
Be the first to leave a comment!
