View in EDS PDF Full Text

Building a reusable test collection for question answering.

Saved in:
Bibliographic Details
Title: Building a reusable test collection for question answering.
Authors: Lin, Jimmy1 jimmylin@umd.edu, Katz, Boris2 boris@csail.mit.edu
Source: Journal of the American Society for Information Science & Technology. May2006, Vol. 57 Issue 7, p851-861. 11p. 1 Diagram, 2 Charts, 3 Graphs.
Subjects: Information storage & retrieval systems, Question answering systems, Information retrieval, Interactive computer systems, Information resources management, Human-computer interaction, Information science, Online data processing, Information theory
Abstract: In contrast to traditional information retrieval systems, which return ranked lists of documents that users must manually browse through, a question answering system attempts to directly answer natural language questions posed by the user. Although such systems possess language-processing capabilities, they still rely on traditional document retrieval techniques to generate an initial candidate set of documents. In this article, the authors argue that document retrieval for question answering represents a task different from retrieving documents in response to more general retrospective information needs. Thus, to guide future system development, specialized question answering test collections must be constructed. They show that the current evaluation resources have major shortcomings; to remedy the situation, they have manually created a small, reusable question answering test collection for research purposes. In this article they describe their methodology for building this test collection and discuss issues they encountered regarding the notion of “answer correctness.” [ABSTRACT FROM AUTHOR]
Copyright of Journal of the American Society for Information Science & Technology is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 20507286
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Building a reusable test collection for question answering.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Lin%2C+Jimmy%22">Lin, Jimmy</searchLink><relatesTo>1</relatesTo><i> jimmylin@umd.edu</i><br /><searchLink fieldCode="AR" term="%22Katz%2C+Boris%22">Katz, Boris</searchLink><relatesTo>2</relatesTo><i> boris@csail.mit.edu</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Journal+of+the+American+Society+for+Information+Science+%26+Technology%22">Journal of the American Society for Information Science & Technology</searchLink>. May2006, Vol. 57 Issue 7, p851-861. 11p. 1 Diagram, 2 Charts, 3 Graphs.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Information+storage+%26+retrieval+systems%22">Information storage & retrieval systems</searchLink><br /><searchLink fieldCode="DE" term="%22Question+answering+systems%22">Question answering systems</searchLink><br /><searchLink fieldCode="DE" term="%22Information+retrieval%22">Information retrieval</searchLink><br /><searchLink fieldCode="DE" term="%22Interactive+computer+systems%22">Interactive computer systems</searchLink><br /><searchLink fieldCode="DE" term="%22Information+resources+management%22">Information resources management</searchLink><br /><searchLink fieldCode="DE" term="%22Human-computer+interaction%22">Human-computer interaction</searchLink><br /><searchLink fieldCode="DE" term="%22Information+science%22">Information science</searchLink><br /><searchLink fieldCode="DE" term="%22Online+data+processing%22">Online data processing</searchLink><br /><searchLink fieldCode="DE" term="%22Information+theory%22">Information theory</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In contrast to traditional information retrieval systems, which return ranked lists of documents that users must manually browse through, a question answering system attempts to directly answer natural language questions posed by the user. Although such systems possess language-processing capabilities, they still rely on traditional document retrieval techniques to generate an initial candidate set of documents. In this article, the authors argue that document retrieval for question answering represents a task different from retrieving documents in response to more general retrospective information needs. Thus, to guide future system development, specialized question answering test collections must be constructed. They show that the current evaluation resources have major shortcomings; to remedy the situation, they have manually created a small, reusable question answering test collection for research purposes. In this article they describe their methodology for building this test collection and discuss issues they encountered regarding the notion of “answer correctness.” [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Journal of the American Society for Information Science & Technology is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=20507286
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/asi.20348
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
        StartPage: 851
    Subjects:
      – SubjectFull: Information storage & retrieval systems
        Type: general
      – SubjectFull: Question answering systems
        Type: general
      – SubjectFull: Information retrieval
        Type: general
      – SubjectFull: Interactive computer systems
        Type: general
      – SubjectFull: Information resources management
        Type: general
      – SubjectFull: Human-computer interaction
        Type: general
      – SubjectFull: Information science
        Type: general
      – SubjectFull: Online data processing
        Type: general
      – SubjectFull: Information theory
        Type: general
    Titles:
      – TitleFull: Building a reusable test collection for question answering.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Lin, Jimmy
      – PersonEntity:
          Name:
            NameFull: Katz, Boris
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 05
              Text: May2006
              Type: published
              Y: 2006
          Identifiers:
            – Type: issn-print
              Value: 15322882
          Numbering:
            – Type: volume
              Value: 57
            – Type: issue
              Value: 7
          Titles:
            – TitleFull: Journal of the American Society for Information Science & Technology
              Type: main
ResultId 1