View in EDS PDF Full Text

Identifying top news using crowdsourcing.

Saved in:
Bibliographic Details
Title: Identifying top news using crowdsourcing.
Authors: McCreadie, Richard1 richardm@dcs.gla.ac.uk, Macdonald, Craig1 craig.macdonald@glasgow.ac.uk, Ounis, Iadh1 iadh.ounis@glasgow.ac.uk
Source: Information Retrieval Journal. Apr2013, Vol. 16 Issue 2, p179-209. 31p.
Subjects: Crowdsourcing, Blogs, Text Retrieval Conference, Information retrieval, Documentation
Abstract: The influential Text REtrieval Conference (TREC) retrieval conference has always relied upon specialist assessors or occasionally participating groups to create relevance judgements for the tracks that it runs. Recently however, crowdsourcing has been championed as a cheap, fast and effective alternative to traditional TREC-like assessments. In 2010, TREC tracks experimented with crowdsourcing for the very first time. In this paper, we report our successful experience in creating relevance assessments for the TREC Blog track 2010 top news stories task using crowdsourcing. In particular, we crowdsourced both real-time newsworthiness assessments for news stories as well as traditional relevance assessments for blog posts. We conclude that crowdsourcing not only appears to be a feasible, but also cheap and fast means to generate relevance assessments. Furthermore, we detail our experiences running the crowdsourced evaluation of the TREC Blog track, discuss the lessons learned, and provide best practices. [ABSTRACT FROM AUTHOR]
Copyright of Information Retrieval Journal is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 86468759
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Identifying top news using crowdsourcing.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22McCreadie%2C+Richard%22">McCreadie, Richard</searchLink><relatesTo>1</relatesTo><i> richardm@dcs.gla.ac.uk</i><br /><searchLink fieldCode="AR" term="%22Macdonald%2C+Craig%22">Macdonald, Craig</searchLink><relatesTo>1</relatesTo><i> craig.macdonald@glasgow.ac.uk</i><br /><searchLink fieldCode="AR" term="%22Ounis%2C+Iadh%22">Ounis, Iadh</searchLink><relatesTo>1</relatesTo><i> iadh.ounis@glasgow.ac.uk</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Information+Retrieval+Journal%22">Information Retrieval Journal</searchLink>. Apr2013, Vol. 16 Issue 2, p179-209. 31p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Crowdsourcing%22">Crowdsourcing</searchLink><br /><searchLink fieldCode="DE" term="%22Blogs%22">Blogs</searchLink><br /><searchLink fieldCode="DE" term="%22Text+Retrieval+Conference%22">Text Retrieval Conference</searchLink><br /><searchLink fieldCode="DE" term="%22Information+retrieval%22">Information retrieval</searchLink><br /><searchLink fieldCode="DE" term="%22Documentation%22">Documentation</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The influential Text REtrieval Conference (TREC) retrieval conference has always relied upon specialist assessors or occasionally participating groups to create relevance judgements for the tracks that it runs. Recently however, crowdsourcing has been championed as a cheap, fast and effective alternative to traditional TREC-like assessments. In 2010, TREC tracks experimented with crowdsourcing for the very first time. In this paper, we report our successful experience in creating relevance assessments for the TREC Blog track 2010 top news stories task using crowdsourcing. In particular, we crowdsourced both real-time newsworthiness assessments for news stories as well as traditional relevance assessments for blog posts. We conclude that crowdsourcing not only appears to be a feasible, but also cheap and fast means to generate relevance assessments. Furthermore, we detail our experiences running the crowdsourced evaluation of the TREC Blog track, discuss the lessons learned, and provide best practices. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Information Retrieval Journal is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=86468759
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10791-012-9186-z
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 31
        StartPage: 179
    Subjects:
      – SubjectFull: Crowdsourcing
        Type: general
      – SubjectFull: Blogs
        Type: general
      – SubjectFull: Text Retrieval Conference
        Type: general
      – SubjectFull: Information retrieval
        Type: general
      – SubjectFull: Documentation
        Type: general
    Titles:
      – TitleFull: Identifying top news using crowdsourcing.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: McCreadie, Richard
      – PersonEntity:
          Name:
            NameFull: Macdonald, Craig
      – PersonEntity:
          Name:
            NameFull: Ounis, Iadh
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 04
              Text: Apr2013
              Type: published
              Y: 2013
          Identifiers:
            – Type: issn-print
              Value: 13864564
          Numbering:
            – Type: volume
              Value: 16
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Information Retrieval Journal
              Type: main
ResultId 1