View in EDS HTML Full Text PDF Full Text

Interrater Reliability.

Saved in:
Bibliographic Details
Title: Interrater Reliability.
Authors: Grayson, Kent, Rust, Roland
Source: Journal of Consumer Psychology (Taylor & Francis Ltd). 2001, Vol. 10 Issue 1/2, p71-73. 3p. 1 Chart.
Subjects: Content analysis, Consumer research, Reliability (Personality trait), Communication methodology, Statistical correlation, Least squares
Abstract: This article assesses interrater reliability in content analysis. Or, stated in a slightly different manner from another researcher, there are several tests that give indexes of rater agreement for nominal data and some other tests or coefficients that give indexes of interrater reliability for metric scale data. For the data based on metric scales, the author have established rater reliability using the intraclass correlation coefficient, but he also want to look at interrater agreement. Richard H. Kolbe and Melissa S. Burnett offered a nice-and pretty damning-critique of the quality of content analysis in consumer research. They highlighted a number of criticisms, one of which is this concern about percentage agreement as a basis for judging the quality of content analysis. The basic concern is that percentages do not take into account the likelihood of chance agreement between raters. Chance is likely to inflate agreement percentages in all cases, but especially with two coders, and low degrees of freedom on each coding choice. That is, if Coder A and Coder B have to decide yes-no whether a coding unit has property X, then mere chance will have them agreeing at least 50% of the time.
Database: Psychology and Behavioral Sciences Collection
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: pbh
DbLabel: Psychology and Behavioral Sciences Collection
An: 4895273
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Interrater Reliability.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Grayson%2C+Kent%22">Grayson, Kent</searchLink><br /><searchLink fieldCode="AR" term="%22Rust%2C+Roland%22">Rust, Roland</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Journal+of+Consumer+Psychology+%28Taylor+%26+Francis+Ltd%29%22">Journal of Consumer Psychology (Taylor & Francis Ltd)</searchLink>. 2001, Vol. 10 Issue 1/2, p71-73. 3p. 1 Chart.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Content+analysis%22">Content analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Consumer+research%22">Consumer research</searchLink><br /><searchLink fieldCode="DE" term="%22Reliability+%28Personality+trait%29%22">Reliability (Personality trait)</searchLink><br /><searchLink fieldCode="DE" term="%22Communication+methodology%22">Communication methodology</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+correlation%22">Statistical correlation</searchLink><br /><searchLink fieldCode="DE" term="%22Least+squares%22">Least squares</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This article assesses interrater reliability in content analysis. Or, stated in a slightly different manner from another researcher, there are several tests that give indexes of rater agreement for nominal data and some other tests or coefficients that give indexes of interrater reliability for metric scale data. For the data based on metric scales, the author have established rater reliability using the intraclass correlation coefficient, but he also want to look at interrater agreement. Richard H. Kolbe and Melissa S. Burnett offered a nice-and pretty damning-critique of the quality of content analysis in consumer research. They highlighted a number of criticisms, one of which is this concern about percentage agreement as a basis for judging the quality of content analysis. The basic concern is that percentages do not take into account the likelihood of chance agreement between raters. Chance is likely to inflate agreement percentages in all cases, but especially with two coders, and low degrees of freedom on each coding choice. That is, if Coder A and Coder B have to decide yes-no whether a coding unit has property X, then mere chance will have them agreeing at least 50% of the time.
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=pbh&AN=4895273
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1207/S15327663JCP1001&2_06
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 3
        StartPage: 71
    Subjects:
      – SubjectFull: Content analysis
        Type: general
      – SubjectFull: Consumer research
        Type: general
      – SubjectFull: Reliability (Personality trait)
        Type: general
      – SubjectFull: Communication methodology
        Type: general
      – SubjectFull: Statistical correlation
        Type: general
      – SubjectFull: Least squares
        Type: general
    Titles:
      – TitleFull: Interrater Reliability.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Grayson, Kent
      – PersonEntity:
          Name:
            NameFull: Rust, Roland
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Text: 2001
              Type: published
              Y: 2001
          Identifiers:
            – Type: issn-print
              Value: 10577408
          Numbering:
            – Type: volume
              Value: 10
            – Type: issue
              Value: 1/2
          Titles:
            – TitleFull: Journal of Consumer Psychology (Taylor & Francis Ltd)
              Type: main
ResultId 1