Interrater Reliability.
Saved in:
| Title: | Interrater Reliability. |
|---|---|
| Authors: | Grayson, Kent, Rust, Roland |
| Source: | Journal of Consumer Psychology (Taylor & Francis Ltd). 2001, Vol. 10 Issue 1/2, p71-73. 3p. 1 Chart. |
| Subjects: | Content analysis, Consumer research, Reliability (Personality trait), Communication methodology, Statistical correlation, Least squares |
| Abstract: | This article assesses interrater reliability in content analysis. Or, stated in a slightly different manner from another researcher, there are several tests that give indexes of rater agreement for nominal data and some other tests or coefficients that give indexes of interrater reliability for metric scale data. For the data based on metric scales, the author have established rater reliability using the intraclass correlation coefficient, but he also want to look at interrater agreement. Richard H. Kolbe and Melissa S. Burnett offered a nice-and pretty damning-critique of the quality of content analysis in consumer research. They highlighted a number of criticisms, one of which is this concern about percentage agreement as a basis for judging the quality of content analysis. The basic concern is that percentages do not take into account the likelihood of chance agreement between raters. Chance is likely to inflate agreement percentages in all cases, but especially with two coders, and low degrees of freedom on each coding choice. That is, if Coder A and Coder B have to decide yes-no whether a coding unit has property X, then mere chance will have them agreeing at least 50% of the time. |
| Database: | Psychology and Behavioral Sciences Collection |
|
Full text is not displayed to guests.
Login for full access.
|
|
Be the first to leave a comment!
