View in EDS

A method for measuring interrater agreement on checklists.

Saved in:
Bibliographic Details
Title: A method for measuring interrater agreement on checklists.
Authors: Sinacore, James M., Connell, Karen J., Olthoff, Allan J., Friedman, Michael H., Gecht, Maureen R., Sinacore, J M1 (AUTHOR), Connell, K J (AUTHOR), Olthoff, A J (AUTHOR), Friedman, M H (AUTHOR), Gecht, M R (AUTHOR)
Source: Evaluation & the Health Professions. Jun99, Vol. 22 Issue 2, p221-234. 14p. 1 Chart, 1 Graph.
Subject Terms: *Evaluation of medical education, *Medical students, Rating
Abstract: A method for measuring interrater agreement on checklists is presented. This technique does not assign individual scores to raters, but computes a single agreement score from the concordance of their check mark configurations. An overall coefficient of agreement, called phi, is derived. The agreement coefficient that is expected by chance and the statistical significance of phi are determined by statistical simulation. Despite the dichotomous nature of the checklist agreement (raters either agree or disagree on items), we show that the binomial distribution does not provide a means for testing the statistical significance of phi. A medical education study is used to illustrate the phi methodology. [ABSTRACT FROM AUTHOR]
Copyright of Evaluation & the Health Professions is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
FullText Text:
  Availability: 0
Header DbId: ehh
DbLabel: Education Research Complete
An: 1938259
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A method for measuring interrater agreement on checklists.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Sinacore%2C+James+M%2E%22">Sinacore, James M.</searchLink><br /><searchLink fieldCode="AR" term="%22Connell%2C+Karen+J%2E%22">Connell, Karen J.</searchLink><br /><searchLink fieldCode="AR" term="%22Olthoff%2C+Allan+J%2E%22">Olthoff, Allan J.</searchLink><br /><searchLink fieldCode="AR" term="%22Friedman%2C+Michael+H%2E%22">Friedman, Michael H.</searchLink><br /><searchLink fieldCode="AR" term="%22Gecht%2C+Maureen+R%2E%22">Gecht, Maureen R.</searchLink><br /><searchLink fieldCode="AR" term="%22Sinacore%2C+J+M%22">Sinacore, J M</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Connell%2C+K+J%22">Connell, K J</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Olthoff%2C+A+J%22">Olthoff, A J</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Friedman%2C+M+H%22">Friedman, M H</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Gecht%2C+M+R%22">Gecht, M R</searchLink> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Evaluation+%26+the+Health+Professions%22">Evaluation & the Health Professions</searchLink>. Jun99, Vol. 22 Issue 2, p221-234. 14p. 1 Chart, 1 Graph.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Evaluation+of+medical+education%22">Evaluation of medical education</searchLink><br />*<searchLink fieldCode="DE" term="%22Medical+students%22">Medical students</searchLink><br /><searchLink fieldCode="DE" term="%22Rating%22">Rating</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: A method for measuring interrater agreement on checklists is presented. This technique does not assign individual scores to raters, but computes a single agreement score from the concordance of their check mark configurations. An overall coefficient of agreement, called phi, is derived. The agreement coefficient that is expected by chance and the statistical significance of phi are determined by statistical simulation. Despite the dichotomous nature of the checklist agreement (raters either agree or disagree on items), we show that the binomial distribution does not provide a means for testing the statistical significance of phi. A medical education study is used to illustrate the phi methodology. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Evaluation & the Health Professions is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=1938259
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/01632789922034284
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 14
        StartPage: 221
    Subjects:
      – SubjectFull: Evaluation of medical education
        Type: general
      – SubjectFull: Medical students
        Type: general
      – SubjectFull: Rating
        Type: general
    Titles:
      – TitleFull: A method for measuring interrater agreement on checklists.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Sinacore, James M.
      – PersonEntity:
          Name:
            NameFull: Connell, Karen J.
      – PersonEntity:
          Name:
            NameFull: Olthoff, Allan J.
      – PersonEntity:
          Name:
            NameFull: Friedman, Michael H.
      – PersonEntity:
          Name:
            NameFull: Gecht, Maureen R.
      – PersonEntity:
          Name:
            NameFull: Sinacore, J M
      – PersonEntity:
          Name:
            NameFull: Connell, K J
      – PersonEntity:
          Name:
            NameFull: Olthoff, A J
      – PersonEntity:
          Name:
            NameFull: Friedman, M H
      – PersonEntity:
          Name:
            NameFull: Gecht, M R
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 06
              Text: Jun99
              Type: published
              Y: 1999
          Identifiers:
            – Type: issn-print
              Value: 01632787
          Numbering:
            – Type: volume
              Value: 22
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Evaluation & the Health Professions
              Type: main
ResultId 1