Bibliographic Details
| Title: |
A method for measuring interrater agreement on checklists. |
| Authors: |
Sinacore, James M., Connell, Karen J., Olthoff, Allan J., Friedman, Michael H., Gecht, Maureen R., Sinacore, J M1 (AUTHOR), Connell, K J (AUTHOR), Olthoff, A J (AUTHOR), Friedman, M H (AUTHOR), Gecht, M R (AUTHOR) |
| Source: |
Evaluation & the Health Professions. Jun99, Vol. 22 Issue 2, p221-234. 14p. 1 Chart, 1 Graph. |
| Subject Terms: |
*Evaluation of medical education, *Medical students, Rating |
| Abstract: |
A method for measuring interrater agreement on checklists is presented. This technique does not assign individual scores to raters, but computes a single agreement score from the concordance of their check mark configurations. An overall coefficient of agreement, called phi, is derived. The agreement coefficient that is expected by chance and the statistical significance of phi are determined by statistical simulation. Despite the dichotomous nature of the checklist agreement (raters either agree or disagree on items), we show that the binomial distribution does not provide a means for testing the statistical significance of phi. A medical education study is used to illustrate the phi methodology. [ABSTRACT FROM AUTHOR] |
|
Copyright of Evaluation & the Health Professions is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) |
| Database: |
Education Research Complete |
