View in EDS HTML Full Text PDF Full Text

Effects of item and rater characteristics on checklist recording: what should we look for?

Saved in:
Bibliographic Details
Title: Effects of item and rater characteristics on checklist recording: what should we look for?
Authors: Huber, Philippe (AUTHOR), Baroffio, Anne (AUTHOR), Chamot, Eric (AUTHOR), Herrmann, François (AUTHOR), Nendaz, Mathieu R (AUTHOR), Vu, Nu V (AUTHOR)
Source: Medical Education. Aug2005, Vol. 39 Issue 8, p852-858. 7p.
Subjects: Medical students, Performance, Evaluation, Patients, Medical schools, Medical education
Abstract: Examinations based on using standardised patients (SPs) commonly use checklist recordings to evaluate students' clinical performance. This paper examines whether and to what extent item and rater characteristics affect the reliability of history checklist recording in an SP-based assessment. Checklist items were reviewed for the presence or absence of 5 item characteristics and a 2-point versus 3-point scoring scale. Agreement between checklist recordings obtained from SPs and clinician-examiners (CEs) were compared by item characteristics, scoring scale and CEs' level of involvement in the assessment. Based on 3179 pairs of recordings, the overall percentage of agreement between SPs and CEs was 83% (kappa = 0.64). Agreement was significantly higher for items scored on a 2-point than on a 3-point scale, and when the CE was also the author and the trainer of the station. After controlling for other factors, item characteristics were only marginally associated with level of interrater agreement. This study suggests that attention should be paid to specific aspects of checklist development and checklist recording training when an SP or CE is used as recorder. [ABSTRACT FROM AUTHOR]
Copyright of Medical Education is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Psychology and Behavioral Sciences Collection
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: pbh
DbLabel: Psychology and Behavioral Sciences Collection
An: 17691239
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Effects of item and rater characteristics on checklist recording: what should we look for?
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Huber%2C+Philippe%22">Huber, Philippe</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Baroffio%2C+Anne%22">Baroffio, Anne</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Chamot%2C+Eric%22">Chamot, Eric</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Herrmann%2C+François%22">Herrmann, François</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Nendaz%2C+Mathieu+R%22">Nendaz, Mathieu R</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Vu%2C+Nu+V%22">Vu, Nu V</searchLink> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Medical+Education%22">Medical Education</searchLink>. Aug2005, Vol. 39 Issue 8, p852-858. 7p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Medical+students%22">Medical students</searchLink><br /><searchLink fieldCode="DE" term="%22Performance%22">Performance</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation%22">Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Patients%22">Patients</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+schools%22">Medical schools</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+education%22">Medical education</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Examinations based on using standardised patients (SPs) commonly use checklist recordings to evaluate students' clinical performance. This paper examines whether and to what extent item and rater characteristics affect the reliability of history checklist recording in an SP-based assessment. Checklist items were reviewed for the presence or absence of 5 item characteristics and a 2-point versus 3-point scoring scale. Agreement between checklist recordings obtained from SPs and clinician-examiners (CEs) were compared by item characteristics, scoring scale and CEs' level of involvement in the assessment. Based on 3179 pairs of recordings, the overall percentage of agreement between SPs and CEs was 83% (kappa = 0.64). Agreement was significantly higher for items scored on a 2-point than on a 3-point scale, and when the CE was also the author and the trainer of the station. After controlling for other factors, item characteristics were only marginally associated with level of interrater agreement. This study suggests that attention should be paid to specific aspects of checklist development and checklist recording training when an SP or CE is used as recorder. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Medical Education is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=pbh&AN=17691239
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/j.1365-2929.2005.02226.x
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 7
        StartPage: 852
    Subjects:
      – SubjectFull: Medical students
        Type: general
      – SubjectFull: Performance
        Type: general
      – SubjectFull: Evaluation
        Type: general
      – SubjectFull: Patients
        Type: general
      – SubjectFull: Medical schools
        Type: general
      – SubjectFull: Medical education
        Type: general
    Titles:
      – TitleFull: Effects of item and rater characteristics on checklist recording: what should we look for?
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Huber, Philippe
      – PersonEntity:
          Name:
            NameFull: Baroffio, Anne
      – PersonEntity:
          Name:
            NameFull: Chamot, Eric
      – PersonEntity:
          Name:
            NameFull: Herrmann, François
      – PersonEntity:
          Name:
            NameFull: Nendaz, Mathieu R
      – PersonEntity:
          Name:
            NameFull: Vu, Nu V
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Text: Aug2005
              Type: published
              Y: 2005
          Identifiers:
            – Type: issn-print
              Value: 03080110
          Numbering:
            – Type: volume
              Value: 39
            – Type: issue
              Value: 8
          Titles:
            – TitleFull: Medical Education
              Type: main
ResultId 1