View in EDS HTML Full Text PDF Full Text

Further challenges in measuring communication skills: accounting for actor effects in standardised patient assessments.

Saved in:
Bibliographic Details
Title: Further challenges in measuring communication skills: accounting for actor effects in standardised patient assessments.
Authors: Lurie, Stephen J (AUTHOR), Mooney, Christopher J (AUTHOR), Nofziger, Anne C (AUTHOR), Meldrum, Sean C (AUTHOR), Epstein, Ronald M (AUTHOR)
Source: Medical Education. Jul2008, Vol. 42 Issue 7, p662-668. 7p. 1 Chart, 1 Graph.
Subjects: Medical research, Patient-professional relations, Health counseling, Medical teaching personnel, Health outcome assessment
Abstract: Context Subjective rating scales for communication skills may yield more personally meaningful responses than more standardised rating schemes. It is unclear, however, whether such evaluations may be overly biased by respondents’ rating styles, which may lead to unreliable measurement of examinees’ communication skills. Methods Our study involved 212 students from the classes of 2005 and 2006 at the University of Rochester School of Medicine and Dentistry. All students were rated by actors depicting standardised patients (SPs) on the same seven cases using the 19-item Rochester Communication Rating Scale (RCRS). Different students were assigned to different actors playing the same SP. We assessed the extent to which actors’ personal rating styles influenced the scores they assigned to students. Main outcome measures were: between-actor variability in responses; the degree to which actors’ response styles contribute to overall scores, and improvements in reliability achieved by standardising actors’ ratings. Results There were statistically significant differences between actors in their mean assigned scores. Scores aggregated over 18 separate SP cases have an expected generalisability coefficient of 0.79. If raw RCRS scores are used, a total of 27 replications of the RCRS are required to achieve a Cronbach’s alpha of 0.8; standardisation reduces this number to 18. Conclusions Although actors are variable in their use of a standardised subjective scale of communication, such differences contribute to an acceptably small proportion of the total variance if scores are combined across a large number of cases. Reliability can be markedly improved by standardising scores across raters. [ABSTRACT FROM AUTHOR]
Copyright of Medical Education is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Psychology and Behavioral Sciences Collection
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: pbh
DbLabel: Psychology and Behavioral Sciences Collection
An: 32563826
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Further challenges in measuring communication skills: accounting for actor effects in standardised patient assessments.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Lurie%2C+Stephen+J%22">Lurie, Stephen J</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Mooney%2C+Christopher+J%22">Mooney, Christopher J</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Nofziger%2C+Anne+C%22">Nofziger, Anne C</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Meldrum%2C+Sean+C%22">Meldrum, Sean C</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Epstein%2C+Ronald+M%22">Epstein, Ronald M</searchLink> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Medical+Education%22">Medical Education</searchLink>. Jul2008, Vol. 42 Issue 7, p662-668. 7p. 1 Chart, 1 Graph.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Medical+research%22">Medical research</searchLink><br /><searchLink fieldCode="DE" term="%22Patient-professional+relations%22">Patient-professional relations</searchLink><br /><searchLink fieldCode="DE" term="%22Health+counseling%22">Health counseling</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+teaching+personnel%22">Medical teaching personnel</searchLink><br /><searchLink fieldCode="DE" term="%22Health+outcome+assessment%22">Health outcome assessment</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Context Subjective rating scales for communication skills may yield more personally meaningful responses than more standardised rating schemes. It is unclear, however, whether such evaluations may be overly biased by respondents’ rating styles, which may lead to unreliable measurement of examinees’ communication skills. Methods Our study involved 212 students from the classes of 2005 and 2006 at the University of Rochester School of Medicine and Dentistry. All students were rated by actors depicting standardised patients (SPs) on the same seven cases using the 19-item Rochester Communication Rating Scale (RCRS). Different students were assigned to different actors playing the same SP. We assessed the extent to which actors’ personal rating styles influenced the scores they assigned to students. Main outcome measures were: between-actor variability in responses; the degree to which actors’ response styles contribute to overall scores, and improvements in reliability achieved by standardising actors’ ratings. Results There were statistically significant differences between actors in their mean assigned scores. Scores aggregated over 18 separate SP cases have an expected generalisability coefficient of 0.79. If raw RCRS scores are used, a total of 27 replications of the RCRS are required to achieve a Cronbach’s alpha of 0.8; standardisation reduces this number to 18. Conclusions Although actors are variable in their use of a standardised subjective scale of communication, such differences contribute to an acceptably small proportion of the total variance if scores are combined across a large number of cases. Reliability can be markedly improved by standardising scores across raters. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Medical Education is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=pbh&AN=32563826
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/j.1365-2923.2008.03080.x
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 7
        StartPage: 662
    Subjects:
      – SubjectFull: Medical research
        Type: general
      – SubjectFull: Patient-professional relations
        Type: general
      – SubjectFull: Health counseling
        Type: general
      – SubjectFull: Medical teaching personnel
        Type: general
      – SubjectFull: Health outcome assessment
        Type: general
    Titles:
      – TitleFull: Further challenges in measuring communication skills: accounting for actor effects in standardised patient assessments.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Lurie, Stephen J
      – PersonEntity:
          Name:
            NameFull: Mooney, Christopher J
      – PersonEntity:
          Name:
            NameFull: Nofziger, Anne C
      – PersonEntity:
          Name:
            NameFull: Meldrum, Sean C
      – PersonEntity:
          Name:
            NameFull: Epstein, Ronald M
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Text: Jul2008
              Type: published
              Y: 2008
          Identifiers:
            – Type: issn-print
              Value: 03080110
          Numbering:
            – Type: volume
              Value: 42
            – Type: issue
              Value: 7
          Titles:
            – TitleFull: Medical Education
              Type: main
ResultId 1