Rater Connectedness Affects Student Achievement Estimates and Ordered Rankings in Formal Music Performance Assessments.
Saved in:
| Title: | Rater Connectedness Affects Student Achievement Estimates and Ordered Rankings in Formal Music Performance Assessments. |
|---|---|
| Authors: | Wind, Stefanie A. (AUTHOR), Wesolowski, Brian C. (AUTHOR), Jones, Eli (AUTHOR) |
| Source: | Journal of Research in Music Education. Apr2026, Vol. 74 Issue 1, p110-131. 22p. |
| Subjects: | Musical performance, Ranking (Statistics), Inter-observer reliability, Academic achievement |
| Abstract: | A persistent concern in music performance assessments is the quality of ratings assigned by judges. Differences in rater judgment are most concerning in cases when student performances are scored by different raters (i.e., disconnected rating designs)—bringing into question the comparability of scores between raters and students. We used data from a formal solo music performance assessment to demonstrate and explore the impact of different data collection designs and a statistical adjustment procedure on the estimates and rank-ordering of student performances. Our results indicated notable discrepancies in conclusions about individual student performances between conditions where all raters scored all students, designs with no common performances between raters, designs that included overlapping performances between raters, and the results from a post hoc adjustment procedure for disconnected designs. We discuss the implications of our results for the design and interpretation of music performance assessments. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Research in Music Education is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Psychology and Behavioral Sciences Collection |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | A persistent concern in music performance assessments is the quality of ratings assigned by judges. Differences in rater judgment are most concerning in cases when student performances are scored by different raters (i.e., disconnected rating designs)—bringing into question the comparability of scores between raters and students. We used data from a formal solo music performance assessment to demonstrate and explore the impact of different data collection designs and a statistical adjustment procedure on the estimates and rank-ordering of student performances. Our results indicated notable discrepancies in conclusions about individual student performances between conditions where all raters scored all students, designs with no common performances between raters, designs that included overlapping performances between raters, and the results from a post hoc adjustment procedure for disconnected designs. We discuss the implications of our results for the design and interpretation of music performance assessments. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 00224294 |
| DOI: | 10.1177/00224294251363327 |
