Detecting Measurement Disturbances in Rater-Mediated Assessments.
Saved in:
| Title: | Detecting Measurement Disturbances in Rater-Mediated Assessments. |
|---|---|
| Authors: | Wind, Stefanie A.1 swind@ua.edu, Schumacker, Randall E.1 |
| Source: | Educational Measurement: Issues & Practice. Winter2017, Vol. 36 Issue 4, p44-51. 8p. |
| Subject Terms: | *Academic achievement, *Educational tests & measurements, SERVQUAL (Service quality framework), Psychometrics, Psychiatric rating scales |
| Abstract: | The term measurement disturbance has been used to describe systematic conditions that affect a measurement process, resulting in a compromised interpretation of person or item estimates. Measurement disturbances have been discussed in relation to systematic response patterns associated with items and persons, such as start-up, plodding, boredom, or fatigue. An understanding of the different types of measurement disturbances can lead to a more complete understanding of persons or items in terms of the construct being measured. Although measurement disturbances have been explored in several contexts, they have not been explicitly considered in the context of performance assessments. The purpose of this study is to illustrate the use of graphical methods to explore measurement disturbances related to raters within the context of a writing assessment. Graphical displays that illustrate the alignment between expected and empirical rater response functions are considered as they relate to indicators of rating quality based on the Rasch model. Results suggest that graphical displays can be used to identify measurement disturbances for raters related to specific ranges of student achievement that suggest potential rater bias. Further, results highlight the added diagnostic value of graphical displays for detecting measurement disturbances that are not captured using Rasch model-data fit statistics. [ABSTRACT FROM AUTHOR] |
| Copyright of Educational Measurement: Issues & Practice is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
Be the first to leave a comment!
