Evaluating Score and Decision Consistency Across Claims in a Validation Argument.
Saved in:
| Title: | Evaluating Score and Decision Consistency Across Claims in a Validation Argument. |
|---|---|
| Authors: | Schmidgall, Jonathan (AUTHOR) |
| Source: | Applied Measurement in Education. Oct-Dec2017, Vol. 30 Issue 4, p287-296. 10p. |
| Subjects: | Decision making, Generalizability theory, Performance evaluation, Cognitive consistency, Educational Testing Service |
| Abstract: | This study utilizes an argument-based approach to validation to examine the implications of reliability in order to further differentiate the concepts of score and decision consistency. In a methodological example, the framework of generalizability theory was used to estimate appropriate indices of score consistency and evaluations of the likelihood of decision errors based on the design of a performance assessment and its intended use. The study illustrates how generalizability theory can be applied to address various claims about consistency when decisions are based on two or more cut scores, and results underscore the importance of considering score and decision consistency separately. [ABSTRACT FROM AUTHOR] |
| Copyright of Applied Measurement in Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Psychology and Behavioral Sciences Collection |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | This study utilizes an argument-based approach to validation to examine the implications of reliability in order to further differentiate the concepts of score and decision consistency. In a methodological example, the framework of generalizability theory was used to estimate appropriate indices of score consistency and evaluations of the likelihood of decision errors based on the design of a performance assessment and its intended use. The study illustrates how generalizability theory can be applied to address various claims about consistency when decisions are based on two or more cut scores, and results underscore the importance of considering score and decision consistency separately. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 08957347 |
| DOI: | 10.1080/08957347.2017.1353988 |
