View in EDS

Checkbox grading of large-scale mathematics exams with multiple assessors: Field study on assessors' inter-rater reliability, time investment and usage experience.

Saved in:
Bibliographic Details
Title: Checkbox grading of large-scale mathematics exams with multiple assessors: Field study on assessors' inter-rater reliability, time investment and usage experience.
Authors: Moons, Filip1,2 (AUTHOR) f.moons@uu.nl, Vandervieren, Ellen1,2 (AUTHOR) ellen.vandervieren@uantwerpen.be, Colpaert, Jozef1,2 (AUTHOR) jozef.colpaert@uantwerpen.be
Source: Studies in Educational Evaluation. Jun2025, Vol. 85, pN.PAG-N.PAG. 1p.
Subject Terms: *Mathematics exams, *Field research, Multiples (Mathematics), Appraisers, Ticks
Abstract: Assessing exams with multiple assessors is challenging regarding inter-rater reliability and feedback. This paper presents 'checkbox grading,' a digital method where exam designers have predefined checkboxes with both feedback and associated partial grades. Assessors then tick the checkboxes relevant to a student solution. Dependencies between checkboxes ensure consistency among assessors in following the grading scheme. Moreover, the approach supports 'blind grading' by hiding the grades associated with the checkboxes, thus focusing assessors on the criteria rather than the scores. The approach was studied during a large-scale mathematics state exam. Results show that assessors perceived checkbox grading as very useful. However, compared to traditional grading—where assessors follow a correction scheme and communicate the resulting grade—more time is spent on checkbox grading, while both approaches are equally reliable. Blind grading improved inter-rater reliability for some tasks. Overall, checkbox grading might lead to a smoother process where feedback, not solely grades, is communicated to students. • Checkbox grading uses feedback-linked checkboxes; assessors tick those that apply. • Checkbox dependencies make grading dynamic, forming a flowchart-like structure. • Scores can be shown (visible) or hidden (blind) from assessors during grading. • Blind checkbox grading improves reliability when strict interpretation is needed. • Assessors prefer visible grading for clarity and validation of their judgments. [ABSTRACT FROM AUTHOR]
Copyright of Studies in Educational Evaluation is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
Description
Abstract:Assessing exams with multiple assessors is challenging regarding inter-rater reliability and feedback. This paper presents 'checkbox grading,' a digital method where exam designers have predefined checkboxes with both feedback and associated partial grades. Assessors then tick the checkboxes relevant to a student solution. Dependencies between checkboxes ensure consistency among assessors in following the grading scheme. Moreover, the approach supports 'blind grading' by hiding the grades associated with the checkboxes, thus focusing assessors on the criteria rather than the scores. The approach was studied during a large-scale mathematics state exam. Results show that assessors perceived checkbox grading as very useful. However, compared to traditional grading—where assessors follow a correction scheme and communicate the resulting grade—more time is spent on checkbox grading, while both approaches are equally reliable. Blind grading improved inter-rater reliability for some tasks. Overall, checkbox grading might lead to a smoother process where feedback, not solely grades, is communicated to students. • Checkbox grading uses feedback-linked checkboxes; assessors tick those that apply. • Checkbox dependencies make grading dynamic, forming a flowchart-like structure. • Scores can be shown (visible) or hidden (blind) from assessors during grading. • Blind checkbox grading improves reliability when strict interpretation is needed. • Assessors prefer visible grading for clarity and validation of their judgments. [ABSTRACT FROM AUTHOR]
ISSN:0191491X
DOI:10.1016/j.stueduc.2024.101443