Checkbox grading of handwritten mathematics exams with multiple assessors: how do students react to the resulting atomic feedback? A mixed-method study.
Saved in:
| Title: | Checkbox grading of handwritten mathematics exams with multiple assessors: how do students react to the resulting atomic feedback? A mixed-method study. |
|---|---|
| Authors: | Moons, Filip1,2 (AUTHOR) f.moons@uu.nl, Iannone, Paola3 (AUTHOR), Vandervieren, Ellen2 (AUTHOR) |
| Source: | ZDM - Mathematics Education. Aug2024, Vol. 56 Issue 4, p757-773. 17p. |
| Subject Terms: | *Psychology of students, *High schools, *Psychological feedback, Multiples (Mathematics), Semi-structured interviews, Appraisers |
| Abstract: | Handwritten tasks are better suited than digital ones to assess higher-order mathematics skills, as students can express themselves more freely. However, maintaining reliability and providing feedback can be challenging when assessing high-stakes, handwritten mathematics exams involving multiple assessors. This paper discusses a new semi-automated grading approach called 'checkbox grading'. Checkbox grading gives each assessor a list of checkboxes consisting of feedback items for each task. The assessor then ticks those feedback items which apply to the student's solution. Dependencies between the checkboxes can be set to ensure all assessors take the same route on the grading scheme. The system then automatically calculates the grade and provides atomic feedback to the student, giving a detailed insight into what went wrong and how the grade was obtained. Atomic feedback consists of a set of format requirements for mathematical feedback items, which has been shown to increase feedback's reusability. Checkbox grading was tested during the final high school mathematics exam (grade 12) organised by the Flemish Exam Commission, with 60 students and 10 assessors. This paper focuses on students' perceptions of the received checkbox grading feedback and how easily they interpreted it. After the exam was graded, all students were sent an online questionnaire, including their personalised exam feedback. The questionnaire was filled in by 36 students, and 4 of them participated in semi-structured interviews. Findings suggest that students could interpret the feedback from checkbox grading well, with no correlation between students' exam scores and feedback understanding. Therefore, we suggest that checkbox grading is an effective way to provide feedback, also for students with shaky subject matter knowledge. [ABSTRACT FROM AUTHOR] |
| Copyright of ZDM - Mathematics Education is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 179459811 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Checkbox grading of handwritten mathematics exams with multiple assessors: how do students react to the resulting atomic feedback? A mixed-method study. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Moons%2C+Filip%22">Moons, Filip</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> f.moons@uu.nl</i><br /><searchLink fieldCode="AR" term="%22Iannone%2C+Paola%22">Iannone, Paola</searchLink><relatesTo>3</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Vandervieren%2C+Ellen%22">Vandervieren, Ellen</searchLink><relatesTo>2</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22ZDM+-+Mathematics+Education%22">ZDM - Mathematics Education</searchLink>. Aug2024, Vol. 56 Issue 4, p757-773. 17p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Psychology+of+students%22">Psychology of students</searchLink><br />*<searchLink fieldCode="DE" term="%22High+schools%22">High schools</searchLink><br />*<searchLink fieldCode="DE" term="%22Psychological+feedback%22">Psychological feedback</searchLink><br /><searchLink fieldCode="DE" term="%22Multiples+%28Mathematics%29%22">Multiples (Mathematics)</searchLink><br /><searchLink fieldCode="DE" term="%22Semi-structured+interviews%22">Semi-structured interviews</searchLink><br /><searchLink fieldCode="DE" term="%22Appraisers%22">Appraisers</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Handwritten tasks are better suited than digital ones to assess higher-order mathematics skills, as students can express themselves more freely. However, maintaining reliability and providing feedback can be challenging when assessing high-stakes, handwritten mathematics exams involving multiple assessors. This paper discusses a new semi-automated grading approach called 'checkbox grading'. Checkbox grading gives each assessor a list of checkboxes consisting of feedback items for each task. The assessor then ticks those feedback items which apply to the student's solution. Dependencies between the checkboxes can be set to ensure all assessors take the same route on the grading scheme. The system then automatically calculates the grade and provides atomic feedback to the student, giving a detailed insight into what went wrong and how the grade was obtained. Atomic feedback consists of a set of format requirements for mathematical feedback items, which has been shown to increase feedback's reusability. Checkbox grading was tested during the final high school mathematics exam (grade 12) organised by the Flemish Exam Commission, with 60 students and 10 assessors. This paper focuses on students' perceptions of the received checkbox grading feedback and how easily they interpreted it. After the exam was graded, all students were sent an online questionnaire, including their personalised exam feedback. The questionnaire was filled in by 36 students, and 4 of them participated in semi-structured interviews. Findings suggest that students could interpret the feedback from checkbox grading well, with no correlation between students' exam scores and feedback understanding. Therefore, we suggest that checkbox grading is an effective way to provide feedback, also for students with shaky subject matter knowledge. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of ZDM - Mathematics Education is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=179459811 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s11858-024-01550-6 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 17 StartPage: 757 Subjects: – SubjectFull: Psychology of students Type: general – SubjectFull: High schools Type: general – SubjectFull: Psychological feedback Type: general – SubjectFull: Multiples (Mathematics) Type: general – SubjectFull: Semi-structured interviews Type: general – SubjectFull: Appraisers Type: general Titles: – TitleFull: Checkbox grading of handwritten mathematics exams with multiple assessors: how do students react to the resulting atomic feedback? A mixed-method study. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Moons, Filip – PersonEntity: Name: NameFull: Iannone, Paola – PersonEntity: Name: NameFull: Vandervieren, Ellen IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 08 Text: Aug2024 Type: published Y: 2024 Identifiers: – Type: issn-print Value: 18639690 Numbering: – Type: volume Value: 56 – Type: issue Value: 4 Titles: – TitleFull: ZDM - Mathematics Education Type: main |
| ResultId | 1 |
