Ordered or Scrambled: How the Forward Sequencing of Multiple Choice Questions Affects Test Item Scores
Saved in:
| Title: | Ordered or Scrambled: How the Forward Sequencing of Multiple Choice Questions Affects Test Item Scores |
|---|---|
| Language: | English |
| Authors: | Cornelis W. Haasnoot (ORCID |
| Source: | Assessment & Evaluation in Higher Education. 2026 51(2):247-262. |
| Availability: | Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals |
| Peer Reviewed: | Y |
| Page Count: | 16 |
| Publication Date: | 2026 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Foreign Countries, Multiple Choice Tests, Sequential Approach, Sequential Learning, Economics Education, Microeconomics, Paper and Pencil Tests, Exit Examinations, Higher Education, Test Results, Scores, Test Bias, Evaluation Methods |
| Geographic Terms: | Netherlands |
| DOI: | 10.1080/02602938.2025.2568187 |
| ISSN: | 0260-2938 1469-297X |
| Abstract: | We analyse the impact on student performance of scrambling the order of multiple choice questions (MCQs) in examinations. We test if students perform better on a question if it is preceded by a question that relates to either the same or the preceding lecture in the course (forward sequenced). Previous research has been unable to find clear evidence of a forward sequencing effect, which we argue could be related to their focus on test level outcomes. By considering test-item level results instead, using a sample of 16,127 MCQ scores from 12 Microeconomics examinations at Radboud University, the Netherlands, we find strong evidence that the forward sequencing of MCQs on examinations positively impacts student performance at the test-item level. An average student has a higher likelihood of answering correctly a MCQ that is in sequence than one that is not. This result holds after controlling for student ability, question difficulty, and a range of other determinants of student performance. The positive effect of forward sequencing builds up in case of a streak of sequenced MCQs. We conclude that the scrambling of MCQs on examinations is not innocuous and we call for consistent scrambling practices across examination versions. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | EJ1499463 |
| Database: | ERIC |
| Abstract: | We analyse the impact on student performance of scrambling the order of multiple choice questions (MCQs) in examinations. We test if students perform better on a question if it is preceded by a question that relates to either the same or the preceding lecture in the course (forward sequenced). Previous research has been unable to find clear evidence of a forward sequencing effect, which we argue could be related to their focus on test level outcomes. By considering test-item level results instead, using a sample of 16,127 MCQ scores from 12 Microeconomics examinations at Radboud University, the Netherlands, we find strong evidence that the forward sequencing of MCQs on examinations positively impacts student performance at the test-item level. An average student has a higher likelihood of answering correctly a MCQ that is in sequence than one that is not. This result holds after controlling for student ability, question difficulty, and a range of other determinants of student performance. The positive effect of forward sequencing builds up in case of a streak of sequenced MCQs. We conclude that the scrambling of MCQs on examinations is not innocuous and we call for consistent scrambling practices across examination versions. |
|---|---|
| ISSN: | 0260-2938 1469-297X |
| DOI: | 10.1080/02602938.2025.2568187 |
