Writing and Reviewing Assessment Questions On-Screen: Issues and Challenges
Saved in:
| Title: | Writing and Reviewing Assessment Questions On-Screen: Issues and Challenges |
|---|---|
| Language: | English |
| Authors: | Crisp, Victoria, Shaw, Stuart |
| Source: | Research Matters. Aut 2020 (30):19-26. |
| Availability: | University of Cambridge Local Examinations Syndicate (Cambridge Assessment). The Triangle Building, Shaftesbury Road, Cambridge, United Kingdom CB2 8EA. Tel: +44-1223-553311; e-mail: info@cambridgeassessment.org.uk; Web site: https://www.cambridgeassessment.org.uk/our-research/all-published-resources/research-matters/ |
| Peer Reviewed: | N |
| Page Count: | 8 |
| Publication Date: | 2020 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Computer Assisted Testing, Test Items, Test Construction, Mathematics Tests, Science Tests, Skills, Knowledge Level, Training, Experience |
| ISSN: | 1755-6031 |
| Abstract: | For assessment contexts where both a paper-based test and an on-screen assessment are available as alternatives, it is still common for the paper-based test to be prepared first with questions later transferred into an on-screen testing platform. One challenge with this is that some questions cannot be transferred. One solution might be for questions to be drafted into the on-screen testing platform and later converted for the paper-based test. This research investigated the issues that might arise if question writers drafted questions directly into the on-screen testing platform and if questions were reviewed directly in the platform. Six assessors with experience of setting and reviewing questions took part. After some familiarisation, each participant attended a research meeting where they drafted some questions into an on-screen testing platform and reviewed some questions in the platform. After each activity, participants completed a workload questionnaire and were interviewed. The findings suggest that training and support would be important. Participants reported feeling restricted when setting items. Evidence suggested that setters may avoid certain item types, write shorter questions than normal or write less creative questions. Setting was also reported to be slower. However, these issues could reduce with greater experience. Overall, it seems that it would be possible for setters to create at least some of their questions within an on-screen testing platform. However, care would be needed to mitigate frustration, ensure question quality and ensure representation of all relevant constructs. |
| Abstractor: | As Provided |
| Entry Date: | 2021 |
| Accession Number: | EJ1286825 |
| Database: | ERIC |
| Abstract: | For assessment contexts where both a paper-based test and an on-screen assessment are available as alternatives, it is still common for the paper-based test to be prepared first with questions later transferred into an on-screen testing platform. One challenge with this is that some questions cannot be transferred. One solution might be for questions to be drafted into the on-screen testing platform and later converted for the paper-based test. This research investigated the issues that might arise if question writers drafted questions directly into the on-screen testing platform and if questions were reviewed directly in the platform. Six assessors with experience of setting and reviewing questions took part. After some familiarisation, each participant attended a research meeting where they drafted some questions into an on-screen testing platform and reviewed some questions in the platform. After each activity, participants completed a workload questionnaire and were interviewed. The findings suggest that training and support would be important. Participants reported feeling restricted when setting items. Evidence suggested that setters may avoid certain item types, write shorter questions than normal or write less creative questions. Setting was also reported to be slower. However, these issues could reduce with greater experience. Overall, it seems that it would be possible for setters to create at least some of their questions within an on-screen testing platform. However, care would be needed to mitigate frustration, ensure question quality and ensure representation of all relevant constructs. |
|---|---|
| ISSN: | 1755-6031 |
