Writing and Reviewing Assessment Questions On-Screen: Issues and Challenges
Saved in:
| Title: | Writing and Reviewing Assessment Questions On-Screen: Issues and Challenges |
|---|---|
| Language: | English |
| Authors: | Crisp, Victoria, Shaw, Stuart |
| Source: | Research Matters. Aut 2020 (30):19-26. |
| Availability: | University of Cambridge Local Examinations Syndicate (Cambridge Assessment). The Triangle Building, Shaftesbury Road, Cambridge, United Kingdom CB2 8EA. Tel: +44-1223-553311; e-mail: info@cambridgeassessment.org.uk; Web site: https://www.cambridgeassessment.org.uk/our-research/all-published-resources/research-matters/ |
| Peer Reviewed: | N |
| Page Count: | 8 |
| Publication Date: | 2020 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Computer Assisted Testing, Test Items, Test Construction, Mathematics Tests, Science Tests, Skills, Knowledge Level, Training, Experience |
| ISSN: | 1755-6031 |
| Abstract: | For assessment contexts where both a paper-based test and an on-screen assessment are available as alternatives, it is still common for the paper-based test to be prepared first with questions later transferred into an on-screen testing platform. One challenge with this is that some questions cannot be transferred. One solution might be for questions to be drafted into the on-screen testing platform and later converted for the paper-based test. This research investigated the issues that might arise if question writers drafted questions directly into the on-screen testing platform and if questions were reviewed directly in the platform. Six assessors with experience of setting and reviewing questions took part. After some familiarisation, each participant attended a research meeting where they drafted some questions into an on-screen testing platform and reviewed some questions in the platform. After each activity, participants completed a workload questionnaire and were interviewed. The findings suggest that training and support would be important. Participants reported feeling restricted when setting items. Evidence suggested that setters may avoid certain item types, write shorter questions than normal or write less creative questions. Setting was also reported to be slower. However, these issues could reduce with greater experience. Overall, it seems that it would be possible for setters to create at least some of their questions within an on-screen testing platform. However, care would be needed to mitigate frustration, ensure question quality and ensure representation of all relevant constructs. |
| Abstractor: | As Provided |
| Entry Date: | 2021 |
| Accession Number: | EJ1286825 |
| Database: | ERIC |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1286825 Name: ERIC Full Text Category: fullText Text: Full Text from ERIC |
|---|---|
| Header | DbId: eric DbLabel: ERIC An: EJ1286825 AccessLevel: 3 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Writing and Reviewing Assessment Questions On-Screen: Issues and Challenges – Name: Language Label: Language Group: Lang Data: English – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Crisp%2C+Victoria%22">Crisp, Victoria</searchLink><br /><searchLink fieldCode="AR" term="%22Shaw%2C+Stuart%22">Shaw, Stuart</searchLink> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="SO" term="%22Research+Matters%22"><i>Research Matters</i></searchLink>. Aut 2020 (30):19-26. – Name: Avail Label: Availability Group: Avail Data: University of Cambridge Local Examinations Syndicate (Cambridge Assessment). The Triangle Building, Shaftesbury Road, Cambridge, United Kingdom CB2 8EA. Tel: +44-1223-553311; e-mail: info@cambridgeassessment.org.uk; Web site: https://www.cambridgeassessment.org.uk/our-research/all-published-resources/research-matters/ – Name: PeerReviewed Label: Peer Reviewed Group: SrcInfo Data: N – Name: Pages Label: Page Count Group: Src Data: 8 – Name: DatePubCY Label: Publication Date Group: Date Data: 2020 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Journal Articles<br />Reports - Research – Name: Subject Label: Descriptors Group: Su Data: <searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematics+Tests%22">Mathematics Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Tests%22">Science Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Skills%22">Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Knowledge+Level%22">Knowledge Level</searchLink><br /><searchLink fieldCode="DE" term="%22Training%22">Training</searchLink><br /><searchLink fieldCode="DE" term="%22Experience%22">Experience</searchLink> – Name: ISSN Label: ISSN Group: ISSN Data: 1755-6031 – Name: Abstract Label: Abstract Group: Ab Data: For assessment contexts where both a paper-based test and an on-screen assessment are available as alternatives, it is still common for the paper-based test to be prepared first with questions later transferred into an on-screen testing platform. One challenge with this is that some questions cannot be transferred. One solution might be for questions to be drafted into the on-screen testing platform and later converted for the paper-based test. This research investigated the issues that might arise if question writers drafted questions directly into the on-screen testing platform and if questions were reviewed directly in the platform. Six assessors with experience of setting and reviewing questions took part. After some familiarisation, each participant attended a research meeting where they drafted some questions into an on-screen testing platform and reviewed some questions in the platform. After each activity, participants completed a workload questionnaire and were interviewed. The findings suggest that training and support would be important. Participants reported feeling restricted when setting items. Evidence suggested that setters may avoid certain item types, write shorter questions than normal or write less creative questions. Setting was also reported to be slower. However, these issues could reduce with greater experience. Overall, it seems that it would be possible for setters to create at least some of their questions within an on-screen testing platform. However, care would be needed to mitigate frustration, ensure question quality and ensure representation of all relevant constructs. – Name: AbstractInfo Label: Abstractor Group: Ab Data: As Provided – Name: DateEntry Label: Entry Date Group: Date Data: 2021 – Name: AN Label: Accession Number Group: ID Data: EJ1286825 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1286825 |
| RecordInfo | BibRecord: BibEntity: Languages: – Text: English PhysicalDescription: Pagination: PageCount: 8 StartPage: 19 Subjects: – SubjectFull: Computer Assisted Testing Type: general – SubjectFull: Test Items Type: general – SubjectFull: Test Construction Type: general – SubjectFull: Mathematics Tests Type: general – SubjectFull: Science Tests Type: general – SubjectFull: Skills Type: general – SubjectFull: Knowledge Level Type: general – SubjectFull: Training Type: general – SubjectFull: Experience Type: general Titles: – TitleFull: Writing and Reviewing Assessment Questions On-Screen: Issues and Challenges Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Crisp, Victoria – PersonEntity: Name: NameFull: Shaw, Stuart IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 01 Type: published Y: 2020 Identifiers: – Type: issn-print Value: 1755-6031 Numbering: – Type: issue Value: 30 Titles: – TitleFull: Research Matters Type: main |
| ResultId | 1 |
