Reliability and Validity of Methods to Assess Undergraduate Healthcare Student Performance in Pharmacology: Comparison of Open Book versus Time-Limited Closed Book Examinations
Saved in:
| Title: | Reliability and Validity of Methods to Assess Undergraduate Healthcare Student Performance in Pharmacology: Comparison of Open Book versus Time-Limited Closed Book Examinations |
|---|---|
| Language: | English |
| Authors: | David Bell, Vikki O'Neill, Vivienne Crawford |
| Source: | Practitioner Research in Higher Education. 2023 15(1):14-23. |
| Availability: | University of Cumbria. LED Research Centre, Fusehill Street, Carlisle, Cumbria, England CA1 2HH. Tel: +44-1228-616338; Web site: https://ojs.cumbria.ac.uk/index.php/prhe |
| Peer Reviewed: | Y |
| Page Count: | 10 |
| Publication Date: | 2023 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Undergraduate Students, Pharmacology, Pharmaceutical Education, Test Format, Timed Tests, Comparative Analysis, Test Reliability, Correlation, Test Items, Predictive Validity, Test Validity, Vignettes, Item Analysis, Pathology, Dental Schools, Dentistry, Foreign Countries |
| Geographic Terms: | United Kingdom (Belfast) |
| ISSN: | 1755-1382 |
| Abstract: | We compared the influence of open-book extended duration versus closed book time-limited format on reliability and validity of written assessments of pharmacology learning outcomes within our medical and dental courses. Our dental cohort undertake a mid-year test (30xfree-response short answer to a question, SAQ) and end-of-year paper (4xSAQ, 1xessay, 1xcase) in pharmacology. For our first year medical cohort, pharmacology is integrated within a larger course, contributing 20xclinical vignette questions (to select the single best answer (SBA) to each question from a choice of 5 plausible answers) to a mid-year test and 3-5xSAQ to an end-of-year paper. Our experience indicates that SAQ are as reliable as SBA for closed-book time-limited assessments; reliability correlates with number of questions employed. We have found good correlation between mid-year and end-of-year performance (predictive validity), between questions (factorial validity) and between pharmacology and other subjects within the assessment (concurrent validity). Adoption of open-book extended duration assessments resulted in only modest reduction in reliability and validity. |
| Abstractor: | As Provided |
| Entry Date: | 2024 |
| Accession Number: | EJ1409491 |
| Database: | ERIC |
| Abstract: | We compared the influence of open-book extended duration versus closed book time-limited format on reliability and validity of written assessments of pharmacology learning outcomes within our medical and dental courses. Our dental cohort undertake a mid-year test (30xfree-response short answer to a question, SAQ) and end-of-year paper (4xSAQ, 1xessay, 1xcase) in pharmacology. For our first year medical cohort, pharmacology is integrated within a larger course, contributing 20xclinical vignette questions (to select the single best answer (SBA) to each question from a choice of 5 plausible answers) to a mid-year test and 3-5xSAQ to an end-of-year paper. Our experience indicates that SAQ are as reliable as SBA for closed-book time-limited assessments; reliability correlates with number of questions employed. We have found good correlation between mid-year and end-of-year performance (predictive validity), between questions (factorial validity) and between pharmacology and other subjects within the assessment (concurrent validity). Adoption of open-book extended duration assessments resulted in only modest reduction in reliability and validity. |
|---|---|
| ISSN: | 1755-1382 |
