View in EDS

Reliability and Validity of Methods to Assess Undergraduate Healthcare Student Performance in Pharmacology: Comparison of Open Book versus Time-Limited Closed Book Examinations

Saved in:
Bibliographic Details
Title: Reliability and Validity of Methods to Assess Undergraduate Healthcare Student Performance in Pharmacology: Comparison of Open Book versus Time-Limited Closed Book Examinations
Language: English
Authors: David Bell, Vikki O'Neill, Vivienne Crawford
Source: Practitioner Research in Higher Education. 2023 15(1):14-23.
Availability: University of Cumbria. LED Research Centre, Fusehill Street, Carlisle, Cumbria, England CA1 2HH. Tel: +44-1228-616338; Web site: https://ojs.cumbria.ac.uk/index.php/prhe
Peer Reviewed: Y
Page Count: 10
Publication Date: 2023
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Undergraduate Students, Pharmacology, Pharmaceutical Education, Test Format, Timed Tests, Comparative Analysis, Test Reliability, Correlation, Test Items, Predictive Validity, Test Validity, Vignettes, Item Analysis, Pathology, Dental Schools, Dentistry, Foreign Countries
Geographic Terms: United Kingdom (Belfast)
ISSN: 1755-1382
Abstract: We compared the influence of open-book extended duration versus closed book time-limited format on reliability and validity of written assessments of pharmacology learning outcomes within our medical and dental courses. Our dental cohort undertake a mid-year test (30xfree-response short answer to a question, SAQ) and end-of-year paper (4xSAQ, 1xessay, 1xcase) in pharmacology. For our first year medical cohort, pharmacology is integrated within a larger course, contributing 20xclinical vignette questions (to select the single best answer (SBA) to each question from a choice of 5 plausible answers) to a mid-year test and 3-5xSAQ to an end-of-year paper. Our experience indicates that SAQ are as reliable as SBA for closed-book time-limited assessments; reliability correlates with number of questions employed. We have found good correlation between mid-year and end-of-year performance (predictive validity), between questions (factorial validity) and between pharmacology and other subjects within the assessment (concurrent validity). Adoption of open-book extended duration assessments resulted in only modest reduction in reliability and validity.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1409491
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1409491
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1409491
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Reliability and Validity of Methods to Assess Undergraduate Healthcare Student Performance in Pharmacology: Comparison of Open Book versus Time-Limited Closed Book Examinations
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22David+Bell%22">David Bell</searchLink><br /><searchLink fieldCode="AR" term="%22Vikki+O'Neill%22">Vikki O'Neill</searchLink><br /><searchLink fieldCode="AR" term="%22Vivienne+Crawford%22">Vivienne Crawford</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Practitioner+Research+in+Higher+Education%22"><i>Practitioner Research in Higher Education</i></searchLink>. 2023 15(1):14-23.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: University of Cumbria. LED Research Centre, Fusehill Street, Carlisle, Cumbria, England CA1 2HH. Tel: +44-1228-616338; Web site: https://ojs.cumbria.ac.uk/index.php/prhe
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 10
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Pharmacology%22">Pharmacology</searchLink><br /><searchLink fieldCode="DE" term="%22Pharmaceutical+Education%22">Pharmaceutical Education</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22Timed+Tests%22">Timed Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Correlation%22">Correlation</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Predictive+Validity%22">Predictive Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Vignettes%22">Vignettes</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Analysis%22">Item Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Pathology%22">Pathology</searchLink><br /><searchLink fieldCode="DE" term="%22Dental+Schools%22">Dental Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Dentistry%22">Dentistry</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22United+Kingdom+%28Belfast%29%22">United Kingdom (Belfast)</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1755-1382
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: We compared the influence of open-book extended duration versus closed book time-limited format on reliability and validity of written assessments of pharmacology learning outcomes within our medical and dental courses. Our dental cohort undertake a mid-year test (30xfree-response short answer to a question, SAQ) and end-of-year paper (4xSAQ, 1xessay, 1xcase) in pharmacology. For our first year medical cohort, pharmacology is integrated within a larger course, contributing 20xclinical vignette questions (to select the single best answer (SBA) to each question from a choice of 5 plausible answers) to a mid-year test and 3-5xSAQ to an end-of-year paper. Our experience indicates that SAQ are as reliable as SBA for closed-book time-limited assessments; reliability correlates with number of questions employed. We have found good correlation between mid-year and end-of-year performance (predictive validity), between questions (factorial validity) and between pharmacology and other subjects within the assessment (concurrent validity). Adoption of open-book extended duration assessments resulted in only modest reduction in reliability and validity.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1409491
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1409491
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 10
        StartPage: 14
    Subjects:
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: Pharmacology
        Type: general
      – SubjectFull: Pharmaceutical Education
        Type: general
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: Timed Tests
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Correlation
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Predictive Validity
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Vignettes
        Type: general
      – SubjectFull: Item Analysis
        Type: general
      – SubjectFull: Pathology
        Type: general
      – SubjectFull: Dental Schools
        Type: general
      – SubjectFull: Dentistry
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: United Kingdom (Belfast)
        Type: general
    Titles:
      – TitleFull: Reliability and Validity of Methods to Assess Undergraduate Healthcare Student Performance in Pharmacology: Comparison of Open Book versus Time-Limited Closed Book Examinations
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: David Bell
      – PersonEntity:
          Name:
            NameFull: Vikki O'Neill
      – PersonEntity:
          Name:
            NameFull: Vivienne Crawford
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-electronic
              Value: 1755-1382
          Numbering:
            – Type: volume
              Value: 15
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Practitioner Research in Higher Education
              Type: main
ResultId 1