Using Process Data to Evaluate the Impact of Shortening Allotted Case Time in a Simulation‐Based Assessment.
Saved in:
| Title: | Using Process Data to Evaluate the Impact of Shortening Allotted Case Time in a Simulation‐Based Assessment. |
|---|---|
| Authors: | Liu, Chunyan1 (AUTHOR), Cuddy, Monica M.1 (AUTHOR), He, Qiwei2 (AUTHOR), Ouyang, Wenli1 (AUTHOR), Artman, Cara1 (AUTHOR) |
| Source: | Educational Measurement: Issues & Practice. Dec2024, Vol. 43 Issue 4, p24-32. 9p. |
| Subject Terms: | Timestamps, Physicians, Percentiles |
| Abstract: | The Computer‐based Case Simulations (CCS) component of the United States Medical Licensing Examination (USMLE) Step 3 was developed to assess the decision‐making and patient‐management skills of physicians. Process data can provide deep insights into examinees' behavioral processes related to completing the CCS assessment task. In this paper, we utilized process data to evaluate the impact of shortening allotted time limit by rescoring the CCS cases based on process data extracted at various timestamps that represented different percentages of the original allotted case time. It was found that examinees' performance as well as the correlation between original and newly generated scores both tended to decrease as the timestamp condition became stricter. The impact of shortening allotted time limit was found marginally associated with case difficulties, but strongly dependent on the case time intensity under the original time setting. [ABSTRACT FROM AUTHOR] |
| Copyright of Educational Measurement: Issues & Practice is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | The Computer‐based Case Simulations (CCS) component of the United States Medical Licensing Examination (USMLE) Step 3 was developed to assess the decision‐making and patient‐management skills of physicians. Process data can provide deep insights into examinees' behavioral processes related to completing the CCS assessment task. In this paper, we utilized process data to evaluate the impact of shortening allotted time limit by rescoring the CCS cases based on process data extracted at various timestamps that represented different percentages of the original allotted case time. It was found that examinees' performance as well as the correlation between original and newly generated scores both tended to decrease as the timestamp condition became stricter. The impact of shortening allotted time limit was found marginally associated with case difficulties, but strongly dependent on the case time intensity under the original time setting. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 07311745 |
| DOI: | 10.1111/emip.12656 |
