Talk It Out: Concurrent Versus Retrospective Think-Aloud for Complex Verbal Assessment Tasks.
Saved in:
| Title: | Talk It Out: Concurrent Versus Retrospective Think-Aloud for Complex Verbal Assessment Tasks. |
|---|---|
| Authors: | Schreurs, Sanne (AUTHOR), Vullers, Lieke I. A. (AUTHOR), Van Gerven, Pascal W. M. (AUTHOR), Beuken, Juliëtte A. (AUTHOR), Durning, Steven J. (AUTHOR) |
| Source: | Applied Measurement in Education. Jul-Dec2025, Vol. 38 Issue 3/4, p304-317. 14p. |
| Subjects: | Protocol analysis (Cognition), Verbal behavior testing, Assessment of education, Quantitative research, Qualitative research, Cognitive psychology |
| Abstract: | Think-aloud methods provide a window into test-takers' real-time cognitive processes, offering valuable evidence for validity based on response processes. Yet, their use in complex verbal assessments remains limited, particularly concerning the choice between concurrent and retrospective protocols. This multiple-method study directly compared these two approaches in the context of a high-stakes verbal selection test, involving 10 biomedical students equally divided between conditions. Using a combination of deductive coding (based on the assessment blueprint) and inductive thematic analysis, we explored the richness and nature of participants' explanations. Quantitative analysis (interpreted cautiously due to the small sample) further supported the conclusions from the qualitative findings. Participants in the concurrent condition verbalized more frequently, often reading items aloud and articulating their reasoning with greater clarity and depth. In contrast, retrospective participants tended to offer shorter, more fragmented responses, with less transparency in their thought processes. Quantitative results revealed higher assessment scores and significantly more verbal engagement in the concurrent group. Moreover, participants reported the concurrent think-aloud as easier and more natural to perform. These findings may challenge the prevailing assumption that concurrent think-aloud is unsuitable for verbal tasks due to cognitive load. Instead, our results suggest that it can elicit richer, more authentic process data, even in complex verbal assessment contexts, although the preliminary nature of the current study must be emphasized. This study offers a practical contribution to the collection of response process evidence, with clear implications for both researchers and practitioners aiming to enhance the validity and design of educational assessments. [ABSTRACT FROM AUTHOR] |
| Copyright of Applied Measurement in Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Psychology and Behavioral Sciences Collection |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Think-aloud methods provide a window into test-takers' real-time cognitive processes, offering valuable evidence for validity based on response processes. Yet, their use in complex verbal assessments remains limited, particularly concerning the choice between concurrent and retrospective protocols. This multiple-method study directly compared these two approaches in the context of a high-stakes verbal selection test, involving 10 biomedical students equally divided between conditions. Using a combination of deductive coding (based on the assessment blueprint) and inductive thematic analysis, we explored the richness and nature of participants' explanations. Quantitative analysis (interpreted cautiously due to the small sample) further supported the conclusions from the qualitative findings. Participants in the concurrent condition verbalized more frequently, often reading items aloud and articulating their reasoning with greater clarity and depth. In contrast, retrospective participants tended to offer shorter, more fragmented responses, with less transparency in their thought processes. Quantitative results revealed higher assessment scores and significantly more verbal engagement in the concurrent group. Moreover, participants reported the concurrent think-aloud as easier and more natural to perform. These findings may challenge the prevailing assumption that concurrent think-aloud is unsuitable for verbal tasks due to cognitive load. Instead, our results suggest that it can elicit richer, more authentic process data, even in complex verbal assessment contexts, although the preliminary nature of the current study must be emphasized. This study offers a practical contribution to the collection of response process evidence, with clear implications for both researchers and practitioners aiming to enhance the validity and design of educational assessments. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 08957347 |
| DOI: | 10.1080/08957347.2025.2587600 |
