View in EDS HTML Full Text PDF Full Text

Talk It Out: Concurrent Versus Retrospective Think-Aloud for Complex Verbal Assessment Tasks.

Saved in:
Bibliographic Details
Title: Talk It Out: Concurrent Versus Retrospective Think-Aloud for Complex Verbal Assessment Tasks.
Authors: Schreurs, Sanne (AUTHOR), Vullers, Lieke I. A. (AUTHOR), Van Gerven, Pascal W. M. (AUTHOR), Beuken, Juliëtte A. (AUTHOR), Durning, Steven J. (AUTHOR)
Source: Applied Measurement in Education. Jul-Dec2025, Vol. 38 Issue 3/4, p304-317. 14p.
Subjects: Protocol analysis (Cognition), Verbal behavior testing, Assessment of education, Quantitative research, Qualitative research, Cognitive psychology
Abstract: Think-aloud methods provide a window into test-takers' real-time cognitive processes, offering valuable evidence for validity based on response processes. Yet, their use in complex verbal assessments remains limited, particularly concerning the choice between concurrent and retrospective protocols. This multiple-method study directly compared these two approaches in the context of a high-stakes verbal selection test, involving 10 biomedical students equally divided between conditions. Using a combination of deductive coding (based on the assessment blueprint) and inductive thematic analysis, we explored the richness and nature of participants' explanations. Quantitative analysis (interpreted cautiously due to the small sample) further supported the conclusions from the qualitative findings. Participants in the concurrent condition verbalized more frequently, often reading items aloud and articulating their reasoning with greater clarity and depth. In contrast, retrospective participants tended to offer shorter, more fragmented responses, with less transparency in their thought processes. Quantitative results revealed higher assessment scores and significantly more verbal engagement in the concurrent group. Moreover, participants reported the concurrent think-aloud as easier and more natural to perform. These findings may challenge the prevailing assumption that concurrent think-aloud is unsuitable for verbal tasks due to cognitive load. Instead, our results suggest that it can elicit richer, more authentic process data, even in complex verbal assessment contexts, although the preliminary nature of the current study must be emphasized. This study offers a practical contribution to the collection of response process evidence, with clear implications for both researchers and practitioners aiming to enhance the validity and design of educational assessments. [ABSTRACT FROM AUTHOR]
Copyright of Applied Measurement in Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Psychology and Behavioral Sciences Collection
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: pbh
DbLabel: Psychology and Behavioral Sciences Collection
An: 192312335
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Talk It Out: Concurrent Versus Retrospective Think-Aloud for Complex Verbal Assessment Tasks.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Schreurs%2C+Sanne%22">Schreurs, Sanne</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Vullers%2C+Lieke+I%2E+A%2E%22">Vullers, Lieke I. A.</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Van+Gerven%2C+Pascal+W%2E+M%2E%22">Van Gerven, Pascal W. M.</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Beuken%2C+Juliëtte+A%2E%22">Beuken, Juliëtte A.</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Durning%2C+Steven+J%2E%22">Durning, Steven J.</searchLink> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Applied+Measurement+in+Education%22">Applied Measurement in Education</searchLink>. Jul-Dec2025, Vol. 38 Issue 3/4, p304-317. 14p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Protocol+analysis+%28Cognition%29%22">Protocol analysis (Cognition)</searchLink><br /><searchLink fieldCode="DE" term="%22Verbal+behavior+testing%22">Verbal behavior testing</searchLink><br /><searchLink fieldCode="DE" term="%22Assessment+of+education%22">Assessment of education</searchLink><br /><searchLink fieldCode="DE" term="%22Quantitative+research%22">Quantitative research</searchLink><br /><searchLink fieldCode="DE" term="%22Qualitative+research%22">Qualitative research</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+psychology%22">Cognitive psychology</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Think-aloud methods provide a window into test-takers' real-time cognitive processes, offering valuable evidence for validity based on response processes. Yet, their use in complex verbal assessments remains limited, particularly concerning the choice between concurrent and retrospective protocols. This multiple-method study directly compared these two approaches in the context of a high-stakes verbal selection test, involving 10 biomedical students equally divided between conditions. Using a combination of deductive coding (based on the assessment blueprint) and inductive thematic analysis, we explored the richness and nature of participants' explanations. Quantitative analysis (interpreted cautiously due to the small sample) further supported the conclusions from the qualitative findings. Participants in the concurrent condition verbalized more frequently, often reading items aloud and articulating their reasoning with greater clarity and depth. In contrast, retrospective participants tended to offer shorter, more fragmented responses, with less transparency in their thought processes. Quantitative results revealed higher assessment scores and significantly more verbal engagement in the concurrent group. Moreover, participants reported the concurrent think-aloud as easier and more natural to perform. These findings may challenge the prevailing assumption that concurrent think-aloud is unsuitable for verbal tasks due to cognitive load. Instead, our results suggest that it can elicit richer, more authentic process data, even in complex verbal assessment contexts, although the preliminary nature of the current study must be emphasized. This study offers a practical contribution to the collection of response process evidence, with clear implications for both researchers and practitioners aiming to enhance the validity and design of educational assessments. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Applied Measurement in Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=pbh&AN=192312335
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/08957347.2025.2587600
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 14
        StartPage: 304
    Subjects:
      – SubjectFull: Protocol analysis (Cognition)
        Type: general
      – SubjectFull: Verbal behavior testing
        Type: general
      – SubjectFull: Assessment of education
        Type: general
      – SubjectFull: Quantitative research
        Type: general
      – SubjectFull: Qualitative research
        Type: general
      – SubjectFull: Cognitive psychology
        Type: general
    Titles:
      – TitleFull: Talk It Out: Concurrent Versus Retrospective Think-Aloud for Complex Verbal Assessment Tasks.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Schreurs, Sanne
      – PersonEntity:
          Name:
            NameFull: Vullers, Lieke I. A.
      – PersonEntity:
          Name:
            NameFull: Van Gerven, Pascal W. M.
      – PersonEntity:
          Name:
            NameFull: Beuken, Juliëtte A.
      – PersonEntity:
          Name:
            NameFull: Durning, Steven J.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Text: Jul-Dec2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 08957347
          Numbering:
            – Type: volume
              Value: 38
            – Type: issue
              Value: 3/4
          Titles:
            – TitleFull: Applied Measurement in Education
              Type: main
ResultId 1