Cognitive Echo: Enhancing Think-Aloud Protocols with LLM-Based Simulated Students
Saved in:
| Title: | Cognitive Echo: Enhancing Think-Aloud Protocols with LLM-Based Simulated Students |
|---|---|
| Language: | English |
| Authors: | Longwei Zheng, Anna He, Changyong Qi, Haomin Zhang, Xiaoqing Gu (ORCID |
| Source: | British Journal of Educational Technology. 2025 56(5):2019-2042. |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 24 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Protocol Analysis, Learning Experience, Computational Linguistics, Computer Software, Learning Processes, Interference (Learning), Artificial Intelligence, Generalization, Learning Strategies, Computer Simulation, Authentic Learning, Playgrounds, Transfer of Training |
| DOI: | 10.1111/bjet.13590 |
| ISSN: | 0007-1013 1467-8535 |
| Abstract: | In the field of education, the think-aloud protocol is commonly used to encourage learners to articulate their thoughts during the learning process, providing observers with valuable insights into learners' cognitive processes beyond the final learning outcomes. However, the implementation of think-aloud protocols faces challenges such as task interference and limitations in completeness and authenticity of verbal reports. This study proposes a method called Cognitive Echo, which leverages large language models (LLMs) trained with simulated student experiences to enhance the completeness and authenticity of think-aloud verbalizations. LLMs have been demonstrated to simulate human-like behaviour more effectively by memorizing experiences. In this work, we introduce specific learner roles and train the LLMs to act as distinct learners. Our method involves integrating transaction data from learners' interactions with a tutoring system and the tutor's content to create interactive experiences between learners and teachers, thereby training the model to become simulated students with learning experiences. To investigate the effectiveness of this approach, we designed a test playground based on the retrospective think-aloud protocol and examined how LLM-trained simulated students improve cognitive process transparency and generalization of learning strategies. The study found that Cognitive Echo not only reveals what simulated students genuinely think about their learning experiences but also enables them to transfer their different cognitive strategies to new tasks. By training simulated students on real learning behaviour data to ensure their cognitive processes reflect authentic learner experiences, this approach will extend think-aloud protocols to more practice-oriented applications. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1480023 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | In the field of education, the think-aloud protocol is commonly used to encourage learners to articulate their thoughts during the learning process, providing observers with valuable insights into learners' cognitive processes beyond the final learning outcomes. However, the implementation of think-aloud protocols faces challenges such as task interference and limitations in completeness and authenticity of verbal reports. This study proposes a method called Cognitive Echo, which leverages large language models (LLMs) trained with simulated student experiences to enhance the completeness and authenticity of think-aloud verbalizations. LLMs have been demonstrated to simulate human-like behaviour more effectively by memorizing experiences. In this work, we introduce specific learner roles and train the LLMs to act as distinct learners. Our method involves integrating transaction data from learners' interactions with a tutoring system and the tutor's content to create interactive experiences between learners and teachers, thereby training the model to become simulated students with learning experiences. To investigate the effectiveness of this approach, we designed a test playground based on the retrospective think-aloud protocol and examined how LLM-trained simulated students improve cognitive process transparency and generalization of learning strategies. The study found that Cognitive Echo not only reveals what simulated students genuinely think about their learning experiences but also enables them to transfer their different cognitive strategies to new tasks. By training simulated students on real learning behaviour data to ensure their cognitive processes reflect authentic learner experiences, this approach will extend think-aloud protocols to more practice-oriented applications. |
|---|---|
| ISSN: | 0007-1013 1467-8535 |
| DOI: | 10.1111/bjet.13590 |
