View in EDS HTML Full Text PDF Full Text

Cognitive Echo: Enhancing Think-Aloud Protocols with LLM-Based Simulated Students

Saved in:
Bibliographic Details
Title: Cognitive Echo: Enhancing Think-Aloud Protocols with LLM-Based Simulated Students
Language: English
Authors: Longwei Zheng, Anna He, Changyong Qi, Haomin Zhang, Xiaoqing Gu (ORCID 0000-0001-8256-5408)
Source: British Journal of Educational Technology. 2025 56(5):2019-2042.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 24
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Descriptors: Protocol Analysis, Learning Experience, Computational Linguistics, Computer Software, Learning Processes, Interference (Learning), Artificial Intelligence, Generalization, Learning Strategies, Computer Simulation, Authentic Learning, Playgrounds, Transfer of Training
DOI: 10.1111/bjet.13590
ISSN: 0007-1013
1467-8535
Abstract: In the field of education, the think-aloud protocol is commonly used to encourage learners to articulate their thoughts during the learning process, providing observers with valuable insights into learners' cognitive processes beyond the final learning outcomes. However, the implementation of think-aloud protocols faces challenges such as task interference and limitations in completeness and authenticity of verbal reports. This study proposes a method called Cognitive Echo, which leverages large language models (LLMs) trained with simulated student experiences to enhance the completeness and authenticity of think-aloud verbalizations. LLMs have been demonstrated to simulate human-like behaviour more effectively by memorizing experiences. In this work, we introduce specific learner roles and train the LLMs to act as distinct learners. Our method involves integrating transaction data from learners' interactions with a tutoring system and the tutor's content to create interactive experiences between learners and teachers, thereby training the model to become simulated students with learning experiences. To investigate the effectiveness of this approach, we designed a test playground based on the retrospective think-aloud protocol and examined how LLM-trained simulated students improve cognitive process transparency and generalization of learning strategies. The study found that Cognitive Echo not only reveals what simulated students genuinely think about their learning experiences but also enables them to transfer their different cognitive strategies to new tasks. By training simulated students on real learning behaviour data to ensure their cognitive processes reflect authentic learner experiences, this approach will extend think-aloud protocols to more practice-oriented applications.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1480023
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwHch38_hdbABnb1sd-pU5WRAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDBuwlP2apoxEGrlQkQIBEICBm_ZQ4Haf1I9Dw49k1eaWpIPfBfoXURF-0xc4l0EcAIzKDF_YvTbTanTGo_VlIgisR15q2Ue2bqkULD53mrp3zb6EG2huzjFtrZuzlrjPY1zUuUJLpMrcrG1ugLXH_cb9YEp-FWSU4wKAr7VJ-vVXqwg6PAnu7DUo6RWVE--PidOM4oTnMS6G9pIjhmx_H9ZBHopeWJZD3LxlOd1D
Text:
  Availability: 1
  Value: <anid>AN0187257455;58i01sep.25;2025Aug14.00:21;v2.2.500</anid> <title id="AN0187257455-1">Cognitive Echo: Enhancing think‐aloud protocols with LLM‐based simulated students </title> <p>In the field of education, the think‐aloud protocol is commonly used to encourage learners to articulate their thoughts during the learning process, providing observers with valuable insights into learners' cognitive processes beyond the final learning outcomes. However, the implementation of think‐aloud protocols faces challenges such as task interference and limitations in completeness and authenticity of verbal reports. This study proposes a method called Cognitive Echo, which leverages large language models (LLMs) trained with simulated student experiences to enhance the completeness and authenticity of think‐aloud verbalizations. LLMs have been demonstrated to simulate human‐like behaviour more effectively by memorizing experiences. In this work, we introduce specific learner roles and train the LLMs to act as distinct learners. Our method involves integrating transaction data from learners' interactions with a tutoring system and the tutor's content to create interactive experiences between learners and teachers, thereby training the model to become simulated students with learning experiences. To investigate the effectiveness of this approach, we designed a test playground based on the retrospective think‐aloud protocol and examined how LLM‐trained simulated students improve cognitive process transparency and generalization of learning strategies. The study found that Cognitive Echo not only reveals what simulated students genuinely think about their learning experiences but also enables them to transfer their different cognitive strategies to new tasks. By training simulated students on real learning behaviour data to ensure their cognitive processes reflect authentic learner experiences, this approach will extend think‐aloud protocols to more practice‐oriented applications. Practitioner notesWhat is already known about this topic Think‐aloud protocols are widely used in educational settings to explore students' cognitive processes by asking them to verbalize their thoughts while solving problems, but they are prone to issues like task interference and incomplete data reporting.Existed applications of simulating student cognition in educational research are rigid and less adaptive to individual learner characteristics.Artificial intelligences, especially large language models, have shown promise in educational contexts, particularly for simulating human‐like behaviours.What this paper adds This paper introduces the concept of Cognitive Echo, a method that integrates LLM‐powered simulated students into think‐aloud protocols, which addresses the limitations of traditional verbalization‐based methods by leveraging retrospective data.The study shows that LLMs, when fine‐tuned with authentic learner experiences, can replicate distinct human‐like cognitive processes, enabling a more complete and authentic simulation of how students think and solve problems.It demonstrates how the use of LLMs to simulate students' cognitive processes can enhance the transparency and completeness of think‐aloud protocols by allowing researchers to capture cognitive strategies and behaviours that would otherwise go unspoken.Implications for practice and/or policy Teacher training programmes can benefit from integrating LLM‐based simulated students, which enable preservice teachers to practice responding to a wide range of cognitive processes and challenges without the constraints of real‐time think‐aloud tasks.The Cognitive Echo method, by offering a more authentic and less intrusive way of capturing student cognition, can be applied in teacher training scenarios where simulation of real‐world classroom dynamics is crucial for developing pedagogical skills.The use of Cognitive Echo could help in the creation of digital twins of educational scenarios, facilitating research into complex educational issues (eg, bullying and learning disabilities) through simulations that model real‐world interactions.</p> <p>Keywords: agents; Cognitive Echo; LLMs; simulated students; think‐aloud protocols</p> <hd id="AN0187257455-2">INTRODUCTION</hd> <p>The think‐aloud protocol, a well‐established method in cognitive psychology, has been widely utilized in educational research. Its primary purpose is to provide researchers with real‐time insights into individuals' cognitive processes during task completion (Fan et al., [<reflink idref="bib23" id="ref1">23</reflink>]; Loxterman et al., [<reflink idref="bib44" id="ref2">44</reflink>]). However, the method has limitations regarding the reliability and completeness of the data it generates. For instance, participants may inadvertently omit certain aspects of their cognitive processes while verbalizing, resulting in incomplete data (Roth et al., [<reflink idref="bib62" id="ref3">62</reflink>]). Moreover, verbalizations from human participants often fail to capture implicit cognitive mechanisms and strategic reasoning, as these processes may remain unspoken or unconscious during task execution (Charters, [<reflink idref="bib11" id="ref4">11</reflink>]; Jacobse & Harskamp, [<reflink idref="bib32" id="ref5">32</reflink>]; Nielsen et al., [<reflink idref="bib50" id="ref6">50</reflink>]). This limitation hinders researchers' ability to obtain a comprehensive understanding of learners' cognitive processes. With the advancement of agents powered by large language models (LLMs), there is increasing potential to address these limitations through integration with think‐aloud protocols (Chu et al., [<reflink idref="bib16" id="ref7">16</reflink>]). To further enhance the practicality of think‐aloud protocols, this study emphasizes training simulated students based on real learner behaviour data, ensuring that LLM‐generated verbalizations are grounded in authentic learning processes and applicable to real‐world educational settings.</p> <p>Recent developments in LLM‐enhanced think‐aloud protocols have shown promise in simulating students within intelligent tutoring systems (ITS). Simulated students are computer systems designed to replicate human learners' behaviours and cognitive processes (Van Lehn et al., [<reflink idref="bib74" id="ref8">74</reflink>]). A few studies on student model development in ITS have utilized machine learning to simulate student learning processes by generalizing learning trajectories from expert demonstrations and refining rules through feedback (Matsuda et al., [<reflink idref="bib47" id="ref9">47</reflink>]). While such methods shed light on learners' cognitive mechanisms, their applicability is constrained by limited flexibility. Recent developments in LLMs have opened up new avenues for simulating cognitive processes. Unlike traditional simulation techniques, LLMs can generate not only coherent and contextually appropriate content but also a broad spectrum of complex human behaviours (Park et al., [<reflink idref="bib54" id="ref10">54</reflink>]; Shao et al., [<reflink idref="bib66" id="ref11">66</reflink>]). Specifically, LLMs can infer latent cognitive processes embedded in learning behaviour data, addressing key limitations of traditional think‐aloud methods. By incorporating authentic cognitive processes, LLMs can assist ITS in generating realistic simulated students that accurately reflect diverse learning trajectories and provide adaptive feedback tailored to individual learners (Stamper et al., [<reflink idref="bib70" id="ref12">70</reflink>]). This capacity further enhances the flexibility and effectiveness of ITS in supporting diverse learners by addressing equity challenges and optimizing personalized instruction (Chen et al., [<reflink idref="bib13" id="ref13">13</reflink>]; Park et al., [<reflink idref="bib55" id="ref14">55</reflink>]). Furthermore, in teacher education, LLM‐enhanced think‐aloud protocols provide preservice teachers with valuable tools to engage with simulated students exhibiting realistic cognitive pathways. These simulations enable teachers to identify cognitive obstacles and adapt instructional strategies through iterative dialogues (Frei‐Landau & Levin, [<reflink idref="bib26" id="ref15">26</reflink>]). Furthermore, LLMs can replicate behaviours such as errors derived from symbolic working memory, offering authentic training scenarios for preservice teachers (Ding, [<reflink idref="bib19" id="ref16">19</reflink>]). The inclusion of emotional states, such as curiosity or frustration, further enhances the realism of these simulations, preparing teachers for nuanced, real‐world classroom dynamics (Park et al., [<reflink idref="bib54" id="ref17">54</reflink>]; Sallam, [<reflink idref="bib64" id="ref18">64</reflink>]).</p> <p>The method proposed in this study, referred to as Cognitive Echo, extends traditional think‐aloud protocols by deriving learners' cognitive processes from retrospective behavioural data. Conventional think‐aloud protocols often require participants to verbalize their thoughts synchronously during problem‐solving tasks. In contrast, Cognitive Echo enables the creation of simulated students who can verbalize thought processes informed by authentic learning experiences. This post hoc approach is particularly effective in capturing the cognitive processes of diverse learners as they solve problems. By addressing the challenges of data completeness and cognitive load (Davis & Bistodeau, [<reflink idref="bib18" id="ref19">18</reflink>]), Cognitive Echo contributes to a more reliable and comprehensive understanding of cognitive processes. To achieve this, a trainable simulated student framework is developed. Unlike conventional prompt‐engineering approaches based on LLMs, this framework incrementally transforms authentic learning records into simulated students' learning experiences. The proposed framework not only advances the methodological rigour of think‐aloud protocols but also provides a foundation for modelling complex educational problems, such as those addressed in digital twin environments.</p> <hd id="AN0187257455-3">RELATED WORKS</hd> <p></p> <hd id="AN0187257455-4">Challenges of think‐loud protocol</hd> <p>The think‐aloud protocol is a research method designed to capture human cognitive data by requiring participants to verbalize their thought processes while performing tasks or solving problems (Fonteyn et al., [<reflink idref="bib24" id="ref20">24</reflink>]). This approach enables researchers to gain insights into participants' working memory processes, providing a deeper understanding of cognitive flow and problem‐solving strategies (Ericsson & Simon, [<reflink idref="bib22" id="ref21">22</reflink>]). This method offers a unique window into cognitive processes, particularly for parallel reasoning involving verbal representations in working memory. The concept aligns with Vygotsky's ([<reflink idref="bib76" id="ref22">76</reflink>]) idea of 'inner speech', which evolves from children's egocentric speech and supports problem solving. Ericsson and Simon ([<reflink idref="bib22" id="ref23">22</reflink>]) further distinguished working memory from long‐term memory, emphasizing the protocol's focus on immediate cognitive flow.</p> <p>The think‐aloud protocol has been employed across various domains that investigate cognitive processes. Its applications include capturing users' perceptions of usability in user experience research (Alshammari et al., [<reflink idref="bib1" id="ref24">1</reflink>]; Doi, [<reflink idref="bib20" id="ref25">20</reflink>]) and exploring decision‐making cues (Kaminski & Sporer, [<reflink idref="bib34" id="ref26">34</reflink>]). In education, the think‐aloud protocol is utilized to observe how students solve problems (Overton et al., [<reflink idref="bib53" id="ref27">53</reflink>]), self‐regulated learning and reflect on learning strategies (Heirweg et al., [<reflink idref="bib28" id="ref28">28</reflink>]). By examining these cognitive aspects, researchers can gain insights into students' metacognitive awareness, enabling them to identify factors that facilitate effective learning (Jacobse & Harskamp, [<reflink idref="bib32" id="ref29">32</reflink>]) and refine instructional strategies accordingly (Beach & Willows, [<reflink idref="bib5" id="ref30">5</reflink>]).</p> <p>The primary value of the think‐aloud protocol lies in its ability to provide researchers with a unique perspective into working memory, which is vital for assessing practical competencies beyond merely observable behaviours (Eccles & Arsal, [<reflink idref="bib21" id="ref31">21</reflink>]). Its real‐time nature allows for an authentic representation of individuals' thought and decision‐making processes, often eluding retrospective analyses (Pike et al., [<reflink idref="bib57" id="ref32">57</reflink>]). However, one of the most prominent concerns is the completeness of the participants' self‐reports. Participants may unconsciously filter their thoughts, resulting in incomplete data. This issue arises when subtle cognitive processes or those perceived as irrelevant during the task are omitted from verbal reports (Charters, [<reflink idref="bib11" id="ref33">11</reflink>]). Additionally, because not all cognitive processes are active within working memory, significant aspects of cognition may remain unreported (Roth et al., [<reflink idref="bib62" id="ref34">62</reflink>]). Some psychological processes may also reside below the threshold of conscious awareness or prove inherently difficult to verbalize (Fox et al., [<reflink idref="bib25" id="ref35">25</reflink>]). For simpler tasks, participants may struggle to articulate behaviours that have become nearly automatic (Jacobse & Harskamp, [<reflink idref="bib32" id="ref36">32</reflink>]). Another major challenge is the potential distortion of natural cognitive processes caused by the act of verbal reporting. When participants translate non‐verbal cognitive activities into verbal expressions, their original thought patterns may be altered (Nielsen et al., [<reflink idref="bib50" id="ref37">50</reflink>]). Moreover, think‐aloud protocols have been shown to significantly increase participants' cognitive load. This additional burden can interfere with natural cognitive flow, potentially leading to artificial thought processes (Doi, [<reflink idref="bib20" id="ref38">20</reflink>]; Hori et al., [<reflink idref="bib29" id="ref39">29</reflink>]).</p> <hd id="AN0187257455-5">Simulated student</hd> <p>A simulated student is an intelligent agent developed to replicate the behaviours and cognitive processes of human learners. During the symbolic processing era of artificial intelligence, cognitive models such as the ACT‐R model relied primarily on predefined rules and conditional logic to simulate learning processes. These models laid the groundwork for early simulated student theories by illustrating how humans solve problems and make strategic decisions (Chrysafiadi & Virvou, [<reflink idref="bib15" id="ref40">15</reflink>]). For example, some ITSs incorporated a student model to infer learners' cognitive processes (Ritter et al., [<reflink idref="bib61" id="ref41">61</reflink>]). Such systems monitor students' problem‐solving steps in real time and provide immediate feedback.</p> <p>Beyond modelling students within ITS frameworks, researchers have advanced the technology to mimic real students' learning behaviours. These advancements facilitate the creation of personalized learning paths and enable the automated validation of educational interventions (Chrysafiadi & Virvou, [<reflink idref="bib15" id="ref42">15</reflink>]). Recently, some studies have utilized supervised and reinforcement learning techniques to generate high‐quality learning trajectories automatically, thereby improving the quality and diversity of simulated students (Li, Peng, et al., [<reflink idref="bib41" id="ref43">41</reflink>]). A notable example is the SimStudent system developed by Matsuda et al. ([<reflink idref="bib47" id="ref44">47</reflink>]), which employs example tracing and inductive learning to simulate learner behaviour. This system uses a two‐step process: first, human instructors demonstrate solution steps for specific problems, which the system records to establish initial rules. Second, the system refines and generalizes these rules through feedback, enabling the simulated student to tackle a broader range of problems (MacLellan et al., [<reflink idref="bib45" id="ref45">45</reflink>]; Matsuda et al., [<reflink idref="bib47" id="ref46">47</reflink>]). As an extension of the SimStudent, the Apprentice Learner model introduces a more versatile approach. It supports the simulated students across diverse knowledge domains by learning from examples and feedback (MacLellan & Koedinger, [<reflink idref="bib46" id="ref47">46</reflink>]). This model is built with modular components, including knowledge structures, learning performances and learning mechanisms. By combining these cognitive components with decision tree and incremental learning techniques, the system effectively represents the cognitive mechanisms underlying learners (MacLellan & Koedinger, [<reflink idref="bib46" id="ref48">46</reflink>]). Despite advancements in generalization capabilities and effectiveness, developing and maintaining these systems remains resource‐intensive and complex (Li, Peng, et al., [<reflink idref="bib41" id="ref49">41</reflink>]). Achieving model completeness and accuracy often requires extensive interactions and feedback, which can entail substantial time and labour investments.</p> <p>Simulated students, as a specific application of human simulacra, are deeply connected to broader research in replicating human‐like behaviours and cognitive processes. Simulacra refer to the reproduction or representation of entities that either lack an original or whose original no longer exists. This concept originally argued the indistinct boundary between reality and simulation (Baudrillard, [<reflink idref="bib4" id="ref50">4</reflink>]). Now the concept of simulacra has expanded beyond symbolic and cultural critiques to include computational simulations of human behaviours. The connection between simulated students and human simulacra lies in their shared reliance on machine learning and cognitive modelling techniques. While simulated students are tailored for educational applications like ITS (Matsuda et al., [<reflink idref="bib47" id="ref51">47</reflink>]), human simulacra encompass a broader range of goals, including the development of believable agents capable of reflecting prior interactions, synthesizing inferences and responding contextually (Park et al., [<reflink idref="bib54" id="ref52">54</reflink>]; Shao et al., [<reflink idref="bib66" id="ref53">66</reflink>]).</p> <p>In the field of artificial cognition, computer simulation techniques are extensively applied to train digital agents within computational systems (Lemaignan et al., [<reflink idref="bib38" id="ref54">38</reflink>]). The emergence of neural networks enables machines to learn and adapt in ways resembling human information processing. Agents now demonstrate human‐like behaviours, significantly enhancing human interaction experiences in digital environments. Such agents have been applied in simulation‐based learning and gaming contexts (Liu et al., [<reflink idref="bib43" id="ref55">43</reflink>]; Siemens et al., [<reflink idref="bib68" id="ref56">68</reflink>]). Recent studies have demonstrated progress in simulating higher‐order human cognitive functions, such as memory, attention and reasoning, by leveraging advanced computational frameworks and neural architectures (Lara et al., [<reflink idref="bib37" id="ref57">37</reflink>]; Park et al., [<reflink idref="bib54" id="ref58">54</reflink>]). Despite these advancements, replicating human cognition remains a significant challenge due to its inherently dynamic and evolving nature (Brooks et al., [<reflink idref="bib7" id="ref59">7</reflink>]; Van Pinxteren et al., [<reflink idref="bib75" id="ref60">75</reflink>]). More importantly, belief states encompass prior knowledge, common sense memory and mental models, all of which are essential for understanding perspectives—a capability referred to as the theory of mind (Lemaignan et al., [<reflink idref="bib38" id="ref61">38</reflink>]; Premack & Woodruff, [<reflink idref="bib59" id="ref62">59</reflink>]). The theory of mind enables entities to infer the mental states of others and reconcile conflicting knowledge. For intelligent agents, this would require constructing and managing independent knowledge models tailored to each interaction target. However, achieving such a capability poses considerable difficulties (Wimmer & Perner, [<reflink idref="bib82" id="ref63">82</reflink>]).</p> <p>Agent techniques have been widely studied, incorporating a range of methodologies such as Bayesian reasoning, finite state machines, behaviour trees, instance‐based learning, symbolic data retrieval and hard‐coded expert models (Lemaignan et al., [<reflink idref="bib38" id="ref64">38</reflink>]; Park et al., [<reflink idref="bib54" id="ref65">54</reflink>]). However, these approaches often simplify simulation environments to enhance the feasibility of their implementation. Such methods lack the capacity to enable agents to acquire new behaviours or develop novel cognitive processes (Laird, [<reflink idref="bib36" id="ref66">36</reflink>]; Park et al., [<reflink idref="bib54" id="ref67">54</reflink>]). To address these limitations, researchers have developed reliable human‐simulating agents. These agents are envisioned to possess the ability to retrieve relevant information over extended periods, recall prior interactions, reflect on past experiences and synthesize higher‐order inferences. Additionally, they would abstract generalizable insights and apply them to formulate plans and produce contextually appropriate responses (Park et al., [<reflink idref="bib54" id="ref68">54</reflink>]; Shao et al., [<reflink idref="bib66" id="ref69">66</reflink>]). Such advancements would foster the creation of agents capable of delivering a convincing illusion of life‐like presence, thereby enhancing the potential for immersive and authentic human‐agent interactions (Bates, [<reflink idref="bib3" id="ref70">3</reflink>]).</p> <hd id="AN0187257455-6">Trainable character agent</hd> <p>LLMs have garnered significant attention in recent years for their ability to capture and generate diverse linguistic patterns (Chang et al., [<reflink idref="bib10" id="ref71">10</reflink>]). The most direct application of LLMs lies in prompt engineering, a technique that focuses the model's outputs on specific tasks or contexts. This approach has been applied in social science studies, such as real‐time discourse coding (Zhang et al., [<reflink idref="bib88" id="ref72">88</reflink>]) and generating digital learning resources from demographic data (Xu et al., [<reflink idref="bib84" id="ref73">84</reflink>]; Xu & Zhang, [<reflink idref="bib83" id="ref74">83</reflink>]). However, the effectiveness of this approach is constrained by the limited size of the model's information window. This limitation can lead to outputs that are overly focused on immediate contexts, lacking validation against broader or prior knowledge, which in turn introduces hallucinations or inconsistencies in logical reasoning (Brown et al., [<reflink idref="bib8" id="ref75">8</reflink>]).</p> <p>Some researchers have proposed using LLMs as intelligent agents to simulate human behaviours and characteristics (Park et al., [<reflink idref="bib54" id="ref76">54</reflink>]; Shao et al., [<reflink idref="bib66" id="ref77">66</reflink>]; Tu et al., [<reflink idref="bib73" id="ref78">73</reflink>]). They advocated for supervised fine‐tuning as a more effective method for character simulation compared with prompt engineering (Li, Hammoud, et al., [<reflink idref="bib40" id="ref79">40</reflink>]; Park et al., [<reflink idref="bib54" id="ref80">54</reflink>]; Shao et al., [<reflink idref="bib66" id="ref81">66</reflink>]; Wang, Xiao, et al., [<reflink idref="bib80" id="ref82">80</reflink>]). Even when trained on small sample datasets, fine‐tuned models demonstrate superior comprehension of complex linguistic structures and contextual relationships for targeted tasks (Brown et al., [<reflink idref="bib8" id="ref83">8</reflink>]). Fine‐tuning enables LLMs to transform general‐purpose models into trainable agents capable of adopting human roles, akin to training an actor to embody a character. Shao et al. ([<reflink idref="bib66" id="ref84">66</reflink>]) proposed an experiential reconstruction and uploading framework for role simulation. This framework allows LLMs to emulate the behaviours and emotions of historical figures or fictional characters. This framework reproduces specific personality traits, experiences and emotional states, making it a robust tool for role‐playing simulations. This capability holds particular significance in the social sciences, where it enables researchers to construct customized experimental environments. Such experimental setups function as a sandbox, providing researchers with the flexibility to conduct experiments (Ruan et al., [<reflink idref="bib63" id="ref85">63</reflink>]).</p> <p>Building on their ability to simulate human behaviours, fine‐tuned LLMs offer significant potential in teacher education by accurately modelling diverse student cognitive processes and providing realistic, consistent pathways for analysis and interaction (Zheng et al., [<reflink idref="bib92" id="ref86">92</reflink>]). Preservice teachers can engage in iterative dialogues with these simulated students to identify cognitive obstacles and refine their teaching strategies (Frei‐Landau & Levin, [<reflink idref="bib26" id="ref87">26</reflink>]). Unlike rule‐based systems, fine‐tuned LLMs derive errors from symbolic working memory and prior experiences, enabling authentic learning interactions (Ding, [<reflink idref="bib19" id="ref88">19</reflink>]). Supported by Cognitive Echo, LLMs can further enhance the realism of simulating environments by replicating emotional states (Park et al., [<reflink idref="bib54" id="ref89">54</reflink>]; Sallam, [<reflink idref="bib64" id="ref90">64</reflink>]). Furthermore, simulations enhanced by LLMs provide a solid foundation for developing digital twins of educational scenarios. These digital twin environments enable the modelling of complex educational challenges, such as school bullying and learning disabilities, by leveraging LLM‐powered simulations (Tagliabue et al., [<reflink idref="bib72" id="ref91">72</reflink>]). Such realistic settings offer preservice teachers valuable opportunities to engage with simulated classroom dynamics, deepening their understanding of student cognition and better equipping them to navigate intricate real‐world challenges.</p> <p>This study seeks to integrate think‐aloud protocols with role‐simulation techniques powered by LLMs to enhance the transparency of human cognitive processes. This integration aims to improve the reliability of cognitive data collection from students. By verbalizing their simulated thought processes, these student models are capable of emulating human learners' cognitive processes, providing researchers with valuable insights into both observable behaviours and underlying cognitive mechanisms. Since the purpose of think‐aloud is to compare the cognitive differences across learners, this work also needs to verify that this method allows LLM to mimic the cognition of different students.</p> <hd id="AN0187257455-7">RESEARCH DESIGN</hd> <p></p> <hd id="AN0187257455-8">Research questions</hd> <p>This study explores enhancing think‐aloud protocols through the use of LLMs to improve the completeness and authenticity of verbal reports. It employs LLMs trained with simulated students characterized by specific learner traits and experiences, thus aiding observers in comprehensively reviewing cognitive processes and perceptions of knowledge acquisition. The research addresses the following two questions:</p> <p></p> <ulist> <item> How can the Cognitive Echo method, leveraging LLM‐trained simulated students, be designed and optimized to enhance the effectiveness of think‐aloud protocols?</item> <p></p> <item> To what extent does training LLMs with transferred learning experiences improve their capacity to simulate distinct human‐like cognition?</item> </ulist> <hd id="AN0187257455-9">Addressing RQ1: Designing the Cognitive Echo method</hd> <p>Figure 1 illustrates the key steps involved in implementing the Cognitive Echo method, from data collection to the fine‐tuning of LLMs and their evaluation. The specific prompts for training procedures are detailed in Appendix S4. The GPT used in Figure 1 refers to OpenAI's GPT‐4. This model was selected as the primary data generation source due to its availability as a stable, closed‐source LLM (Shao et al., [<reflink idref="bib66" id="ref92">66</reflink>]). While GPT‐4o, a continuously updated variant, shares similar ability with GPT‐4, OpenAI ([<reflink idref="bib52" id="ref93">52</reflink>]) notes that its output stability may be slightly inferior to GPT‐4. Given the need for consistent and reliable data generation, GPT‐4 was deemed more appropriate for this study.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/58I/01sep25/bjet13590-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="bjet13590-fig-0001.jpg" title="1 Steps of the Cognitive Echo method." /> </p> <p></p> <hd id="AN0187257455-11">Collecting learning records</hd> <p>The original learning records were obtained from the transactions of ITS in the DataShop repository (Koedinger et al., [<reflink idref="bib35" id="ref94">35</reflink>]). DataShop is a widely used repository that serves educational data mining. Researchers extracted learning records from DataShop, which include attempts, hints requested, inputs and meta‐information related to each action. These collected records comprehensively capture detailed contextual descriptions of learning and reflect the authentic process of how they solve problems.</p> <p>The records used in this study come from the public dataset, 'Fractions Lab Experiment 2012', which contains authentic data from a mathematics ITS in 2012 and 2013. Due to the limited learning data from individual students, researchers aggregated students of similar proficiency levels to create a representative cohort. Accordingly, students were classified into three performance groups based on their error rates: limited, moderate and advanced. A total of 2742 samples were included in this experiment, with each sample representing all the learning records of one human student on a specific task. Furthermore, due to the fact that some learners may not have engaged in substantial learning during their interactions with the ITS (ie, their learning records were very limited, with only one or two attempts before they either solved or abandoned the problem), this could potentially affect the quality of the data. To address the data volume and diversity, researchers utilized the GPT‐4 interface to enrich 95 samples (representing 3.5% of the total) with simulated learning records. This simulation generated data that closely resembled the authentic data. To guarantee the validity and consistency of the generated content, the knowledge scope of the synthetic data was constrained to the real learning records data of students in the same proficiency groups. Additionally, the researchers conducted a thorough review of these samples to verify their accuracy and relevance. Specifically, researchers used OpenAI's GPT‐4 model for generation, selecting the default temperature value. Prompts can be found in Appendix S4.</p> <hd id="AN0187257455-12">Inferencing tutoring scenes</hd> <p>This study requires inferring tutoring scenarios to simulate instructional activities. The researchers deduced corresponding tutoring scenarios based on content and the design of ITS. Tutoring content corresponding to the learning records was obtained from TutorShop, a repository for ITS. The researchers organized the relevant information, including problem content, answers, hints, expert solution pathways, knowledge components and problem metadata. Initially, they extracted or identified this information from the tutor package downloaded from TutorShop. Subsequently, by prompting GPT‐4, the researchers reconstructed the tutoring scenarios for the selected learning records. Prompts and examples of generated scenes can be found in Appendix S4.</p> <p>It is important to note that only a few of the tutors corresponding to the learning records can be retrieved from TutorShop. The retrievable tutors were selectively paired with their learning records and included as samples in the prompts for GPT‐4 to facilitate small sample learning so that inferring the unretrievable tutoring scenarios. Then, the unretrievable tutors and their contextual information can be predicted according to corresponding learning records.</p> <hd id="AN0187257455-13">Reconstructing learning experience</hd> <p>The researchers utilize GPT‐4 to create learning experiences that present teacher–student interactions and student's reflections based on learning records and tutoring scenarios. These experiences were designed in a script‐like format, starting with a scene title that contextualizes the addressed problems. Each scene features roles for both the teacher and the student. The scripted interactions simulate multi‐turn conversations where the teacher guides the student through problem solving, drawing on student–tutor interactions.</p> <p>In this script, the teacher is represented by a personified agent of the ITS, responsible for introducing background knowledge, presenting problem content, emphasizing hints, guiding problem‐solving strategies and providing hints. The student responds with answers that reflect their problem‐solving paths, input attempts, corrections and hint requests as recorded in the real interactions within ITSs. Moreover, the student explicitly articulates reflections on their learning process. For example, when a student corrects an initial error after multiple attempts, they might reflect on the correction process by comparing their previous answers. The reflections expressed by the student are inferred from the input behaviour sequences of real ITS learners, particularly when students modify their input behaviours after identifying errors or seeking help. These reflections are constructed by analysing patterns in the students' reattempts to answer correctly, which serve as a proxy for reflective thinking. This approach is inspired by the work of Shao et al. ([<reflink idref="bib66" id="ref95">66</reflink>]), who introduced inner monologue lines to simulate reflective processes in agent‐based role‐playing models. To generate these learning experiences, the researchers used OpenAI's GPT‐4 model, selecting the default temperature value. Prompts and generated examples can be found in Appendix S4. A total of 2742 scripts of learning experience were created.</p> <p>To align LLM‐generated responses with different proficiency levels, additional constraints were incorporated during training. For instance, to ensure that limited‐performing students exhibit expected deficiencies, the researchers trained LLMs to demonstrate incomplete problem‐solving capabilities on specific knowledge components. Scenarios were designed to depict how low‐performing students struggle with certain problems, such as making repeated errors, failing to grasp teacher explanations or frequently seeking help. These experiences were encoded into the model via fine‐tuning, enabling the student to remember its past challenges. As a result, the model generates responses that accurately reflect the learning limitations of underperforming students when encountering similar problems.</p> <p>To ensure the quality of the generated learning experiences, the researchers randomly selected 274 samples (10% of the total) for manual inspection. Two reviewers independently examined these samples to identify any significant hallucinations, specifically instances where the generated learning experiences conflicted with authentic learning records. During this process, each reviewer cross‐checked the generated scripts against corresponding learner records and problem contexts in the ITS. When a reviewer identified a potential hallucination instance, it was explicitly flagged for further discussion and validation. For the purpose of this manual inspection, hallucination instances referred specifically to factual inaccuracies, illogical reasoning patterns or deviations from recorded learner interactions obtained from transactions in the DataShop repository. In cases where reviewers disagreed on the validity of a script, a discussion was conducted to reach consensus before inclusion. Only when both reviewers confirmed the validity of the sampled experiences was the entire dataset included for further analysis. During this inspection, all sampled scripts passed the manual review and were subsequently incorporated into the dataset.</p> <hd id="AN0187257455-14">Supplementary protective memory</hd> <p>The researchers incorporated protective experiences to address the issue of hallucination in simulated character. LLMs' broad knowledge can undermine the credibility of the simulated students, as the models may inadvertently express information that is inconsistent with the students' levels, leading to a sense of discord. To mitigate the risk of simulated students inheriting inappropriate knowledge from the LLMs, the researchers adopted the protective experience approach proposed by Shao et al. ([<reflink idref="bib66" id="ref96">66</reflink>]) to facilitate the forgetting of unreasonable knowledge.</p> <p>Specifically, the researchers designed a series of protective scenarios in which a teacher continuously posed questions to the simulated students that were significantly above their current knowledge levels. In response, the simulated students exhibited a lack of knowledge or confusion, refraining from providing accurate or relevant answers. After training through these protective experiences, when faced with unreasonable questions, the simulated students would either refuse to answer or provide incorrect responses.</p> <hd id="AN0187257455-15">Uploading experience</hd> <p>The researchers developed a dataset comprising 2850 teaching scenarios to facilitate model training. Using fine‐tuning techniques, they trained the Qwen2‐7B model (Yang et al., [<reflink idref="bib86" id="ref97">86</reflink>]) to simulate three distinct student profiles, classified based on varying learning performance. Qwen2‐7B was selected due to its status as one of the most commonly used small‐scale open‐source LLMs in studies focusing on agent‐based simulations (eg, Shen et al., [<reflink idref="bib67" id="ref98">67</reflink>]; Wang, Lian, et al., [<reflink idref="bib78" id="ref99">78</reflink>]; Wang, Zhang, et al., [<reflink idref="bib79" id="ref100">79</reflink>]). Furthermore, these studies have demonstrated that Qwen2‐7B outperforms other LLMs of a similar parameter scale, particularly in tasks requiring fine‐tuned role‐based adaptations. Each simulated student was programmed to interact with 914 tutoring scenarios and 36 protective scenarios. To enhance scalability and reduce the need for repetitive fine‐tuning across different learner profiles, parameter‐efficient fine‐tuning techniques such as Low‐Rank Adaptation (LoRA) (Hu et al., [<reflink idref="bib30" id="ref101">30</reflink>]) were implemented. This approach allows the model to retain shared knowledge while adapting minimal task‐specific parameters for distinct learner levels, thereby optimizing computational efficiency. The detailed hyper‐parameter configuration for model fine‐tuning is provided in Appendix S1.</p> <hd id="AN0187257455-16">Addressing RQ2: Evaluating the simulation via think‐aloud</hd> <p>Researchers constructed a simulated think‐aloud environment to evaluate the performance of the simulated students. The students' capabilities were assessed through single‐turn or multi‐turn interviews. The interview questions were designed to be similar to, but not identical to, the training data. This study employed one of the most advanced LLMs as a judge. As in most previous studies, GPT‐4 was selected as the evaluator. To ensure the validity of the evaluation, two human reviewers were included alongside GPT‐4 to collaboratively assess the responses of the simulated students across four dimensions. The trained simulated students were compared with non‐fine‐tuned models, including Qwen2‐7B, Vicuna 1.5‐7B (Chiang et al., [<reflink idref="bib14" id="ref102">14</reflink>]) and GPT‐3.5.</p> <hd id="AN0187257455-17">Metrics</hd> <p></p> <hd id="AN0187257455-18">Cognitive Completeness</hd> <p>Cognitive Completeness measures the extent to which the responses from simulated students reflect the cognitive processes and reasoning necessary for expressing thought during problem solving. This metric emphasizes the comprehensiveness of the solution space during the think‐aloud process. Prior evaluations of problem‐solving steps have focused on aspects such as the structure of cognitive architectures (Anderson et al., [<reflink idref="bib2" id="ref103">2</reflink>]), the effective management of cognitive load (Sweller et al., [<reflink idref="bib71" id="ref104">71</reflink>]), the coordination of different types of knowledge (Jonassen, [<reflink idref="bib33" id="ref105">33</reflink>]) and key behaviours that promote the transfer of problem solving, such as reflection and regulation (Phye, [<reflink idref="bib56" id="ref106">56</reflink>]). The framework of this study builds upon Schoenfeld's ([<reflink idref="bib65" id="ref107">65</reflink>]) dimensions of problem solving (resources, heuristics, control and beliefs) and Pólya's ([<reflink idref="bib58" id="ref108">58</reflink>]) four‐step model (problem understanding, strategy formulation, execution and reflection). A total of seven indicators were considered. To operationalize Cognitive Completeness, seven indicators and their rubrics were developed (see Appendices S2 and S3). These indicators encompass: resources identification, understanding the problem, devising a plan, strategy execution, evaluation of the result, reflection and regulation.</p> <hd id="AN0187257455-19">Strategy Rationale</hd> <p>Strategy Rationale evaluates the appropriateness and effectiveness of cognitive strategies applied by simulated students in specific tasks. This metric considers how problem solving involves navigating a problem space using operators that transform the current state into a solution, emphasizing the importance of selecting appropriate spaces and operators (Newell & Simon, [<reflink idref="bib49" id="ref109">49</reflink>]). The orchestration of fundamental cognitive processes, such as abstraction, searching, heuristics and analogy, is considered crucial for performing problem solving as a higher‐layer cognitive process (Wang & Chiew, [<reflink idref="bib77" id="ref110">77</reflink>]). This study emphasizes how effectively problem‐solving strategies are tailored by simulated students to meet task‐specific requirements and how they enhance both logic and efficiency. The nine evaluation criteria are derived from the strategic frameworks for problem solving outlined by Schoenfeld's ([<reflink idref="bib65" id="ref111">65</reflink>]) dimensions of problem solving and Pólya's ([<reflink idref="bib58" id="ref112">58</reflink>]) four‐step model. A total of seven indicators were developed: problem decomposition, diagram drawing, sub‐hypothesis establishing, heuristic reasoning, self‐monitoring and strategy transfer. Additionally, two ITS‐specific behaviour indicators were incorporated: seeking help and enhancing confidence through positive feedback. Detailed sub‐indicators and rubrics are provided in Appendices S2 and S3.</p> <hd id="AN0187257455-20">Mastery Accuracy</hd> <p>Mastery Accuracy assesses whether the knowledge mastered by simulated students aligns with their expected knowledge levels. The rationale for this metric lies in its ability to verify the cognitive fidelity of simulated students, which is critical for ensuring their reliability as proxies for real learners in educational scenarios (Xu & Zhang, [<reflink idref="bib83" id="ref113">83</reflink>]). Evaluators use a set of validation questions, which have not been utilized for model fine‐tuning, to ascertain whether the responses from simulated students accurately represent the targeted knowledge mastery. This approach aligns with the practice of probing the intrinsic knowledge boundaries of training data in LLMs (Huang et al., [<reflink idref="bib31" id="ref114">31</reflink>]). These questions span identical scopes and levels of difficulty. The outcomes of the evaluation serve to assess the degree of alignment between the cognitive levels of the agents and the capabilities of the students they are designed to simulate.</p> <hd id="AN0187257455-21">Hallucination</hd> <p>Hallucination assessment determines whether simulated students can maintain role credibility (Shao et al., [<reflink idref="bib66" id="ref115">66</reflink>]). This metric specifically addresses the challenge of faithfulness hallucination, where LLMs confidently generate falsehoods without recognizing the limits of their own knowledge (Huang et al., [<reflink idref="bib31" id="ref116">31</reflink>]; Ren et al., [<reflink idref="bib60" id="ref117">60</reflink>]; Zhao et al., [<reflink idref="bib90" id="ref118">90</reflink>]). Previous studies have explored methods to detect such hallucinations by quantifying the uncertainty in model outputs, often through evaluating the similarity of sentences with ambiguous meanings, to determine whether LLMs can encode questions they are unable to answer (Huang et al., [<reflink idref="bib31" id="ref119">31</reflink>]; Slobodkin et al., [<reflink idref="bib69" id="ref120">69</reflink>]). In this study, hallucination assessment focuses on whether simulated students remain faithful to their predefined knowledge scope, ensuring their behaviour does not exceed these boundaries. Inspired by the work of Shao et al. ([<reflink idref="bib66" id="ref121">66</reflink>]), we examine whether simulated students can discard knowledge beyond their modelled experiences. For instance, in response to unreasonable queries, credible simulated students should refuse to answer or provide an appropriately incorrect response.</p> <hd id="AN0187257455-22">Think‐aloud questions</hd> <p>The think‐aloud questions and outlines were generated by GPT‐4 using topics not involved in the fine‐tuning process. To enhance the diversity of the question content, researchers prompted ChatGPT to produce items comparable in difficulty and knowledge to real intelligent tutors, integrating these into the evaluation tools. Each question and outline were manually reviewed and elements that deviated from the target were removed to ensure effective think‐aloud results. Ten distinct think‐aloud experimental environments were established, each containing 10 independent questions and 10 multi‐round questions. The specific prompts for evaluation procedures are detailed in Appendix S5. The response examples across different models are provided in Appendices S6 and S7.</p> <hd id="AN0187257455-23">Single question</hd> <p>Interviewers pose only one question at a time to simulated students, without considering the interview history of previous questions. By minimizing the influence of prior context, they can ask a series of questions to thoroughly explore the model's inherent memory and knowledge. Due to the lack of conversational context, the single‐question format imposes stricter demands on the model. Single questions are used to assess two metrics: Cognitive Completeness and Strategy Rationale. For each metric, four to five questions are designed based on its specific indicators. Each model is permitted to respond to each question up to five times, and all responses are independently evaluated. In addition, single‐question interviews are also employed to measure hallucination. Each model is tested with 15 questions exceeding the expected knowledge level of the student played.</p> <hd id="AN0187257455-24">Multi‐turn question</hd> <p>In multi‐turn interviews, interviewers engage in continuous dialogues with simulated students, focusing on problem solving to evaluate the agents' performance based on sustained cognitive changes. This process simulates the think‐aloud method, where follow‐up questions are used to elicit expressions of continuous working memory. This method evaluates the stability of the agents' responses across multiple related questions and reveals their logical coherence and depth of thought during extended conversations. To ensure consistency in numerous multi‐round interviews, GPT‐4 serves as the interviewer, prompted to question the simulated students based on predetermined topics and outlines. When the interaction history in multi‐round interviews exceeds token limits, earlier dialogues are pruned and summarized. The response examples from different models are provided in Appendices S7 and S8.</p> <hd id="AN0187257455-25">Evaluation steps</hd> <p>Existing research on step‐by‐step scoring methods for simulated roles guided the evaluation of simulated students' performance (Shao et al., [<reflink idref="bib66" id="ref122">66</reflink>]; Wei et al., [<reflink idref="bib81" id="ref123">81</reflink>]). The GPT‐4 model served as the evaluator, scoring the responses of simulated students across five metrics, with subsequent reviews by human evaluators to ensure reliability. Each task involved the evaluation of simulated students on a single dimension. The instructions to GPT‐4 included assessment criteria, coding frameworks, scoring rubrics and procedural steps. During the assessment, GPT‐4 was tasked with: (<reflink idref="bib1" id="ref124">1</reflink>) identifying key elements related to students' performance and the assigned task; (<reflink idref="bib2" id="ref125">2</reflink>) identifying cognitive evidence and patterns reflective of students' problem‐solving strategies and role characteristics; (<reflink idref="bib3" id="ref126">3</reflink>) comparing the performance of the simulated students against the rubric criteria; and (<reflink idref="bib4" id="ref127">4</reflink>) assigning a final score using a 7‐point Likert scale. The qualitative examples of evaluation results for different scores can be found in Appendices S8 and S9.</p> <p>Two human evaluators independently reviewed the scoring results from GPT‐4. The evaluators conducted their assessments independently and blindly, without knowing which model the outcomes originated from. All evaluation results were retained only if approved by both evaluators. In cases of disagreement, the evaluations were revised through further human assessment. If consensus could not be reached, the question and all associated evaluations were discarded to ensure consistent evaluation standards across all models. Table 1 presents the evaluation agreement rate and number of retained interviews. As shown in Table 1, the agreement rate between human evaluators was 83%, while the agreement rate between human evaluators and GPT‐4 was 80%.</p> <p>1 TABLE Evaluation agreement rate and number of retained interviews for four metrics.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Metric</th><th align="left">Interview type</th><th align="left">Agreement rate between humans (%)</th><th align="left">Agreement rate between humans and GPT‐4 (%)</th><th align="left">Number of retained questions/multi‐turn interviews</th></tr></thead><tbody valign="top"><tr><td align="left">Cognitive Completeness</td><td align="left">Single</td><td align="char" char=".">79</td><td align="char" char=".">77</td><td align="char" char=".">145</td></tr><tr><td align="left">Multi‐turn</td><td align="char" char=".">84</td><td align="char" char=".">82</td><td align="char" char=".">15</td></tr><tr><td align="left">Strategy Rationale</td><td align="left">Single</td><td align="char" char=".">88</td><td align="char" char=".">80</td><td align="char" char=".">190</td></tr><tr><td align="left">Multi‐turn</td><td align="char" char=".">83</td><td align="char" char=".">77</td><td align="char" char=".">15</td></tr><tr><td align="left">Mastery Accuracy</td><td align="left">Multi‐turn</td><td align="char" char=".">100</td><td align="char" char=".">100</td><td align="char" char=".">15</td></tr><tr><td align="left">Hallucination</td><td align="left">Single</td><td align="char" char=".">100</td><td align="char" char=".">100</td><td align="char" char=".">15</td></tr><tr><td align="left">Total</td><td align="left">–</td><td align="char" char=".">83</td><td align="char" char=".">80</td><td align="char" char=".">–</td></tr></tbody></table> </ephtml> </p> <hd id="AN0187257455-26">RESULTS</hd> <p></p> <hd id="AN0187257455-27">Enhanced Cognitive Completeness and Strategy Rationale</hd> <p>Table 2 presents the evaluations of trained simulated students compared with other untrained models across the metrics. To avoid the influence of cognitive level imitation on the results, only advanced simulated students were included in the comparison. The other baseline models were similarly prompted to simulate high‐performing students. Across both single‐turn and multi‐turn interviews, the trained model significantly outperformed the untrained baseline models, Qwen and Vicuna. This indicates that, after being trained on learning experiences, the simulated students exhibit a more comprehensive cognitive process and employ more reasonable cognitive strategies. Additionally, GPT‐3.5, with its larger parameter set, demonstrated an advantage in multi‐turn interviews and surpassed the simulated students in Cognitive Completeness.</p> <p>2 TABLE Evaluation results of models and paired sample tests.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Metric</th><th align="left">Evaluation results (SD)</th><th align="left">Paired sample tests with trained model of simulated student (T)</th></tr><tr><th align="left">Simulated student (S)</th><th align="left">Qwen (Q)</th><th align="left">Vicuna (V)</th><th align="left">GPT‐3.5 (C)</th></tr></thead><tbody valign="top"><tr><td align="left">Cognitive Completeness (single)</td><td align="char" char="(">5.34 (0.61)</td><td align="char" char="(">4.48 (0.73)</td><td align="char" char="(">3.85 (0.85)</td><td align="char" char="(">5.17 (0.64)</td><td align="left">S > Q<xref ref-type="fn" rid="tfn3" />, S > V<xref ref-type="fn" rid="tfn3" /></td></tr><tr><td align="left">Strategy Rationale (single)</td><td align="char" char="(">4.81 (0.87)</td><td align="char" char="(">4.41 (0.74)</td><td align="char" char="(">3.88 (0.84)</td><td align="char" char="(">4.74 (0.72)</td><td align="left">S > Q<xref ref-type="fn" rid="tfn3" />, S > V<xref ref-type="fn" rid="tfn3" /></td></tr><tr><td align="left">Cognitive Completeness (multi‐turn)</td><td align="char" char="(">5.34 (0.76)</td><td align="char" char="(">4.77 (1.11)</td><td align="char" char="(">3.14 (1.57)</td><td align="char" char="(">5.86 (0.60)</td><td align="left">S > Q<xref ref-type="fn" rid="tfn1" />, S > V<xref ref-type="fn" rid="tfn2" />, C > S<xref ref-type="fn" rid="tfn2" /></td></tr><tr><td align="left">Strategy Rationale (multi‐turn)</td><td align="char" char="(">4.44 (1.03)</td><td align="char" char="(">3.86 (1.29)</td><td align="char" char="(">2.46 (1.39)</td><td align="char" char="(">4.72 (1.65)</td><td align="left">S > Q<xref ref-type="fn" rid="tfn1" />, S > V<xref ref-type="fn" rid="tfn2" /></td></tr></tbody></table> </ephtml> </p> <p>1 * <emph>p</emph> < 0.05;</p> <ulist> <item>2 ** <emph>p</emph> < 0.01;</item> <item>3 *** <emph>p</emph> < 0.001.</item> </ulist> <hd id="AN0187257455-28">Regulated knowledge mastery and hallucination</hd> <p>Table 3 presents the evaluation results for Mastery Accuracy and hallucination. The findings indicate that simulated students with learning memory exhibit knowledge mastery patterns that are more similar to those of human students. Other models struggled to accurately simulate the knowledge mastery levels of low‐ability student groups. When asked questions beyond their knowledge level, the simulated students were found to either decline to answer or provide incorrect responses in most cases (67%). In contrast, the other three models demonstrated significant hallucination issues. It can be inferred that incorporating protective mechanisms has a certain effect in suppressing the capabilities of simulated student agents.</p> <p>3 TABLE Evaluation results on Mastery Accuracy and hallucination.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Student/model</th><th align="left">Mastery Accuracy (correction rate)</th><th align="left">Hallucination (the proportion of instances with no refusals or corrections)</th></tr><tr><th align="left">Limited (%)</th><th align="left">Moderate (%)</th><th align="left">Advanced (%)</th></tr></thead><tbody valign="top"><tr><td align="left">Real student</td><td align="char" char=".">5</td><td align="char" char=".">27</td><td align="char" char=".">73</td><td align="char" char=".">–</td></tr><tr><td align="left">Simulated student</td><td align="char" char=".">16</td><td align="char" char=".">42</td><td align="char" char=".">62</td><td align="char" char=".">33</td></tr><tr><td align="left">Qwen</td><td align="char" char=".">90</td><td align="char" char=".">77</td><td align="char" char=".">67</td><td align="char" char=".">100</td></tr><tr><td align="left">Vicuna</td><td align="char" char=".">71</td><td align="char" char=".">45</td><td align="char" char=".">53</td><td align="char" char=".">60</td></tr><tr><td align="left">GPT‐3.5</td><td align="char" char=".">70</td><td align="char" char=".">77</td><td align="char" char=".">74</td><td align="char" char=".">100</td></tr></tbody></table> </ephtml> </p> <hd id="AN0187257455-29">Differentiation in cognitive performance</hd> <p>Table 4 presents the evaluation results of models simulating different knowledge levels on cognitive metrics. In both single‐ and multi‐turn interviews, the pre‐trained models (Qwen and Vicuna) demonstrated non‐monotonic or non‐significant differences in cognitive performance across various levels, indicating their inability to understand the differences in cognitive integrity and cognitive strategies among students with varying knowledge levels. In contrast, the trained simulated students exhibited monotonic changes, with significant differences observed between most adjacent levels. This suggests that the simulated students externalized differentiated cognitive performance through their own memory. In single‐question interviews, GPT‐3.5 simulated students with stronger differentiation across knowledge levels. However, in more in‐depth multi‐round interviews, only the trained simulated students maintained significant differentiation. This highlights that simulated students trained with memory‐based learning experiences demonstrated greater consistency and stability.</p> <p>4 TABLE One‐way ANOVA tests on evaluation of models across three levels of students.</p> <p> <ephtml> <table><thead valign="bottom"><tr><th align="left">Metric</th><th align="left">Model</th><th align="left">Evaluation on seven Likert scale across three levels of student (SD)</th><th align="left"><italic>F</italic></th><th align="left">Partial ETA square</th><th align="left">Pairwise comparison</th></tr><tr><th align="left">Limited (L)</th><th align="left">Moderate (M)</th><th align="left">Advanced (A)</th></tr></thead><tbody valign="top"><tr><td align="left">Cognitive Completeness (single)</td><td align="left">Simulated student</td><td align="char" char="(">4.80 (0.80)</td><td align="char" char="(">4.98 (0.67)</td><td align="char" char="(">5.34 (0.61)</td><td align="left">4.58<xref ref-type="fn" rid="tfn4" /></td><td align="left">0.05</td><td align="left">A > L<xref ref-type="fn" rid="tfn6" />A > M<xref ref-type="fn" rid="tfn4" /></td></tr><tr><td align="left">Qwen</td><td align="char" char="(">4.78 (0.54)</td><td align="char" char="(">5.07 (0.64)</td><td align="char" char="(">4.48 (0.73)</td><td align="left">6.00<xref ref-type="fn" rid="tfn5" /></td><td align="left">0.07</td><td align="left">M > L<xref ref-type="fn" rid="tfn6" />M > A<xref ref-type="fn" rid="tfn4" />L > A<xref ref-type="fn" rid="tfn4" /></td></tr><tr><td align="left">Vicuna</td><td align="char" char="(">3.43 (0.65)</td><td align="char" char="(">3.95 (0.68)</td><td align="char" char="(">3.85 (0.85)</td><td align="left">4.18<xref ref-type="fn" rid="tfn4" /></td><td align="left">0.02</td><td align="left">M > L<xref ref-type="fn" rid="tfn6" />A > L<xref ref-type="fn" rid="tfn4" /></td></tr><tr><td align="left">GPT‐3.5</td><td align="char" char="(">4.06 (0.87)</td><td align="char" char="(">4.79 (0.66)</td><td align="char" char="(">5.17 (0.64)</td><td align="left">17.09<xref ref-type="fn" rid="tfn6" /></td><td align="left">0.20</td><td align="left">M > L<xref ref-type="fn" rid="tfn6" />A > L <xref ref-type="fn" rid="tfn6" />A > M <xref ref-type="fn" rid="tfn5" /></td></tr><tr><td align="left">Strategy Rationale (single)</td><td align="left">Simulated student</td><td align="char" char="(">4.23 (0.69)</td><td align="char" char="(">4.56 (0.78)</td><td align="char" char="(">4.80 (0.87)</td><td align="left">5.12<xref ref-type="fn" rid="tfn5" /></td><td align="left">0.05</td><td align="left">M > L<xref ref-type="fn" rid="tfn5" />A > L<xref ref-type="fn" rid="tfn6" />A > M<xref ref-type="fn" rid="tfn4" /></td></tr><tr><td align="left">Qwen</td><td align="char" char="(">4.23 (0.91)</td><td align="char" char="(">4.20 (1.02)</td><td align="char" char="(">4.41 (0.74)</td><td align="left">0.58</td><td align="left">0.01</td><td align="left">–</td></tr><tr><td align="left">Vicuna</td><td align="char" char="(">3.25 (0.95)</td><td align="char" char="(">3.78 (0.87)</td><td align="char" char="(">3.88 (0.84)</td><td align="left">5.49<xref ref-type="fn" rid="tfn5" /></td><td align="left">0.05</td><td align="left">M > L<xref ref-type="fn" rid="tfn6" />A > L<xref ref-type="fn" rid="tfn6" /></td></tr><tr><td align="left">GPT‐3.5</td><td align="char" char="(">3.71 (1.00)</td><td align="char" char="(">4.31 (0.89)</td><td align="char" char="(">4.74 (0.73)</td><td align="left">13.30<xref ref-type="fn" rid="tfn6" /></td><td align="left">0.12</td><td align="left">M > L<xref ref-type="fn" rid="tfn6" />A > L<xref ref-type="fn" rid="tfn6" />A > M<xref ref-type="fn" rid="tfn6" /></td></tr><tr><td align="left">Cognitive Completeness (multi)</td><td align="left">Simulated student</td><td align="char" char="(">4.12 (1.70)</td><td align="char" char="(">4.80 (0.99)</td><td align="char" char="(">5.34 (0.76)</td><td align="left">8.80<xref ref-type="fn" rid="tfn5" /></td><td align="left">0.15</td><td align="left">A > L<xref ref-type="fn" rid="tfn5" /></td></tr><tr><td align="left">Qwen</td><td align="char" char="(">4.35 (1.43)</td><td align="char" char="(">4.54 (1.07)</td><td align="char" char="(">4.77 (1.12)</td><td align="left">1.03</td><td align="left">0.02</td><td align="left">–</td></tr><tr><td align="left">Vicuna</td><td align="char" char="(">3.35 (1.45)</td><td align="char" char="(">2.71 (1.10)</td><td align="char" char="(">3.14 (1.57)</td><td align="left">1.90</td><td align="left">0.04</td><td align="left">–</td></tr><tr><td align="left">GPT‐3.5</td><td align="char" char="(">5.74 (0.83)</td><td align="char" char="(">5.77 (0.49)</td><td align="char" char="(">5.85 (0.60)</td><td align="left">0.32</td><td align="left">0.01</td><td align="left">–</td></tr><tr><td align="left">Strategy Rationale (multi)</td><td align="left">Simulated student</td><td align="char" char="(">3.50 (1.32)</td><td align="char" char="(">3.52 (1.71)</td><td align="char" char="(">4.44 (1.03)</td><td align="left">7.48<xref ref-type="fn" rid="tfn5" /></td><td align="left">0.09</td><td align="left">A > L<xref ref-type="fn" rid="tfn4" />A > M<xref ref-type="fn" rid="tfn4" /></td></tr><tr><td align="left">Qwen</td><td align="char" char="(">3.34 (1.34)</td><td align="char" char="(">3.86 (1.34)</td><td align="char" char="(">3.86 (1.29)</td><td align="left">2.55</td><td align="left">0.03</td><td align="left">–</td></tr><tr><td align="left">Vicuna</td><td align="char" char="(">2.68 (1.33)</td><td align="char" char="(">2.10 (0.91)</td><td align="char" char="(">2.46 (1.39)</td><td align="left">2.84</td><td align="left">0.04</td><td align="left">–</td></tr><tr><td align="left">GPT‐3.5</td><td align="char" char="(">4.66 (1.39)</td><td align="char" char="(">4.70 (1.30)</td><td align="char" char="(">4.72 (1.65)</td><td align="left">0.02</td><td align="left">0.00</td><td align="left">–</td></tr></tbody></table> </ephtml> </p> <ulist> <item>4 * <emph>p</emph> < 0.05;</item> <item>5 ** <emph>p</emph> < 0.01;</item> <item>6 *** <emph>p</emph> < 0.001.</item> </ulist> <hd id="AN0187257455-30">DISCUSSION</hd> <p></p> <hd id="AN0187257455-31">Enhancing the think‐aloud protocols</hd> <p>This study proposes Cognitive Echo as a potential solution to address the challenges associated with think‐aloud protocols. The evaluation results indicate that LLMs can enhance the imitation of human learners' cognitive processes by integrating learning experiences. This approach has the potential to improve the traditional think‐aloud method by minimizing interference and enabling the collection of cognitive data without constraints.</p> <p>Unlike conventional protocols that rely on real‐time verbalization, Cognitive Echo addresses this issue by leveraging simulated students to extract learning experiences from pre‐existing data, thereby eliminating interference with learners' natural cognitive flow (Doi, [<reflink idref="bib20" id="ref128">20</reflink>]; Nielsen et al., [<reflink idref="bib50" id="ref129">50</reflink>]). Instead of placing an additional cognitive burden on participants, simulated students extract reflective content from learning records. This facilitates the investigation of working memory processes without interrupting task engagement or increasing cognitive load (Hori et al., [<reflink idref="bib29" id="ref130">29</reflink>]).</p> <p>Additionally, the Cognitive Echo can improve data completeness by overcoming the temporal and spatial constraints inherent in think‐aloud protocol. Without synchronous requirement, the asynchronous Cognitive Echo allows researchers to capture cognitive processes without requiring predefined data collection strategies. It allows researchers to retrospectively analyse complex cognitive transitions and implicit processes that participants may not consciously verbalize (Fox et al., [<reflink idref="bib25" id="ref131">25</reflink>]; Roth et al., [<reflink idref="bib62" id="ref132">62</reflink>]). As a result, Cognitive Echo directly addresses longstanding concerns about data omissions and task distortion in think‐aloud protocols, providing a more comprehensive and authentic account of learners' cognition.</p> <hd id="AN0187257455-32">Transferring different human cognition into LLMs</hd> <p>The findings indicate that uploading distinct learning experiences for fine‐tuning LLMs can effectively transfer learners' cognitive characteristics. Evaluations of different models further demonstrate that this method surpasses the effectiveness of prompt‐based output generation. Since the uploaded experiences are based on authentic learning processes, this approach ensures that the outputs of LLMs consistently align with typical human‐centred behaviours. It also reduces the randomness and unpredictability often observed when LLMs are applied to educational tasks. Findings from multiple‐turn interviews suggest that Cognitive Echo enables the models to exhibit stable cognitive variability and consistent linguistic outputs across dialogues. Future applications of this method may enhance the technological readiness of LLMs for educational purposes (Yan et al., [<reflink idref="bib85" id="ref133">85</reflink>]).</p> <p>While the pre‐training objectives of LLMs typically focus on minimizing contextual word prediction errors (Zhang et al., [<reflink idref="bib89" id="ref134">89</reflink>]), the Cognitive Echo method adjusts these objectives by integrating authentic educational data, aligning them more closely with educational goals. This realignment enables the outputs to better reflect distinct human cognitive patterns and domain‐specific knowledge (Park et al., [<reflink idref="bib54" id="ref135">54</reflink>]). Consequently, simulated students prioritize not only knowledge accuracy but also the inclusion of reasoning processes and reflective content tailored to representative learners' characteristics. This adaptation enhances the educational utility of LLM outputs by enabling simulated students to understand human participants' perspectives more comprehensively, rather than relying solely on structured examples or abstract rules (Lemaignan et al., [<reflink idref="bib38" id="ref136">38</reflink>]). This capability facilitates the delivery of personalized feedback that mirrors the distinct characteristics of various students. Such personalization is vital for helping educators address educational equity, as it demonstrates how diverse teaching strategies can adapt to different students (Frei‐Landau & Levin, [<reflink idref="bib26" id="ref137">26</reflink>]).</p> <p>Research on simulation‐based learning suggests that when learners are able to suspend disbelief and immerse themselves fully in a learning scenario, their outcomes are significantly enhanced (Muckler, [<reflink idref="bib48" id="ref138">48</reflink>]). Within the Cognitive Echo method, LLMs function as simulated students and employ highly realistic role simulations. These realistic interactions help human participants achieve suspension of disbelief, allowing them to engage more deeply with learning tasks. The cognition transfer approach proposed in this study enables LLMs to embody distinct student archetypes, fostering suspension of disbelief among participants and enhancing their engagement in immersive educational experiences (Dalinger et al., [<reflink idref="bib17" id="ref139">17</reflink>]).</p> <hd id="AN0187257455-33">Insights from evaluations of simulated students</hd> <p>Evaluation results reveal that LLMs face challenges in applying appropriate and effective cognitive strategies to meet task‐specific requirements. One possible reason is the insufficient training in orchestrating fundamental cognitive processes required for forming higher‐layer cognitive processes. To address this issue, the Cognitive Echo method can be employed to incorporate experiences of higher‐order cognitive processes into the training data or prompts, particularly during problem‐solving tasks (Park et al., [<reflink idref="bib54" id="ref140">54</reflink>]). Additionally, chain‐of‐thought prompting can encourage LLMs to approach problems step by step, improving their ability to orchestrate fundamental cognitive processes more effectively (Wei et al., [<reflink idref="bib81" id="ref141">81</reflink>]). The evaluation results also highlight that LLMs struggle to recognize the limits of their own knowledge. Previous research has suggested that LLMs contain latent structures within their activation space that relate to beliefs about truthfulness (Huang et al., [<reflink idref="bib31" id="ref142">31</reflink>]). In this study, uploading memory elements related to knowledge boundaries has been proposed as a potential solution to mitigate this issue (Shao et al., [<reflink idref="bib66" id="ref143">66</reflink>]). Furthermore, employing retrieval‐augmented generation (RAG) can help address hallucinations related to knowledge gaps, ensuring more accurate outputs (Huang et al., [<reflink idref="bib31" id="ref144">31</reflink>]; Lewis et al., [<reflink idref="bib39" id="ref145">39</reflink>]; Park et al., [<reflink idref="bib54" id="ref146">54</reflink>]).</p> <p>The superior performance of GPT‐3.5 in single‐turn cases suggests that it is well‐optimized for generating contextually coherent responses in isolated interactions. However, its relatively weaker performance in multi‐turn cases may stem from challenges in maintaining response consistency and adapting strategies dynamically across evolving problem contexts, a limitation noted in recent studies on LLM conversational alignment (Yi et al., [<reflink idref="bib87" id="ref147">87</reflink>]). This discrepancy indicates that while GPT‐3.5 benefits from extensive pre‐training on diverse knowledge sources, its ability to sustain coherent problem‐solving strategies over multiple exchanges may require further optimization. Enhancing contextual awareness through structured multi‐turn educational dialogues could be a viable approach. For instance, incorporating directional stimulus prompting into prompts (Li, Shi, et al., [<reflink idref="bib42" id="ref148">42</reflink>]) or leveraging historical context summarization (Zhao et al., [<reflink idref="bib91" id="ref149">91</reflink>]) may help reinforce response continuity and strategic adaptation.</p> <p>It is worth noting that, for experimental purposes, this study relied solely on fine‐tuning LLMs to train the simulated agents. However, in practical applications, a hybrid approach could enhance efficiency. Rather than fine‐tuning separate models for each student profile, a limited number of pre‐trained models (eg, those representing low, intermediate and high proficiency levels) could serve as a foundation. Prompt tuning and RAG could then be applied to dynamically adjust and personalize the simulated students, allowing for more flexible and scalable adaptations to specific learner characteristics (Yi et al., [<reflink idref="bib87" id="ref150">87</reflink>]).</p> <hd id="AN0187257455-34">Practical implications</hd> <p>In teacher education, fine‐tuned LLMs can simulate the cognitive processes of students at different levels, producing realistic and logically consistent cognitive pathways. Preservice teachers can engage in iterative dialogues with these simulated students to identify potential cognitive obstacles faced by learners and adapt their teaching strategies accordingly. These simulations offer an effective means for preservice teachers to practice and refine their instructional skills (Frei‐Landau & Levin, [<reflink idref="bib26" id="ref151">26</reflink>]). Building on this foundation, our study extends these capabilities by introducing Cognitive Echo as a think‐aloud‐based tracking mechanism, enabling the generation of continuous, structured cognitive reports during simulated teacher–student interactions. Unlike general‐purpose dialogue agents, Cognitive Echo allows simulated students to verbalize reflective reasoning over time, offering researchers and educators access to the learner's evolving cognitive structure. This enables preservice teachers not only to respond to observable errors but also to recognize deeper learning difficulties and strategy shifts.</p> <p>Simulated students' cognitive reports serve as independent observation mechanisms that run parallel to teacher–student interactions, providing an objective basis for evaluating teaching effectiveness. In studies testing multi‐turn think‐aloud dialogue, simulated students showed marked improvements in Cognitive Completeness and strategic rationale. These think‐aloud‐generated cognitive reports capture implicit learning processes, providing a quantifiable foundation for preservice teacher training. In this context, think‐aloud functions as a synchronized cognitive tracking mechanism embedded in simulated students. This approach allows the system to continuously monitor and extract insights from simulated students' cognitive progressions, much like an invisible interviewer prompting learners to verbalize their evolving thought processes in real time. On the one hand, think‐aloud techniques can be used within a single teacher–student interaction to assess whether LLMs have assimilated knowledge more effectively, offering a metric for evaluating teacher training outcomes. On the other hand, analysing problem‐solving processes during teacher‐simulated student interactions allows researchers to determine whether teaching strategies effectively elicit deeper student thinking. This quantitative approach could address the challenge of a lack of evaluation criteria in simulation‐based learning processes (Cant & Cooper, [<reflink idref="bib9" id="ref152">9</reflink>]).</p> <p>Furthermore, the methodology has applications in creating <emph>digital twins</emph> of educational scenarios. By incorporating Cognitive Echo, the study provides a framework for modelling complex educational problems as digital twin environments through the integration of learning records and teaching contexts. This framework facilitates educational research and training under conditions that would otherwise be challenging to replicate. By leveraging empirical data collected from existing studies, researchers can reconstruct educational scenarios and simulate them using LLMs. This approach enables the testing of theoretical hypotheses related to complex educational challenges (Biri et al., [<reflink idref="bib6" id="ref153">6</reflink>]). For instance, mapping real‐world data onto LLMs to reconstruct behavioural patterns in school bullying scenarios allows practitioners to interact with the simulated environment, identify potential bullying behaviours and develop effective intervention strategies.</p> <hd id="AN0187257455-35">LIMITATION AND FUTURE RESEARCH</hd> <p>One limitation is the absence of real human think‐aloud data for comparison. Future research could compare simulated students' think‐aloud data with human students' think‐aloud data to validate the effectiveness of the Cognitive Echo method. Additionally, the scope of the training data was restricted. As most data were derived from public datasets, only a small subset that aligned with the requirements was ultimately selected. Future research should consider the diversity of data sources that reflect learning processes more comprehensively and increase the overall volume of training data. Another limitation lies in the validation process involving human coders and the generated learning experiences. Although human coders were engaged to review evaluating results, the study did not incorporate independent evaluations. To enhance the reliability of empirical findings, future research could introduce independent human evaluators to verify the results. Additionally, the quality assurance process for the generated learning experiences was limited to basic manual review. Future studies should consider developing explicit evaluation criteria for automated assessment protocols. In addition, future studies could benefit from incorporating feedback and learning gains from teachers or preservice teachers to demonstrate how simulated students function in practice. With regard to protective learning experiences, this study lacked a robust theoretical foundation for generating these experiences. Future studies could aim to develop a more comprehensive framework that accounts for ethical requirements. Furthermore, the cost of querying commercial LLM remains a significant concern, particularly when simulations are deployed at scale. While this study used GPT‐4 for data generation, future research could investigate more cost‐effective approaches by leveraging open‐source models for both experience generation and fine‐tuning (eg, DeepSeek‐R1 [Guo et al., [<reflink idref="bib27" id="ref154">27</reflink>]]). Recent studies also demonstrated that routing strategies combining multiple LLMs can substantially reduce costs without sacrificing performance (Chen et al., [<reflink idref="bib12" id="ref155">12</reflink>]; Ong et al., [<reflink idref="bib51" id="ref156">51</reflink>]). Integrating such strategies into future implementations of Cognitive Echo could help optimize the trade‐off between performance and cost, thereby supporting its sustainable use in educational contexts.</p> <hd id="AN0187257455-36">ACKNOWLEDGEMENTS</hd> <p>This work was supported by the National Social Science Foundation of China (Education) (grant number: BCA240054).</p> <hd id="AN0187257455-37">CONFLICT OF INTEREST STATEMENT</hd> <p>None.</p> <hd id="AN0187257455-38">DATA AVAILABILITY STATEMENT</hd> <p>Should the paper be accepted for publication, the key portions of the fine‐tuning training data for the large language model will be made openly accessible on GitHub, in accordance with open‐source data‐sharing practices. Access to the data will be granted under the terms specified in the repository.</p> <hd id="AN0187257455-39">ETHICS STATEMENT</hd> <p>This study was approved by the University Committee on Human Research Protection at East China Normal University (Approval No. HR 459‐2024). The study did not involve human participants or personally identifiable information. It used retrospective interaction records and AI‐simulated data generated from the public datasets.</p> <p>GRAPH: Appendix S1.</p> <ref id="AN0187257455-40"> <title> REFERENCES </title> <blist> <bibl id="bib1" idref="ref24" type="bt">1</bibl> <bibtext> Alshammari, T., Alhadreti, O., & Mayhew, P. (2015). When to ask participants to think aloud: A comparative study of concurrent and retrospective think‐aloud methods. International Journal of Human‐Computer Interaction, 6 (3), 48 – 64.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref103" type="bt">2</bibl> <bibtext> Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111 (4), 1036 – 1060.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref70" type="bt">3</bibl> <bibtext> Bates, J. (1994). The role of emotion in believable agents. Communications of the ACM, 37 (7), 122 – 125.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref50" type="bt">4</bibl> <bibtext> Baudrillard, J. (1994). Simulacra and simulation (S. F. Glaser, Trans.). University of Michigan Press (Original work published 1981).</bibtext> </blist> <blist> <bibl id="bib5" idref="ref30" type="bt">5</bibl> <bibtext> Beach, P., & Willows, D. (2017). Understanding teachers' cognitive processes during online professional learning: A methodological comparison. Online Learning, 21 (1), 60 – 84.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref153" type="bt">6</bibl> <bibtext> Biri, S. K., Kumar, S., Panigrahi, M., Mondal, S., Behera, J. K., & Mondal, H. (2023). Assessing the utilization of large language models in medical education: Insights from undergraduate medical students. Cureus, 15 (10), e47468. https://doi.org/10.7759/cureus.47468</bibtext> </blist> <blist> <bibl id="bib7" idref="ref59" type="bt">7</bibl> <bibtext> Brooks, R. A., Breazeal, C., Marjanović, M., Scassellati, B., & Williamson, M. M. (1998). The cog project: Building a humanoid robot. In International workshop on computation for metaphors, analogy, and agents (pp. 52 – 87). Springer Berlin Heidelberg.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref75" type="bt">8</bibl> <bibtext> Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert‐Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., ... Amodei, D. (2020). Language models are few‐shot learners. Advances in Neural Information Processing Systems, 33, 1877 – 1901.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref152" type="bt">9</bibl> <bibtext> Cant, R. P., & Cooper, S. J. (2010). Simulation‐based learning in nurse education: Systematic review. Journal of Advanced Nursing, 66 (1), 3 – 15.</bibtext> </blist> <blist> <bibtext> Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., Ye, W., Zhang, Y., Yu, P. S., Yang, Q., & Xie, X. (2024). A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15 (3), 1 – 45. https://doi.org/10.1145/3641289</bibtext> </blist> <blist> <bibtext> Charters, E. (2003). The use of think‐aloud methods in qualitative research an introduction to think‐aloud methods. Brock Education Journal, 12 (2), 68 – 82. https://doi.org/10.26522/brocked.v12i2.38</bibtext> </blist> <blist> <bibtext> Chen, L., Zaharia, M., & Zou, J. (2023). FrugalGPT: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176.</bibtext> </blist> <blist> <bibtext> Chen, Y., Ding, N., Zheng, H. T., Liu, Z., Sun, M., & Zhou, B. (2024). Empowering private tutoring by chaining large language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, Boise, ID, USA (pp. 354 – 364).</bibtext> </blist> <blist> <bibtext> Chiang, W.‐L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., & Xing, E. P. (2023, March). Vicuna: An open‐source chatbot impressing GPT‐4 with 90% ChatGPT quality. LMSYS. https://lmsys.org/blog/2023‐03‐30‐vicuna/</bibtext> </blist> <blist> <bibtext> Chrysafiadi, K., & Virvou, M. (2013). Student modeling approaches: A literature review for the last decade. Expert Systems with Applications, 40 (11), 4715 – 4729.</bibtext> </blist> <blist> <bibtext> Chu, S., Kim, J., & Yi, M. (2024). Think together and work better: Combining humans' and LLMs' think‐aloud outcomes for effective text evaluation. arXiv preprint arXiv:2409.07355.</bibtext> </blist> <blist> <bibtext> Dalinger, T., Thomas, K. B., Stansberry, S., & Xiu, Y. (2020). A mixed reality simulation offers strategic practice for pre‐service teachers. Computers & Education, 144, 103696. https://doi.org/10.1016/j.compedu.2019.103696</bibtext> </blist> <blist> <bibtext> Davis, J. N., & Bistodeau, L. (1993). How do L1 and L2 Reading differ? Evidence from think aloud protocols. The Modern Language Journal, 77 (4), 459 – 472.</bibtext> </blist> <blist> <bibtext> Ding, Q. (2023). Unraveling the landscape of large language models: A systematic review and future perspectives. Journal of Electronic Business & Digital Economics, 3 (1), 3 – 19. https://doi.org/10.1108/JEBDE‐08‐2023‐0015</bibtext> </blist> <blist> <bibtext> Doi, T. (2021). Usability textual data analysis: A formulaic coding think‐aloud protocol method for usability evaluation. Applied Sciences, 11 (15), 7047. https://doi.org/10.3390/app11157047</bibtext> </blist> <blist> <bibtext> Eccles, D. W., & Arsal, G. (2017). The think aloud method: What is it and how do I use it? Qualitative Research in Sport, Exercise and Health, 9 (4), 514 – 531. https://doi.org/10.1080/2159676X.2017.1331501</bibtext> </blist> <blist> <bibtext> Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87 (3), 215 – 251.</bibtext> </blist> <blist> <bibtext> Fan, M., Lin, J., Chung, C., & Truong, K. N. (2019). Concurrent think‐aloud verbalizations and usability problems. ACM Transactions on Computer‐Human Interaction (TOCHI), 26 (5), 1 – 35. https://doi.org/10.1145/3325281</bibtext> </blist> <blist> <bibtext> Fonteyn, M. E., Kuipers, B., & Grobe, S. J. (1993). A description of think aloud method and protocol analysis. Qualitative Health Research, 3 (4), 430 – 441. https://doi.org/10.1177/104973239300300403</bibtext> </blist> <blist> <bibtext> Fox, M. C., Ericsson, K. A., & Best, R. (2011). Do procedures for verbal reporting of thinking have to be reactive? A meta‐analysis and recommendations for best reporting methods. Psychological Bulletin, 137 (2), 316 – 344.</bibtext> </blist> <blist> <bibtext> Frei‐Landau, R., & Levin, O. (2022). The virtual Sim (HU) lation model: Conceptualization and implementation in the context of distant learning in teacher education. Teaching and Teacher Education, 117, 103798.</bibtext> </blist> <blist> <bibtext> Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., Zhang, X., Yu, X., Wu, Y., Wu, Z. F., Gou, Z., Shao, Z., Li, Z., Gao, Z., Liu, A., ... Zhang, Z. (2025). DeepSeek‐R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948.</bibtext> </blist> <blist> <bibtext> Heirweg, S., De Smul, M., Devos, G., & Van Keer, H. (2019). Profiling upper primary school students' self‐regulated learning through self‐report questionnaires and think‐aloud protocol analysis. Learning and Individual Differences, 70, 155 – 168.</bibtext> </blist> <blist> <bibtext> Hori, M., Kihara, Y., & Kato, T. (2011). Investigation of indirect oral operation method for think aloud usability testing. In Human Centered Design: Second International Conference, HCD 2011, Held as Part of HCI International 2011, Orlando, FL, USA, July 9–14, 2011. Proceedings 2 (pp. 38 – 46). Springer Berlin Heidelberg.</bibtext> </blist> <blist> <bibtext> Hu, E. J., Shen, Y., Wallis, P., Allen‐Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021). Lora: Low‐rank adaptation of large language models. arXiv preprint arXiv:2106.09685.</bibtext> </blist> <blist> <bibtext> Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., & Liu, T. (2025). A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43 (2), 1 – 55. https://doi.org/10.1145/3703155</bibtext> </blist> <blist> <bibtext> Jacobse, A. E., & Harskamp, E. G. (2012). Towards efficient measurement of metacognition in mathematical problem solving. Metacognition and Learning, 7 (2), 133 – 149. https://doi.org/10.1007/s11409‐012‐9088‐x</bibtext> </blist> <blist> <bibtext> Jonassen, D. H. (1997). Instructional design models for well‐structured and III‐structured problem‐solving learning outcomes. Educational Technology Research and Development, 45 (1), 65 – 94.</bibtext> </blist> <blist> <bibtext> Kaminski, K. S., & Sporer, S. L. (2017). Discriminating between correct and incorrect eyewitness identifications: The use of appropriate cues. Journal of Experimental Psychology: Applied, 23 (1), 59 – 70.</bibtext> </blist> <blist> <bibtext> Koedinger, K. R., Baker, R. S., Cunningham, K., Skogsholm, A., Leber, B., & Stamper, J. (2010). A data repository for the EDM community: The PSLC DataShop. Handbook of Educational Data Mining, 43, 43 – 56.</bibtext> </blist> <blist> <bibtext> Laird, J. E. (2001). It knows what you're going to do: Adding anticipation to a Quakebot. In Proceedings of the Fifth International Conference on Autonomous Agents, Montreal, Quebec, Canada (pp. 385 – 392).</bibtext> </blist> <blist> <bibtext> Lara, B., Gaona, W., Escobar, E., Pardo, J. M., & Hermosillo‐Valadez, J. (2021). Development of body‐based spatial knowledge through mental imagery in an artificial agent. Adaptive Behavior, 29 (4), 349 – 368.</bibtext> </blist> <blist> <bibtext> Lemaignan, S., Warnier, M., Sisbot, E. A., Clodic, A., & Alami, R. (2017). Artificial cognition for social human–robot interaction: An implementation. Artificial Intelligence, 247, 45 – 69. https://doi.org/10.1016/j.artint.2016.07.002</bibtext> </blist> <blist> <bibtext> Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.‐T., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval‐augmented generation for knowledge‐intensive NLP tasks. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in neural information processing systems (Vol. 33, pp. 9459 – 9474). Curran Associates, Inc.</bibtext> </blist> <blist> <bibtext> Li, G., Hammoud, H., Itani, H., Khizbullin, D., & Ghanem, B. (2023). Camel: Communicative agents for mind exploration of large language model society. Advances in Neural Information Processing Systems, 36, 51991 – 52008.</bibtext> </blist> <blist> <bibtext> Li, Z., Peng, B., He, P., Galley, M., Gao, J., & Yan, X. (2023). Guiding large language models via directional stimulus prompting. Advances in Neural Information Processing Systems, 36, 62630 – 62656.</bibtext> </blist> <blist> <bibtext> Li, Z., Shi, L., Wang, J., Cristea, A. I., & Zhou, Y. (2023). Sim‐GAIL: A generative adversarial imitation learning approach of student modelling for intelligent tutoring systems. Neural Computing and Applications, 35 (34), 24369 – 24388.</bibtext> </blist> <blist> <bibtext> Liu, C., Zhu, E., Zhang, Q., & Wei, X. (2018). Modeling of agent cognition in extensive games via artificial neural networks. IEEE Transactions on Neural Networks and Learning Systems, 29 (10), 4857 – 4868.</bibtext> </blist> <blist> <bibtext> Loxterman, J. A., Beck, I. L., & McKeown, M. G. (1994). The effects of thinking aloud during reading on students' comprehension of more or less coherent text. Reading Research Quarterly, 29, 353 – 367.</bibtext> </blist> <blist> <bibtext> MacLellan, C. J., Harpstead, E., Wiese, E. S., Zou, M., Matsuda, N., Aleven, V., & Koedinger, K. R. (2015). Authoring tutors with complex solutions: A comparative analysis of example tracing and SimStudent. In Workshops at the 17th International Conference on Artificial Intelligence in Education, Madrid, Spain (PP. 35 – 44).</bibtext> </blist> <blist> <bibtext> MacLellan, C. J., & Koedinger, K. R. (2022). Domain‐general tutor authoring with apprentice learner models. International Journal of Artificial Intelligence in Education, 32 (1), 76 – 117. https://doi.org/10.1007/s40593‐020‐00214‐2</bibtext> </blist> <blist> <bibtext> Matsuda, N., Cohen, W. W., & Koedinger, K. R. (2015). Teaching the teacher: Tutoring SimStudent leads to more effective cognitive tutor authoring. International Journal of Artificial Intelligence in Education, 25 (1), 1 – 34.</bibtext> </blist> <blist> <bibtext> Muckler, V. C. (2017). Exploring suspension of disbelief during simulation‐based learning. Clinical Simulation in Nursing, 13 (1), 3 – 9. https://doi.org/10.1016/j.ecns.2016.09.004</bibtext> </blist> <blist> <bibtext> Newell, A., & Simon, H. A. (1972). Human problem solving. Prentice Hall.</bibtext> </blist> <blist> <bibtext> Nielsen, J., Clemmensen, T., & Yssing, C. (2002). Getting access to what goes on in people's heads? Reflections on the think‐aloud technique. In Proceedings of the Second Nordic Conference on Human‐Computer Interaction, Aarhus, Denmark (pp. 101 – 110).</bibtext> </blist> <blist> <bibtext> Ong, I., Almahairi, A., Wu, V., Chiang, W.‐L., Wu, T., Gonzalez, J. E., Kadous, M. W., & Stoica, I. (2024). RouteLLM: An open‐source framework for cost‐effective LLM routing. LMSYS Org. https://lmsys.org/blog/2024‐07‐01‐routellm/</bibtext> </blist> <blist> <bibtext> OpenAI. (2025). GPT‐4 turbo and GPT‐4. OpenAI Documentation. https://platform.openai.com/docs/models#gpt‐4‐turbo‐and‐gpt‐4</bibtext> </blist> <blist> <bibtext> Overton, T., Potter, N., & Leng, C. (2013). A study of approaches to solving open‐ended problems in chemistry. Chemistry Education Research and Practice, 14 (4), 468 – 475.</bibtext> </blist> <blist> <bibtext> Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, San Francisco, CA, USA (pp. 1 – 22).</bibtext> </blist> <blist> <bibtext> Park, M., Kim, S., Lee, S., Kwon, S., & Kim, K. (2024). Empowering personalized learning through a conversation‐based tutoring system with student modeling. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems ,Honolulu, HI, USA (pp. 1 – 10).</bibtext> </blist> <blist> <bibtext> Phye, G. D. (2001). Problem‐solving instruction and problem‐solving transfer: The correspondence issue. Journal of Educational Psychology, 93 (3), 571 – 578.</bibtext> </blist> <blist> <bibtext> Pike, M. F., Maior, H. A., Porcheron, M., Sharples, S. C., & Wilson, M. L. (2014, April). Measuring the effect of think aloud protocols on workload using fNIRS. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, Ontario, Canada (pp. 3807 – 3816).</bibtext> </blist> <blist> <bibtext> Pólya, G. (1945). How to solve it: A new aspect of mathematical method. Princeton University Press.</bibtext> </blist> <blist> <bibtext> Premack, D., & Woodruff, G. (1978). Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences, 1 (4), 515 – 526. https://doi.org/10.1017/S0140525X00076512</bibtext> </blist> <blist> <bibtext> Ren, R., Wang, Y., Qu, Y., Zhao, W. X., Liu, J., Wu, H., Wen, J.‐R., & Wang, H. (2025). Investigating the factual knowledge boundary of large language models with retrieval augmentation. In Proceedings of the 31st International Conference on Computational Linguistics (pp. 3697 – 3715). Association for Computational Linguistics.</bibtext> </blist> <blist> <bibtext> Ritter, S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007). Cognitive tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14, 249 – 255.</bibtext> </blist> <blist> <bibtext> Roth, W. M., Oliveri, M. E., Sandilands, D. D., Lyons‐Thomas, J., & Ercikan, K. (2013). Investigating linguistic sources of differential item functioning using expert think‐aloud protocols in science achievement tests. International Journal of Science Education, 35 (4), 546 – 576.</bibtext> </blist> <blist> <bibtext> Ruan, Y., Dong, H., Wang, A., Pitis, S., Zhou, Y., Ba, J., Dubois, Y., Maddison, C. J., & Hashimoto, T. (2023). Identifying the risks of lm agents with an LM‐emulated sandbox. arXiv preprint arXiv:2309.15817.</bibtext> </blist> <blist> <bibtext> Sallam, M. (2023). Chatgpt utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns. Healthcare, 11 (6), 887. https://doi.org/10.3390/healthcare11060887</bibtext> </blist> <blist> <bibtext> Schoenfeld, A. H. (1985). Mathematical problem solving. Elsevier Science.</bibtext> </blist> <blist> <bibtext> Shao, Y., Li, L., Dai, J., & Qiu, X. (2023). Character‐LLM: A trainable agent for role‐playing. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore (pp. 13153 – 13187).</bibtext> </blist> <blist> <bibtext> Shen, J., Jain, A., Xiao, Z., Amlekar, I., Hadji, M., Podolny, A., & Talwalkar, A. (2024). ScribeAgent: Towards specialized web agents using production‐scale workflow data. arXiv preprint arXiv:2411.15004.</bibtext> </blist> <blist> <bibtext> Siemens, G., Marmolejo‐Ramos, F., Gabriel, F., Medeiros, K., Marrone, R., Joksimovic, S., & de Laat, M. (2022). Human and artificial cognition. Computers and Education: Artificial Intelligence, 3, 100107. https://doi.org/10.1016/j.caeai.2022.100107</bibtext> </blist> <blist> <bibtext> Slobodkin, A., Goldman, O., Caciularu, A., Dagan, I., & Ravfogel, S. (2023). The curious case of hallucinatory (un)answerability: Finding truths in the hidden states of over‐confident large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore (pp. 3607 – 3625).</bibtext> </blist> <blist> <bibtext> Stamper, J., Xiao, R., & Hou, X. (2024). Enhancing LLM‐based feedback: Insights from intelligent tutoring systems and the learning sciences. In International Conference on Artificial Intelligence in Education (pp. 32 – 43). Springer Nature Switzerland.</bibtext> </blist> <blist> <bibtext> Sweller, J., Van Merrienboer, J. J., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251 – 296.</bibtext> </blist> <blist> <bibtext> Tagliabue, L. C., Cecconi, F. R., Maltese, S., Rinaldi, S., Ciribini, A. L. C., & Flammini, A. (2021). Leveraging digital twin for sustainability assessment of an educational building. Sustainability, 13 (2), 480. https://doi.org/10.3390/su13020480</bibtext> </blist> <blist> <bibtext> Tu, Q., Fan, S., Tian, Z., & Yan, R. (2024). CharacterEval: A Chinese benchmark for role‐playing conversational agent evaluation. arXiv preprint arXiv:2401.01275.</bibtext> </blist> <blist> <bibtext> Van Lehn, K., Ohlsson, S., & Nason, R. (1994). Applications of simulated students: An exploration. Journal of Artificial Intelligence in Education, 5, 135.</bibtext> </blist> <blist> <bibtext> Van Pinxteren, M. M., Pluymaekers, M., & Lemmink, J. G. (2020). Human‐like communication in conversational agents: A literature review and research agenda. Journal of Service Management, 31 (2), 203 – 225.</bibtext> </blist> <blist> <bibtext> Vygotsky, L. S. (2012). Thought and language. MIT Press.</bibtext> </blist> <blist> <bibtext> Wang, Y., & Chiew, V. (2010). On the cognitive process of human problem solving. Cognitive Systems Research, 11 (1), 81 – 92. https://doi.org/10.1016/j.cogsys.2008.08.003</bibtext> </blist> <blist> <bibtext> Wang, L., Lian, J., Huang, Y., Dai, Y., Li, H., Chen, X., Xie, X., & Wen, J. R. (2024). CharacterBox: Evaluating the role‐playing capabilities of LLMs in text‐based virtual worlds. arXiv preprint arXiv:2412.05631.</bibtext> </blist> <blist> <bibtext> Wang, M., Zhang, X., Chen, K., Xu, N., Yu, H., Huang, F., Mao, W., & Li, Y. (2024). Reframing dialogue interaction with fine‐grained element modeling. arXiv preprint arXiv:2412.04905.</bibtext> </blist> <blist> <bibtext> Wang, X., Xiao, Y., Huang, J. T., Yuan, S., Xu, R., Guo, H., Tu, Q., Fei, Y., Leng, Z., Wang, W., Chen, J., Li, C., & Xiao, Y. (2024). Incharacter: Evaluating personality fidelity in role‐playing agents through psychological interviews. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand (pp. 1840 – 1873).</bibtext> </blist> <blist> <bibtext> Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022). Chain‐of‐thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35, 24824 – 24837.</bibtext> </blist> <blist> <bibtext> Wimmer, H., & Perner, J. (1983). Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children's understanding of deception. Cognition, 13 (1), 103 – 128.</bibtext> </blist> <blist> <bibtext> Xu, S., & Zhang, X. (2023). Leveraging generative artificial intelligence to simulate student learning behavior. arXiv preprint arXiv:2310.19206.</bibtext> </blist> <blist> <bibtext> Xu, S., Zhang, X., & Qin, L. (2024). EduAgent: Generative Student Agents in Learning. arXiv preprint arXiv:2404.07963.</bibtext> </blist> <blist> <bibtext> Yan, L., Sha, L., Zhao, L., Li, Y., Martinez‐Maldonado, R., Chen, G., Li, X., Jin, Y., & Gašević, D. (2024). Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55 (1), 90 – 112. https://doi.org/10.1111/bjet.13370</bibtext> </blist> <blist> <bibtext> Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., Lin, H., Tang, J., Wang, J., Yang, J., Wang, J., Yang, J., Tu, J., ... Fan, Z. (2024). Qwen2 technical report. arXiv preprint arXiv:2407.10671.</bibtext> </blist> <blist> <bibtext> Yi, Z., Ouyang, J., Liu, Y., Liao, T., Xu, Z., & Shen, Y. (2024). A survey on recent advances in LLM‐based multi‐turn dialogue systems. arXiv preprint arXiv:2402.18013.</bibtext> </blist> <blist> <bibtext> Zhang, J., Borchers, C., Aleven, V., & Baker, R. S. (2024). Using large language models to detect self‐regulated learning in think‐aloud protocols. In Proceedings of the 17th International Conference on Educational Data Mining, Atlanta, GA, USA.</bibtext> </blist> <blist> <bibtext> Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., Wu, F., & Wang, G. (2023). Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792.</bibtext> </blist> <blist> <bibtext> Zhao, Y., Yan, L., Sun, W., Xing, G., Meng, C., Wang, S., Cheng, Z., Ren, Z., & Yin, D. (2024). Knowing what LLMs do not know: A simple yet effective self‐detection method. In Proceedings of the 2024 conference of the north American chapter of the Association for Computational Linguistics: Human language technologies (Volume 1: Long Papers) (pp. 7051 – 7063). Association for Computational Linguistics.</bibtext> </blist> <blist> <bibtext> Zhao, Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021). Calibrate before use: Improving few‐shot performance of language models. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Virtual (Vol. 139, pp. 12697 – 12706).</bibtext> </blist> <blist> <bibtext> Zheng, L., Jiang, F., Gu, X., Li, Y., Wang, G., & Zhang, H. (2025). Teaching via LLM‐enhanced simulations: Authenticity and barriers to suspension of disbelief. The Internet and Higher Education, 65, 100990. https://doi.org/10.1016/j.iheduc.2024.100990</bibtext> </blist> </ref> <aug> <p>By Longwei Zheng; Anna He; Changyong Qi; Haomin Zhang and Xiaoqing Gu</p> <p>Reported by Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib23" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib44" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib62" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib11" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib32" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib50" firstref="ref6"></nolink> <nolink nlid="nl7" bibid="bib16" firstref="ref7"></nolink> <nolink nlid="nl8" bibid="bib74" firstref="ref8"></nolink> <nolink nlid="nl9" bibid="bib47" firstref="ref9"></nolink> <nolink nlid="nl10" bibid="bib54" firstref="ref10"></nolink> <nolink nlid="nl11" bibid="bib66" firstref="ref11"></nolink> <nolink nlid="nl12" bibid="bib70" firstref="ref12"></nolink> <nolink nlid="nl13" bibid="bib13" firstref="ref13"></nolink> <nolink nlid="nl14" bibid="bib55" firstref="ref14"></nolink> <nolink nlid="nl15" bibid="bib26" firstref="ref15"></nolink> <nolink nlid="nl16" bibid="bib19" firstref="ref16"></nolink> <nolink nlid="nl17" bibid="bib64" firstref="ref18"></nolink> <nolink nlid="nl18" bibid="bib18" firstref="ref19"></nolink> <nolink nlid="nl19" bibid="bib24" firstref="ref20"></nolink> <nolink nlid="nl20" bibid="bib22" firstref="ref21"></nolink> <nolink nlid="nl21" bibid="bib76" firstref="ref22"></nolink> <nolink nlid="nl22" bibid="bib20" firstref="ref25"></nolink> <nolink nlid="nl23" bibid="bib34" firstref="ref26"></nolink> <nolink nlid="nl24" bibid="bib53" firstref="ref27"></nolink> <nolink nlid="nl25" bibid="bib28" firstref="ref28"></nolink> <nolink nlid="nl26" bibid="bib21" firstref="ref31"></nolink> <nolink nlid="nl27" bibid="bib57" firstref="ref32"></nolink> <nolink nlid="nl28" bibid="bib25" firstref="ref35"></nolink> <nolink nlid="nl29" bibid="bib29" firstref="ref39"></nolink> <nolink nlid="nl30" bibid="bib15" firstref="ref40"></nolink> <nolink nlid="nl31" bibid="bib61" firstref="ref41"></nolink> <nolink nlid="nl32" bibid="bib41" firstref="ref43"></nolink> <nolink nlid="nl33" bibid="bib45" firstref="ref45"></nolink> <nolink nlid="nl34" bibid="bib46" firstref="ref47"></nolink> <nolink nlid="nl35" bibid="bib38" firstref="ref54"></nolink> <nolink nlid="nl36" bibid="bib43" firstref="ref55"></nolink> <nolink nlid="nl37" bibid="bib68" firstref="ref56"></nolink> <nolink nlid="nl38" bibid="bib37" firstref="ref57"></nolink> <nolink nlid="nl39" bibid="bib75" firstref="ref60"></nolink> <nolink nlid="nl40" bibid="bib59" firstref="ref62"></nolink> <nolink nlid="nl41" bibid="bib82" firstref="ref63"></nolink> <nolink nlid="nl42" bibid="bib36" firstref="ref66"></nolink> <nolink nlid="nl43" bibid="bib10" firstref="ref71"></nolink> <nolink nlid="nl44" bibid="bib88" firstref="ref72"></nolink> <nolink nlid="nl45" bibid="bib84" firstref="ref73"></nolink> <nolink nlid="nl46" bibid="bib83" firstref="ref74"></nolink> <nolink nlid="nl47" bibid="bib73" firstref="ref78"></nolink> <nolink nlid="nl48" bibid="bib40" firstref="ref79"></nolink> <nolink nlid="nl49" bibid="bib80" firstref="ref82"></nolink> <nolink nlid="nl50" bibid="bib63" firstref="ref85"></nolink> <nolink nlid="nl51" bibid="bib92" firstref="ref86"></nolink> <nolink nlid="nl52" bibid="bib72" firstref="ref91"></nolink> <nolink nlid="nl53" bibid="bib52" firstref="ref93"></nolink> <nolink nlid="nl54" bibid="bib35" firstref="ref94"></nolink> <nolink nlid="nl55" bibid="bib86" firstref="ref97"></nolink> <nolink nlid="nl56" bibid="bib67" firstref="ref98"></nolink> <nolink nlid="nl57" bibid="bib78" firstref="ref99"></nolink> <nolink nlid="nl58" bibid="bib79" firstref="ref100"></nolink> <nolink nlid="nl59" bibid="bib30" firstref="ref101"></nolink> <nolink nlid="nl60" bibid="bib14" firstref="ref102"></nolink> <nolink nlid="nl61" bibid="bib71" firstref="ref104"></nolink> <nolink nlid="nl62" bibid="bib33" firstref="ref105"></nolink> <nolink nlid="nl63" bibid="bib56" firstref="ref106"></nolink> <nolink nlid="nl64" bibid="bib65" firstref="ref107"></nolink> <nolink nlid="nl65" bibid="bib58" firstref="ref108"></nolink> <nolink nlid="nl66" bibid="bib49" firstref="ref109"></nolink> <nolink nlid="nl67" bibid="bib77" firstref="ref110"></nolink> <nolink nlid="nl68" bibid="bib31" firstref="ref114"></nolink> <nolink nlid="nl69" bibid="bib60" firstref="ref117"></nolink> <nolink nlid="nl70" bibid="bib90" firstref="ref118"></nolink> <nolink nlid="nl71" bibid="bib69" firstref="ref120"></nolink> <nolink nlid="nl72" bibid="bib81" firstref="ref123"></nolink> <nolink nlid="nl73" bibid="bib85" firstref="ref133"></nolink> <nolink nlid="nl74" bibid="bib89" firstref="ref134"></nolink> <nolink nlid="nl75" bibid="bib48" firstref="ref138"></nolink> <nolink nlid="nl76" bibid="bib17" firstref="ref139"></nolink> <nolink nlid="nl77" bibid="bib39" firstref="ref145"></nolink> <nolink nlid="nl78" bibid="bib87" firstref="ref147"></nolink> <nolink nlid="nl79" bibid="bib42" firstref="ref148"></nolink> <nolink nlid="nl80" bibid="bib91" firstref="ref149"></nolink> <nolink nlid="nl81" bibid="bib27" firstref="ref154"></nolink> <nolink nlid="nl82" bibid="bib12" firstref="ref155"></nolink> <nolink nlid="nl83" bibid="bib51" firstref="ref156"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1480023
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Cognitive Echo: Enhancing Think-Aloud Protocols with LLM-Based Simulated Students
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Longwei+Zheng%22">Longwei Zheng</searchLink><br /><searchLink fieldCode="AR" term="%22Anna+He%22">Anna He</searchLink><br /><searchLink fieldCode="AR" term="%22Changyong+Qi%22">Changyong Qi</searchLink><br /><searchLink fieldCode="AR" term="%22Haomin+Zhang%22">Haomin Zhang</searchLink><br /><searchLink fieldCode="AR" term="%22Xiaoqing+Gu%22">Xiaoqing Gu</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-8256-5408">0000-0001-8256-5408</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22British+Journal+of+Educational+Technology%22"><i>British Journal of Educational Technology</i></searchLink>. 2025 56(5):2019-2042.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 24
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Protocol+Analysis%22">Protocol Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Experience%22">Learning Experience</searchLink><br /><searchLink fieldCode="DE" term="%22Computational+Linguistics%22">Computational Linguistics</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software%22">Computer Software</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Processes%22">Learning Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Interference+%28Learning%29%22">Interference (Learning)</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Generalization%22">Generalization</searchLink><br /><searchLink fieldCode="DE" term="%22Learning+Strategies%22">Learning Strategies</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Simulation%22">Computer Simulation</searchLink><br /><searchLink fieldCode="DE" term="%22Authentic+Learning%22">Authentic Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Playgrounds%22">Playgrounds</searchLink><br /><searchLink fieldCode="DE" term="%22Transfer+of+Training%22">Transfer of Training</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/bjet.13590
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0007-1013<br />1467-8535
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In the field of education, the think-aloud protocol is commonly used to encourage learners to articulate their thoughts during the learning process, providing observers with valuable insights into learners' cognitive processes beyond the final learning outcomes. However, the implementation of think-aloud protocols faces challenges such as task interference and limitations in completeness and authenticity of verbal reports. This study proposes a method called Cognitive Echo, which leverages large language models (LLMs) trained with simulated student experiences to enhance the completeness and authenticity of think-aloud verbalizations. LLMs have been demonstrated to simulate human-like behaviour more effectively by memorizing experiences. In this work, we introduce specific learner roles and train the LLMs to act as distinct learners. Our method involves integrating transaction data from learners' interactions with a tutoring system and the tutor's content to create interactive experiences between learners and teachers, thereby training the model to become simulated students with learning experiences. To investigate the effectiveness of this approach, we designed a test playground based on the retrospective think-aloud protocol and examined how LLM-trained simulated students improve cognitive process transparency and generalization of learning strategies. The study found that Cognitive Echo not only reveals what simulated students genuinely think about their learning experiences but also enables them to transfer their different cognitive strategies to new tasks. By training simulated students on real learning behaviour data to ensure their cognitive processes reflect authentic learner experiences, this approach will extend think-aloud protocols to more practice-oriented applications.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1480023
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1480023
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/bjet.13590
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 24
        StartPage: 2019
    Subjects:
      – SubjectFull: Protocol Analysis
        Type: general
      – SubjectFull: Learning Experience
        Type: general
      – SubjectFull: Computational Linguistics
        Type: general
      – SubjectFull: Computer Software
        Type: general
      – SubjectFull: Learning Processes
        Type: general
      – SubjectFull: Interference (Learning)
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Generalization
        Type: general
      – SubjectFull: Learning Strategies
        Type: general
      – SubjectFull: Computer Simulation
        Type: general
      – SubjectFull: Authentic Learning
        Type: general
      – SubjectFull: Playgrounds
        Type: general
      – SubjectFull: Transfer of Training
        Type: general
    Titles:
      – TitleFull: Cognitive Echo: Enhancing Think-Aloud Protocols with LLM-Based Simulated Students
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Longwei Zheng
      – PersonEntity:
          Name:
            NameFull: Anna He
      – PersonEntity:
          Name:
            NameFull: Changyong Qi
      – PersonEntity:
          Name:
            NameFull: Haomin Zhang
      – PersonEntity:
          Name:
            NameFull: Xiaoqing Gu
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 09
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0007-1013
            – Type: issn-electronic
              Value: 1467-8535
          Numbering:
            – Type: volume
              Value: 56
            – Type: issue
              Value: 5
          Titles:
            – TitleFull: British Journal of Educational Technology
              Type: main
ResultId 1