Voice-augmented extended reality digital twin instruction: the case of the Rubik's Cube.
Saved in:
| Title: | Voice-augmented extended reality digital twin instruction: the case of the Rubik's Cube. |
|---|---|
| Authors: | Hajahmadi, Shirin1 (AUTHOR) shirin.hajahmadi2@unibo.it, Cascarano, Pasquale2 (AUTHOR), Stacchio, Lorenzo3 (AUTHOR), Giacché, Alessandro1 (AUTHOR), Marfia, Gustavo2 (AUTHOR) |
| Source: | Virtual Reality. Sep2026, Vol. 30 Issue 3, p1-20. 20p. |
| Subjects: | Digital twin, Auditory learning, Cognitive load, User experience, Cubes, Teaching methods, Motor learning, Mixed reality |
| Abstract: | Extended Reality (XR) and Digital Twins (DTs) present promising opportunities for immersive, state-aware procedural learning, making them well-suited for step-by-step instructional tasks. Their ability to simulate real-world contexts aligns closely with experiential learning principles, which emphasize active engagement, reflection, and learning by doing in authentic environments. Building on this potential, our prior work introduced a DT-driven XR framework for learning procedural Rubik's Cube notations, comparing three instructional modalities— a baseline visual static information condition with physical cube manipulation (M1), in-situ visual overlays with physical cube manipulation (M2), and in-situ visual overlays with virtual-then-physical cube manipulation (M3). However, the potential of multimodal guidance remained underexplored. This study extends the previous work by introducing and evaluating a fourth instructional modality, namely voice-augmented visual guidance (M4), which synchronizes auditory narration with in-situ visual overlays during physical cube manipulation. We assessed all four modalities across two post-training phases using task accuracy, completion time, and intra-action pause duration, alongside global measures of usability, acceptance, and cognitive effort. In the evaluated setting, M4 did not yield statistically significant differences relative to the other modalities (M1–M3), but achieved comparable performance and user-experience outcomes without additional cognitive or usability costs. These results delineate boundary conditions for multimodal instruction in this context and suggest that voice augmentation is best viewed as a complementary design element rather than an inherently performance-enhancing mechanism. [ABSTRACT FROM AUTHOR] |
| Copyright of Virtual Reality is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Extended Reality (XR) and Digital Twins (DTs) present promising opportunities for immersive, state-aware procedural learning, making them well-suited for step-by-step instructional tasks. Their ability to simulate real-world contexts aligns closely with experiential learning principles, which emphasize active engagement, reflection, and learning by doing in authentic environments. Building on this potential, our prior work introduced a DT-driven XR framework for learning procedural Rubik's Cube notations, comparing three instructional modalities— a baseline visual static information condition with physical cube manipulation (M1), in-situ visual overlays with physical cube manipulation (M2), and in-situ visual overlays with virtual-then-physical cube manipulation (M3). However, the potential of multimodal guidance remained underexplored. This study extends the previous work by introducing and evaluating a fourth instructional modality, namely voice-augmented visual guidance (M4), which synchronizes auditory narration with in-situ visual overlays during physical cube manipulation. We assessed all four modalities across two post-training phases using task accuracy, completion time, and intra-action pause duration, alongside global measures of usability, acceptance, and cognitive effort. In the evaluated setting, M4 did not yield statistically significant differences relative to the other modalities (M1–M3), but achieved comparable performance and user-experience outcomes without additional cognitive or usability costs. These results delineate boundary conditions for multimodal instruction in this context and suggest that voice augmentation is best viewed as a complementary design element rather than an inherently performance-enhancing mechanism. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 13594338 |
| DOI: | 10.1007/s10055-026-01386-3 |
