The Role of Hand Tracking and Spatial Scanning Based User Interactions in Immersive Technologies Supporting Active Learning

Abstract

Virtual (VR) and mixed reality (MR) technologies continue to evolve, offering new opportunities for immersive educational experiences, particularly in educational museum environments where physical access to certain exhibits or environments may be restricted. Recent advances in user interaction modalities, such as hand tracking and spatial scanning, present promising possibilities for creating more natural, engaging, and memorable user experiences. However, their effectiveness compared to traditional controller-based VR interactions remains unclear yet. This thesis investigates how these emerging modalities can enhance presence, engagement, and memory retention in immersive educational contexts. Two proof-of-concept prototypes were developed: a stationary VR experience using controllers, and an MR experience incorporating hand tracking and spatial scanning. Both were situated in a realistic herbarium scenario developed in collaboration with the Natural History Museum in Kristiansand.

Following a human-centered design (HCD) approach, the study drew upon state-of-the-art literature and key theoretical frameworks, including embodied cognition, experiential learning, and Mayer’s principles of multimedia learning, alongside preliminary user interviews with six museum visitors to validate requirements and interests. To evaluate the immersive prototypes, a qualitative-dominant mixed methods evaluation with 16 participants assessed user's perceived level of presence, engagement and memory retention through observations, questionnaires, and interviews. The study's findings are promising as both VR and MR user interaction modalities can effectively be designed to support active learning. However, it is evident that MR interactions led to increased presence and engagement due to their embodied and intuitive nature of user interactions, in contrast to VR, which was often limited by restricted movement and a challenging controller learning curve. While both prototype approaches supported short-term memory retention, the impact on long-term retention remains unexplored and presents an opportunity for future research. The study also identifies fundamental design decisions such as onboarding, multimodal guidance, and interaction clarity in supporting effective learning in immersive environments. These insights contribute to the design of meaningful, user-centered VR and MR experiences in educational and museum settings.