Assessing Learning Processes with Multimodal Data in Virtual Reality Learning Environments

2026-07-16

AI Summary

This study addresses the limitations of traditional pre- and post-test assessments in virtual reality (VR) learning environments, which often fail to capture higher-order cognitive processes during immersive experiences. By implementing a VR escape-room task, the research pioneers a systematic integration of behavioral log data and verbal reflective responses to construct a multimodal, process-oriented assessment framework. This approach enables a fine-grained analysis of learners’ problem-solving strategies and metacognitive behaviors while engaging with logical puzzles. The study successfully distinguishes between deliberate reasoning and random guessing patterns, offering a novel pathway to uncover and understand authentic learning processes within VR-based educational settings.

Abstract

Assessing learning in virtual reality (VR) environments typically relied on traditional pre-post content retention tests, revealing little about the process of learnng within such immersive environments. Multimodal data from player activity in VR is promising to better measure learning processes and higher-order skills, but little research in the learning sciences has explored how such data can be combined to provide meaningful measures. To address this, we explored multimodal sources of data from a VR escape room containing hands-on logic puzzles that require problem-solving strategies and engagement in verbal metacognitive reflections. We leverage VR's affordances to immerse users in sustained dialog and promote multimodal interactions in a digital environment to understand how logfile and verbal data reveal participants' reasoning skills versus guesswork.

Problem

• virtual reality
• multimodal data
• learning processes
• assessment
• metacognition

Innovation

• multimodal data
• virtual reality
• learning processes
• metacognitive reflection
• problem-solving strategies