Cognitive Assessment and Training in Extended Reality: Multimodal Systems, Clinical Utility, and Current Challenges

2025-01-13

Encyclopedia

AI Summary

Current XR-based cognitive assessment and training systems suffer from low ecological validity, poor user experience, high cost, and limited cross-population adaptability—particularly hindering clinical deployment among older adults and individuals with cognitive impairments. To address these limitations, this study introduces the first closed-loop feedback framework integrating VR, AR, and MR with multimodal real-time biosensing (EEG, galvanic skin response, eye tracking, and hand/body motion tracking), overcoming the constraints of unimodal approaches. The framework systematically identifies clinical implementation bottlenecks and establishes a scalable, age- and ability-inclusive adaptation pathway. The implemented system significantly enhances ecological validity and user engagement in cognitive tasks while enabling personalized intervention. It establishes a novel, low-cost, easily deployable, and ecologically valid digital paradigm for cognitive assessment and rehabilitation, with direct applications in neuroscience and clinical neurorehabilitation.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Extended reality (XR) technologies—encompassing virtual reality (VR), augmented reality (AR), and mixed reality (MR)—are transforming cognitive assessment and training by offering immersive, interactive environments that simulate real-world tasks. XR enhances ecological validity while enabling real-time, multimodal data collection through tools such as galvanic skin response (GSR), electroencephalography (EEG), eye tracking (ET), hand tracking, and body tracking. This allows for a more comprehensive understanding of cognitive and emotional processes, as well as adaptive, personalized interventions for users. Despite these advancements, current XR applications often underutilize the full potential of multimodal integration, relying primarily on visual and auditory inputs. Challenges such as cybersickness, usability concerns, and accessibility barriers further limit the widespread adoption of XR tools in cognitive science and clinical practice. This review examines XR-based cognitive assessment and training, focusing on its advantages over traditional methods, including ecological validity, engagement, and adaptability. It also explores unresolved challenges such as system usability, cost, and the need for multimodal feedback integration. The review concludes by identifying opportunities for optimizing XR tools to improve cognitive evaluation and rehabilitation outcomes, particularly for diverse populations, including older adults and individuals with cognitive impairments.

Problem

• XR technology
• brain assessment
• cognitive disorders

Innovation

• Extended Reality (XR) in Brain Assessment and Training
• Comprehensive Data Collection
• Personalized Training in Neuroscience