Evaluating the Impact of Adaptive Extended Reality on Human-Robot Interaction Across the Reality-Virtuality Continuum

2026-09-25

AI Summary

This study addresses the limitation that fixed extended reality (XR) modes impose on human-computer interaction by proposing an adaptive interface that dynamically adjusts XR modalities along the reality-virtuality continuum. An XR application supporting runtime modality switching was developed and integrated with a mobile manipulation robot interface. Augmented reality (AR), augmented virtuality (AV), virtual reality (VR), and the adaptive continuum mode were comparatively evaluated in multi-room tasks, with user experience quantified using NASA-TLX and SUS. Results demonstrate that the dynamic continuum mode achieves the highest throughput and lowest workload, significantly outperforming conventional fixed modalities. By overcoming the interaction constraints inherent to single XR modes, this work establishes a novel paradigm for efficient robot teleoperation.

Abstract

As populations in developed countries age and labor shortages intensify, Cybernetic Avatars (CAs) are proposed to extend human capabilities through robotic embodiments, requiring effective Human-Robot Interaction (HRI) frameworks. Extended Reality (XR), an umbrella term for Augmented Reality (AR), Augmented Virtuality (AV), and Virtual Reality (VR), offers such interfaces, but prior research typically fixes the XR modality without evaluating its effect on task outcomes. This study examines whether the XR modality impacts HRI performance and whether an adaptive interface adjusting the level of virtuality along the Reality-Virtuality Continuum (RVC) at runtime improves it. A custom XR application interfaced with a mobile manipulator supports immersive control and runtime modality switching. In a within-participant multi-room pick-and-place experiment comparing fixed AR, AV, and VR with dynamic RVC through task metrics, the NASA-TLX, and the System Usability Scale (SUS), this study demonstrates that 1) the fixed reality modality affects HRI results, and 2) dynamically changing the modality along the RVC improves them. AR yielded significantly lower mental demand, effort, and frustration than AV and VR, while the dynamic RVC condition achieved the highest throughput and lowest workload, highlighting the value of adaptive XR interfaces for human-robot symbiosis. The implementation is available at https://github.com/CarlTornberg/XR-HRI.

Problem

• Human-Robot Interaction
• Extended Reality
• Reality-Virtuality Continuum
• Cybernetic Avatars
• Adaptive Interface

Innovation

• Adaptive Extended Reality
• Reality-Virtuality Continuum
• Human-Robot Interaction
• Runtime Modality Switching
• Cybernetic Avatars