Extended Reality as a Mediation Layer for Situated Human Control in Human-Robot Teaming

2026-07-27

AI Summary

This work addresses the challenge of enabling humans to effectively understand and regulate robot behavior in dynamic human-robot collaboration, particularly within complex physical, social, and temporal contexts. To bridge this gap, the paper proposes extended reality (XR) as a mediating layer for contextualized control, articulating four core functionalities—intent communication, plan adaptation, role switching, and task recovery—and six design dimensions that integrate multimodal interaction, shared control, and context awareness. The resulting framework supports actionable and accountable human-robot teamwork and has been validated in scenarios such as robot-assisted caregiving, multi-arm monitoring, and distracted assembly. Furthermore, it offers a testable and tunable design paradigm and research agenda for XR systems operating in dynamically shared environments.

Abstract

Extended Reality (XR) is increasingly used in human-robot interaction to communicate robot intent, planned motion, reachability, and state. We argue that XR should also be understood as a mediation layer for situated human control in human-robot teaming. Situated human control denotes the human collaborator's ability to understand, shape, authorize, and interrupt robot action within the concrete physical, social, and temporal context in which that action unfolds. We ground this perspective in scenarios from robot-assisted bedside nursing, multi-arm supervisory control, and collaborative assembly under divided attention. Across these scenarios, robot autonomy must remain inspectable and adjustable as people move, goals change, sensing is incomplete, control roles shift, and plans become invalid. We identify four mediation functions connecting human intent and robot autonomy, robot plans and human judgment, levels of shared control, and team roles, handover, and recovery. Building on these functions, we derive six design dimensions: joint action possibilities, socio-physical constraints, uncertainty and plan validity, multimodal control and correction, roles, handover, and accountability, and anticipatory recovery. The paper outlines a research agenda for XR systems that make robot autonomy more actionable and accountable in dynamic shared environments.

Problem

• Extended Reality
• Human-Robot Teaming
• Situated Human Control
• Robot Autonomy
• Shared Control

Innovation

• Extended Reality
• Situated Human Control
• Human-Robot Teaming
• Mediation Layer
• Actionable Autonomy