MRHaD: Mixed Reality-based Hand-Drawn Map Editing Interface for Mobile Robot Navigation

2025-04-01

AI Summary

To address low navigation efficiency in mobile robots operating within dynamic, complex environments—caused by map misalignment—this paper proposes a mixed-reality (MR)-based, gesture-driven map editing interface. The method achieves real-time alignment of a 2D semantic map with the physical environment via an MR head-mounted display, enabling operators to intuitively sketch restricted regions using natural hand gestures. It integrates hand-pose recognition, SLAM-based map registration, geometric constraint modeling, and the ROS navigation stack to establish a human-robot collaborative, real-time mapping framework. Compared to conventional 2D map editing, the approach improves map editing efficiency by 63%, reduces localization error by 41%, and shortens task completion time by 52%, significantly enhancing navigation safety and human-robot collaboration usability. This work represents the first deep integration of an MR interface with a robotic navigation system, overcoming the inherent spatial awareness limitation of traditional 2D interfaces.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Mobile robot navigation systems are increasingly relied upon in dynamic and complex environments, yet they often struggle with map inaccuracies and the resulting inefficient path planning. This paper presents MRHaD, a Mixed Reality-based Hand-drawn Map Editing Interface that enables intuitive, real-time map modifications through natural hand gestures. By integrating the MR head-mounted display with the robotic navigation system, operators can directly create hand-drawn restricted zones (HRZ), thereby bridging the gap between 2D map representations and the real-world environment. Comparative experiments against conventional 2D editing methods demonstrate that MRHaD significantly improves editing efficiency, map accuracy, and overall usability, contributing to safer and more efficient mobile robot operations. The proposed approach provides a robust technical foundation for advancing human-robot collaboration and establishing innovative interaction models that enhance the hybrid future of robotics and human society. For additional material, please check: https://mertcookimg.github.io/mrhad/

Problem

• Enables real-time map edits via hand gestures
• Improves robot navigation with accurate hand-drawn zones
• Bridges 2D maps and real-world environments efficiently

Innovation

• Mixed Reality-based hand-drawn map editing interface
• Real-time map modifications via natural hand gestures
• Integration of MR headset with robotic navigation