Augmented Reality Interfaces for Human-Robot Collaboration: Development of a ROS 2-Based Sensor Streaming Framework and Validation via SLAM Algorithms

2026-09-25

AI Summary

This work addresses the pressing demand for efficient bidirectional communication in human-robot collaboration within Industry 4.0 by proposing a real-time sensor streaming framework that bridges Magic Leap 2 and ROS 2. By integrating Unity, ROSTCP-Connector, and SLAM algorithms, the proposed architecture achieves seamless interoperability between augmented reality headsets and robotic ecosystems, enabling stable publishing of multimodal sensory data alongside enhanced spatial perception. This study overcomes critical data transmission bottlenecks inherent in heterogeneous systems, thereby establishing a reliable technical foundation for intuitive robot control and safe human-robot interaction in complex industrial environments.

Abstract

In recent years, Human-Robot Collaboration (HRC) has taken on a central role in Industry 4.0 and collaborative robotics, demanding communication channels that are increasingly bidirectional, intuitive, and efficient. In this context, Augmented Reality (AR) presents itself as a fundamental enabling technology, capable of both displaying information to the operator and gathering spatial data about the surrounding environment. This thesis presents the development of a sensor streaming framework that connects the Magic Leap 2 AR headset with the ROS 2 (Robot Operating System) ecosystem. Using the Unity development environment and the ROSTCP-Connector package, an on-board application for the headset was developed, capable of acquiring real-time data from the integrated sensors (pose tracking, cameras, and environmental sensors) and publishing it to dedicated ROS 2 topics. In order to test the accuracy, latency, and robustness of the generated data stream, the framework was validated using SLAM (Simultaneous Localization and Mapping) algorithms known in the literature. The experimental results demonstrate that the proposed architecture ensures stable data transmission, laying the groundwork for safe real-time interaction and shared spatial awareness, and opening up new perspectives for the control and supervision of robotic systems in complex HRC scenarios.

Problem

• Human-Robot Collaboration
• Augmented Reality
• ROS 2
• Sensor Streaming
• SLAM

Innovation

• Augmented Reality
• ROS 2
• Human-Robot Collaboration
• Sensor Streaming
• SLAM