TUM Teleoperation: Open Source Software for Remote Driving and Assistance of Automated Vehicles

2025-06-16

AI Summary

Existing open-source teleoperation software lacks integrated support for remote driving (steering/pedal control), high-level remote assistance (interaction with autonomous driving modules), and real-vehicle integration. Method: This paper introduces the first open-source, modular ROS 2 teleoperation software stack that unifies remote driving, semantic-level remote assistance, and native interoperability with autonomous driving frameworks (e.g., Autoware) and physical vehicle platforms. Contribution/Results: It is the first to integrate these three teleoperation paradigms; provides standardized APIs, extensible human–machine interfaces, and a cross-platform, low-latency communication protocol achieving end-to-end latency <100 ms; adopts a microservice architecture enabling seamless deployment in both simulation and real-vehicle environments. The implementation is fully open-sourced, validated across diverse hardware platforms, and supports collaborative development and real-user studies.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Multiagent Systems: Agent Communication
• Application Domains: Mobility, Driving & Flight

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Teleoperation is a key enabler for future mobility, supporting Automated Vehicles in rare and complex scenarios beyond the capabilities of their automation. Despite ongoing research, no open source software currently combines Remote Driving, e.g., via steering wheel and pedals, Remote Assistance through high-level interaction with automated driving software modules, and integration with a real-world vehicle for practical testing. To address this gap, we present a modular, open source teleoperation software stack that can interact with an automated driving software, e.g., Autoware, enabling Remote Assistance and Remote Driving. The software featuresstandardized interfaces for seamless integration with various real-world and simulation platforms, while allowing for flexible design of the human-machine interface. The system is designed for modularity and ease of extension, serving as a foundation for collaborative development on individual software components as well as realistic testing and user studies. To demonstrate the applicability of our software, we evaluated the latency and performance of different vehicle platforms in simulation and real-world. The source code is available on GitHub

Problem

• Lack of open source software for remote driving and assistance
• Need for integration with real-world automated vehicles
• Absence of modular teleoperation solutions for collaborative development

Innovation

• Modular open source teleoperation software stack
• Standardized interfaces for seamless integration
• Supports Remote Assistance and Remote Driving