CD-TWINSAFE: A ROS-enabled Digital Twin for Scene Understanding and Safety Emerging V2I Technology

2026-01-18

AI Summary

This study addresses the challenges of real-time perception and safety warning for autonomous vehicles in complex traffic scenarios by proposing an integrated solution that combines vehicle-to-infrastructure (V2I) communication with digital twin technology. The system leverages onboard stereo vision for scene understanding and employs a ROS 2 architecture with UDP over 4G to achieve low-latency data transmission. A dynamically synchronized virtual environment is constructed in Unreal Engine 5, enabling real-time replication of vehicle states and surrounding traffic entities. This framework facilitates remote monitoring and early warning based on active safety metrics such as time-to-collision (TTC) and time headway. Experimental results demonstrate the system’s real-time performance and effectiveness across multiple scenarios, achieving efficient closed-loop coordination among perception, communication, and digital twin components.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Vision for Robotics & Autonomous Driving
• Multiagent Systems: Agent Communication

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Data transparency and provenance

Abstract

In this paper, the CD-TWINSAFE is introduced, a V2I-based digital twin for Autonomous Vehicles. The proposed architecture is composed of two stacks running simultaneously, an on-board driving stack that includes a stereo camera for scene understanding, and a digital twin stack that runs an Unreal Engine 5 replica of the scene viewed by the camera as well as returning safety alerts to the cockpit. The on-board stack is implemented on the vehicle side including 2 main autonomous modules; localization and perception. The position and orientation of the ego vehicle are obtained using on-board sensors. Furthermore, the perception module is responsible for processing 20-fps images from stereo camera and understands the scene through two complementary pipelines. The pipeline are working on object detection and feature extraction including object velocity, yaw and the safety metrics time-to-collision and time-headway. The collected data form the driving stack are sent to the infrastructure side through the ROS-enabled architecture in the form of custom ROS2 messages and sent over UDP links that ride a 4G modem for V2I communication. The environment is monitored via the digital twin through the shared messages which update the information of the spawned ego vehicle and detected objects based on the real-time localization and perception data. Several tests with different driving scenarios to confirm the validity and real-time response of the proposed architecture.

Problem

• Digital Twin
• Scene Understanding
• V2I
• Autonomous Vehicles
• Safety Alert

Innovation

• Digital Twin
• V2I Communication
• ROS2
• Scene Understanding
• Safety Alert