OpenTwinMap: An Open-Source Digital Twin Generator for Urban Autonomous Driving

2025-11-26

AI Summary

Existing urban digital twin tools exhibit tight coupling with specific simulators (e.g., CARLA) and rely heavily on C++ plugins, resulting in poor flexibility and high extension costs. Method: This paper proposes a modular, extensible Python framework that decouples underlying engine dependencies and supports dual-platform operation—Unreal Engine and CARLA. It integrates LiDAR point clouds with OpenStreetMap data to enable semantic-segmentation-driven automatic generation of static scene assets, covering road meshing, terrain modeling, and cross-format conversion. Contribution/Results: Compared to conventional approaches, the framework enables parallel processing, rapid prototyping, and seamless cross-city transfer, substantially lowering the barrier for customized deployment. The complete implementation is open-sourced, facilitating autonomous driving simulation validation and generative world model research.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Digital twins of urban environments play a critical role in advancing autonomous vehicle (AV) research by enabling simulation, validation, and integration with emerging generative world models. While existing tools have demonstrated value, many publicly available solutions are tightly coupled to specific simulators, difficult to extend, or introduce significant technical overhead. For example, CARLA-the most widely used open-source AV simulator-provides a digital twin framework implemented entirely as an Unreal Engine C++ plugin, limiting flexibility and rapid prototyping. In this work, we propose OpenTwinMap, an open-source, Python-based framework for generating high-fidelity 3D urban digital twins. The completed framework will ingest LiDAR scans and OpenStreetMap (OSM) data to produce semantically segmented static environment assets, including road networks, terrain, and urban structures, which can be exported into Unreal Engine for AV simulation. OpenTwinMap emphasizes extensibility and parallelization, lowering the barrier for researchers to adapt and scale the pipeline to diverse urban contexts. We describe the current capabilities of the OpenTwinMap, which includes preprocessing of OSM and LiDAR data, basic road mesh and terrain generation, and preliminary support for CARLA integration.

Problem

• Generates high-fidelity 3D urban digital twins for autonomous driving simulation
• Overcomes limitations of existing tools that are inflexible and hard to extend
• Processes LiDAR and OpenStreetMap data to create exportable environment assets

Innovation

• Python-based open-source framework for urban digital twins
• Ingests LiDAR and OpenStreetMap data for semantic segmentation
• Emphasizes extensibility and parallelization for diverse urban contexts