M3TR: Generalist HD Map Construction with Variable Map Priors

2024-11-15

arXiv.org

AI Summary

High-definition (HD) maps suffer from rapid obsolescence and exhibit heterogeneous prior knowledge—ranging from missing or outdated maps to semantically inconsistent representations—posing significant challenges for robust online map updating. Method: We propose the first general-purpose online HD map construction framework compatible with arbitrary prior map types. Our approach introduces Multi-Mask Mask Transformer (M3TR), featuring dynamic query modeling and cross-prior joint training to enable robust fusion of diverse priors. We further establish the first real-world, semantically diverse evaluation benchmark with heterogeneous priors and develop a ground-truth correction generation pipeline based on Argoverse 2 and nuScenes. Contribution/Results: Experiments demonstrate a 4.3% mAP improvement over standard benchmarks; remarkably, a single unified model matches the performance of specialized expert models tailored to individual prior types. The code is publicly released and production-ready.

Technology Category

• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Autonomous vehicles require road information for their operation, usually in form of HD maps. Since offline maps eventually become outdated or may only be partially available, online HD map construction methods have been proposed to infer map information from live sensor data. A key issue remains how to exploit such partial or outdated map information as a prior. We introduce M3TR (Multi-Masking Map Transformer), a generalist approach for HD map construction both with and without map priors. We address shortcomings in ground truth generation for Argoverse 2 and nuScenes and propose the first realistic scenarios with semantically diverse map priors. Examining various query designs, we use an improved method for integrating prior map elements into a HD map construction model, increasing performance by +4.3 mAP. Finally, we show that training across all prior scenarios yields a single Generalist model, whose performance is on par with previous Expert models that can handle only one specific type of map prior. M3TR thus is the first model capable of leveraging variable map priors, making it suitable for real-world deployment. Code is available at https://github.com/immel-f/m3tr

Problem

• Develops M3TR for HD map completion using live sensor data.
• Addresses outdated offline HD maps for autonomous vehicles.
• Proposes a benchmark and handles all map changes effectively.

Innovation

• M3TR: Generalist model for HD map completion
• Handles all map changes, outperforms Expert models
• Utilizes prior HD map elements, optimizes query designs