Failure Modes for Deep Learning-Based Online Mapping: How to Measure and Address Them

2026-03-20

AI Summary

This work addresses the limited generalization of online mapping models in unfamiliar environments, primarily attributed to the coupling between geometric overfitting and feature memorization. To disentangle these factors, the authors propose a method that constructs geographically and geometrically controlled validation subsets and introduces a threshold-free Fréchet distance–based reconstruction statistic to quantify localization fidelity and geometric overfitting. Innovatively, they employ Minimum Spanning Tree (MST) metrics to assess geometric diversity in the training set and devise a sparsification strategy to enhance data efficiency. Experiments on nuScenes and Argoverse 2 demonstrate that training on geometrically diverse and balanced datasets significantly improves model generalization and the reliability of evaluation.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: Localization, Mapping, and Navigation

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Deep learning-based online mapping has emerged as a cornerstone of autonomous driving, yet these models frequently fail to generalize beyond familiar environments. We propose a framework to identify and measure the underlying failure modes by disentangling two effects: Memorization of input features and overfitting to known map geometries. We propose measures based on evaluation subsets that control for geographical proximity and geometric similarity between training and validation scenes. We introduce Fréchet distance-based reconstruction statistics that capture per-element shape fidelity without threshold tuning, and define complementary failure-mode scores: a localization overfitting score quantifying the performance drop when geographic cues disappear, and a map geometry overfitting score measuring degradation as scenes become geometrically novel. Beyond models, we analyze dataset biases and contribute map geometry-aware diagnostics: A minimum-spanning-tree (MST) diversity measure for training sets and a symmetric coverage measure to quantify geometric similarity between splits. Leveraging these, we formulate an MST-based sparsification strategy that reduces redundancy and improves balancing and performance while shrinking training size. Experiments on nuScenes and Argoverse 2 across multiple state-of-the-art models yield more trustworthy assessment of generalization and show that map geometry-diverse and balanced training sets lead to improved performance. Our results motivate failure-mode-aware protocols and map geometry-centric dataset design for deployable online mapping.

Problem

• failure modes
• online mapping
• generalization
• overfitting
• map geometry

Innovation

• failure mode analysis
• map geometry overfitting
• Fréchet reconstruction statistics
• minimum-spanning-tree sparsification
• generalization diagnostics