Faster Training, Fewer Labels: Self-Supervised Pretraining for Fine-Grained BEV Segmentation

2026-02-20

AI Summary

This work addresses the challenge of scaling BEV semantic segmentation for autonomous driving, which is hindered by the reliance on costly and inconsistent multi-camera ground-truth annotations. The authors propose a two-stage self-supervised pretraining approach: first, leveraging image-plane pseudo-labels generated by Mask2Former together with differentiable reprojection to learn cross-view consistent BEV features, augmented by a temporal consistency loss; second, fine-tuning the model using only 50% of the annotated data. This study presents the first integration of differentiable reprojection and image-view pseudo-labels for BEV self-supervised pretraining, substantially reducing annotation dependency while enhancing model transferability. On the nuScenes benchmark, the method achieves a 2.5-point mIoU improvement with half the labeled data and reduces total training time by two-thirds.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Unsupervised & Self-Supervised Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Dense Bird's Eye View (BEV) semantic maps are central to autonomous driving, yet current multi-camera methods depend on costly, inconsistently annotated BEV ground truth. We address this limitation with a two-phase training strategy for fine-grained road marking segmentation that removes full supervision during pretraining and halves the amount of training data during fine-tuning while still outperforming the comparable supervised baseline model. During the self-supervised pretraining, BEVFormer predictions are differentiably reprojected into the image plane and trained against multi-view semantic pseudo-labels generated by the widely used semantic segmentation model Mask2Former. A temporal loss encourages consistency across frames. The subsequent supervised fine-tuning phase requires only 50% of the dataset and significantly less training time. With our method, the fine-tuning benefits from rich priors learned during pretraining boosting the performance and BEV segmentation quality (up to +2.5pp mIoU over the fully supervised baseline) on nuScenes. It simultaneously halves the usage of annotation data and reduces total training time by up to two thirds. The results demonstrate that differentiable reprojection plus camera perspective pseudo labels yields transferable BEV features and a scalable path toward reduced-label autonomous perception.

Problem

• BEV segmentation
• self-supervised pretraining
• annotation efficiency
• autonomous driving
• fine-grained road marking

Innovation

• self-supervised pretraining
• BEV segmentation
• differentiable reprojection
• pseudo-labeling
• annotation-efficient learning