ActiveLang: Active Open-Vocabulary 3D Mapping with Semantic-Uncertainty-Guided Exploration

2026-10-07

AI Summary

This study addresses the challenge of enabling robots to achieve open-vocabulary semantic understanding and efficiently construct language-annotated 3D maps in unknown environments. To this end, we propose a novel active exploration strategy driven by semantic uncertainty, which leverages information gain to guide path planning. Furthermore, by integrating dual Gaussian splatting with an online language feature adaptation algorithm, our approach compresses language features into a compact dual-Gaussian representation, facilitating the joint reconstruction of geometry, appearance, and semantics. The proposed method significantly enhances mapping efficiency while maintaining low memory overhead. Extensive experiments on the Replica and ScanNet++ datasets demonstrate that our approach substantially outperforms existing baselines under limited observations, achieving more efficient 2D and 3D open-vocabulary segmentation.

Abstract

As robots increasingly assist humans with diverse tasks, they need both geometric and semantic understanding of their surroundings. Moreover, robots often operate in unfamiliar environments and take on new tasks without knowing the relevant concepts ahead of time. This motivates language-annotated 3D maps that support open-vocabulary scene understanding and human-robot interaction. We introduce ActiveLang, an autonomous system for active open-vocabulary 3D mapping with semantic-uncertainty-guided exploration. ActiveLang performs online language-feature adaptation on a compact dual-Gaussian representation to jointly reconstruct scene geometry, appearance, and open-vocabulary semantics with modest memory overhead. Its planner efficiently selects informative viewpoints, enabling effective mapping with fewer observations and lower computational cost. Experiments on Replica and ScanNet++ demonstrate substantial improvements in 2D and 3D open-vocabulary segmentation over both online and offline baselines, highlighting that actively exploring scenes builds language-annotated 3D maps more efficiently.

Problem

• open-vocabulary 3D mapping
• semantic understanding
• language-annotated 3D maps
• active exploration
• robot navigation

Innovation

• Open-Vocabulary 3D Mapping
• Semantic Uncertainty
• Dual-Gaussian Representation
• Active Exploration
• Language-Feature Adaptation