VLM-GLoc: Vision-Language Model Enhanced Monte Carlo Localization for Robust Semantic Global Localization in Cluttered Quasi-Static Environments

2026-05-28

AI Summary

This work addresses the challenge of insufficient robustness in global localization for mobile robots operating in geometrically aliased, semantically complex, and visually cluttered indoor environments such as supermarkets and offices. The authors propose a hierarchical semantic Monte Carlo localization method that, for the first time, integrates an open-vocabulary vision-language model (VLM) as a unified semantic observation front-end. A novel inverse semantic proposal mechanism leverages text-to-map retrieval to initialize particles effectively. The key innovation lies in exploiting the VLM to extract highly discriminative semantic features, implicitly filter out dynamic or ambiguous objects, and enhance data association through permanence reasoning. Evaluated in real-world supermarket and laboratory settings, the approach achieves localization success rates of 70% and 74%, respectively, significantly outperforming conventional geometric and domain-specific baseline methods.

Abstract

Global localization in geometrically aliased, quasi-static environments such as grocery stores, offices, schools, and hospitals poses a significant challenge for mobile robots. Grocery stores with parallel aisles and a long tailed distribution of products, as well as offices and labs with repetitive furniture such as chairs, desks, monitors, and doors, exemplify common indoor environments that present geometric and even semantic ambiguity. Traditional approaches rely either on distinct geometric features or on domain-specific vision pipelines that struggle with long-tail semantic distributions and transient visual clutter. We present VLM-GLoc, a method for hierarchical semantic Monte Carlo Localization (MCL) that leverages open-vocabulary Vision-Language Models (VLMs) as a unified semantic observation front-end. We hypothesize a three-fold benefit from VLMs: (1) extracting highly discriminative rich text features, (2) implicit quality filtering of blurry or dynamic objects, and (3) permanence reasoning for targeted data augmentation. We introduce an inverse semantic proposal mechanism that seeds particles via text-to-map retrieval. Evaluated across two real-world environments with different characteristics and two different platforms: a 3,500 sq. ft. grocery store with a cellphone and a 3,700 sq. ft. lab space with a quadruped, VLM-GLoc achieves 70% and 74% global localization success respectively, substantially outperforming traditional geometry-only and domain-specific baselines.

Problem

• global localization
• geometric aliasing
• quasi-static environments
• semantic ambiguity
• mobile robots

Innovation

• Vision-Language Model
• Monte Carlo Localization
• Semantic Global Localization
• Open-Vocabulary Perception
• Inverse Semantic Proposal