SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models

2026-08-03

AI Summary

Current vision-language models exhibit limited performance in visual spatial reasoning and often rely on additional 3D priors or external encoders, which compromises their generalizability. To address this, this work proposes SpatioLM—a parameter-efficient, plug-and-play framework that enhances a model’s intrinsic spatial reasoning capabilities through a non-intrusive spatial vision module. The approach leverages pseudo-depth estimation and camera parameters as weak supervision signals, eliminating the need for external 3D priors. Without sacrificing general-purpose functionality, SpatioLM achieves substantial gains in spatial understanding, surpassing the 70-point threshold on VSI-Bench with a score of 71.6. It further demonstrates superior performance across diverse tasks involving spatial perception, comprehension, and embodied manipulation transfer.

Abstract

Vision-Language Models (VLMs) perform well on commonsense reasoning tasks but struggle with visual spatial reasoning. Most existing solutions introduce extra 3D prior inputs or external spatial encoders, which increase complexity and degrade the underlying VLMs' general-purpose capabilities after spatial fine-tuning. To this end, we propose a parameter-efficient \textit{\textbf{Spatio}-vision \textbf{L}anguage \textbf{M}odels (SpatioLM)}, that enhances spatial intelligence without extra 3D prior inputs or third-party spatial encoders. Concretely, we design a plug-and-play and non-invasive spatio-vision module that elicits the spatial knowledge inherent in VLMs. Furthermore, we innovatively leverage pseudo depth and camera information as supervision to guide the model in learning physically coherent representations. Extensive experiments show that SpatioLM achieves significant improvements in diverse tasks, including spatial perception and understanding while effectively limiting the degradation of general capabilities. Notably, the model achieves an impressive score of 71.6 on the VSI-Bench (the first model to surpass 70). In addition, it attains competitive performance when transferred to embodied manipulation tasks. Code is available at \href{https://github.com/xiaomi-research/spatio-lm}{\faGithub~spatio-lm}.

Problem

• spatial reasoning
• vision-language models
• 3D prior
• general-purpose capability
• visual spatial intelligence

Innovation

• SpatioLM
• spatial reasoning
• vision-language models
• pseudo depth supervision
• parameter-efficient learning