DSI-Bench: A Benchmark for Dynamic Spatial Intelligence

2025-10-21

AI Summary

Current vision-language and vision-only models exhibit severe deficiencies in spatial relational reasoning for dynamic 3D scenes—particularly under coordinated motion between observers and objects. Method: We introduce the concept of *Dynamic Spatial Intelligence* and propose DSI-Bench, the first benchmark dedicated to dynamic spatial reasoning. It comprises nearly 1,000 videos and 1,700+ expert-annotated questions, systematically designed around nine spatiotemporally symmetric and disentangled motion patterns to enable the first principled, decoupled evaluation of egomotion versus object-motion reasoning. Annotation rigor and multi-dimensional motion modeling minimize model biases. Contribution/Results: Comprehensive evaluation across 14 state-of-the-art models reveals pervasive failures—including motion-source confusion, semantic bias, and misjudgment of relative spatial relations—highlighting critical gaps in dynamic spatial understanding. DSI-Bench establishes a reproducible, disentangled, and standardized evaluation paradigm for advancing research in dynamic spatial intelligence.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: Motion & Tracking
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Reasoning about dynamic spatial relationships is essential, as both observers and objects often move simultaneously. Although vision-language models (VLMs) and visual expertise models excel in 2D tasks and static scenarios, their ability to fully understand dynamic 3D scenarios remains limited. We introduce Dynamic Spatial Intelligence and propose DSI-Bench, a benchmark with nearly 1,000 dynamic videos and over 1,700 manually annotated questions covering nine decoupled motion patterns of observers and objects. Spatially and temporally symmetric designs reduce biases and enable systematic evaluation of models'reasoning about self-motion and object motion. Our evaluation of 14 VLMs and expert models reveals key limitations: models often conflate observer and object motion, exhibit semantic biases, and fail to accurately infer relative relationships in dynamic scenarios. Our DSI-Bench provides valuable findings and insights about the future development of general and expertise models with dynamic spatial intelligence.

Problem

• Evaluating models' ability to understand dynamic 3D spatial relationships
• Addressing limitations in reasoning about simultaneous observer and object motion
• Benchmarking dynamic spatial intelligence across various motion patterns

Innovation

• Introduces DSI-Bench benchmark for dynamic spatial intelligence
• Uses symmetric designs to reduce biases in evaluation
• Evaluates models on decoupled observer and object motion