SVoT: State-aware Visualization-of-Thought for Spatial Reasoning via Reinforcement Learning

2026-06-10

AI Summary

This work addresses the fragility of multimodal large language models in multi-hop spatial reasoning, which stems from their inability to reliably model intermediate states and their transitions. To overcome this limitation, the authors propose a State-aware Visualized Chain-of-Thought (SVoT) framework that explicitly formulates state transitions as verifiable interleaved text-visual reasoning chains and introduces transition-aware supervisory signals. The model is trained via reinforcement learning using Group Relative Policy Optimization (GRPO) with a fine-grained reward mechanism. Additionally, the study introduces a new benchmark environment encompassing multi-object interactions and numerical reasoning to enable systematic evaluation. Experimental results demonstrate that SVoT achieves state-of-the-art performance across multiple domains, with absolute accuracy improvements of up to 65% on out-of-distribution test sets.

Abstract

Spatial reasoning remains a challenge for Multimodal Large Language Models (MLLMs), as it requires reliable multi-hop inference over both intermediate states and state transitions. Current studies often leave intermediate states unverified and treat state transitions as implicit processes, which limits reliability in multi-hop spatial reasoning. To address this, we propose State-aware Visualization-of-Thought (SVoT), a reinforcement learning framework that generates interleaved, verifiable intermediate states and visualizations. SVoT integrates transition reasoning chains into the generation processes, enabling the model to verify action preconditions and effects through interleaved textual and visual reasoning. We train SVoT via Group Relative Policy Optimization (GRPO), instantiating verification through reward design and evaluating the efficacy of different fine-grained rewards. As existing benchmarks reduce state transitions to single-variable updates, substantially simplifying the problems, we establish five domains by extending classical environments and introducing two novel domains, Pacman and Gather, that require multi-object interactions and numerical reasoning. These domains support systematic evaluation of multi-hop spatial reasoning with quantitative verification of generated intermediate states and transition reasoning. SVoT with transition-aware supervision achieves state-of-the-art performance across the introduced domains, yielding up to a 65% absolute accuracy gain on out-of-distribution test sets.

Problem

• spatial reasoning
• multimodal large language models
• intermediate states
• state transitions
• multi-hop inference

Innovation

• State-aware Visualization-of-Thought
• spatial reasoning
• reinforcement learning
• intermediate state verification
• transition reasoning