Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs

2026-08-04

AI Summary

This work addresses the challenges faced by multimodal large language models in reasoning, where conflation of perceptual ambiguity and logical uncertainty often leads to high computational costs from explicit chain-of-thought reasoning, visual hallucinations, or unstable performance with untrained implicit methods. The authors propose a training-free inference strategy that decouples perception from reasoning and introduces, for the first time, a vision-text attention ratio as a cognitive state indicator. This metric dynamically determines the cognitive focus and enables adaptive switching between explicit textual generation and implicit continuous-space reasoning. The approach achieves dynamic synergy between explicit and implicit reasoning in multimodal settings, significantly improving accuracy across multiple benchmarks while reducing autoregressive steps and latency, thereby balancing both efficiency and effectiveness.

Abstract

Reasoning in Multimodal Large Language Models (MLLMs) requires both fine-grained visual perception and rigorous logical deduction. Explicit text-based Chain-of-Thought (CoT) is computationally expensive and prone to visual hallucinations, while existing latent reasoning methods typically require costly training. Furthermore, directly adapting training-free LLM reasoning mechanisms to the multimodal setting yields unstable performance. We identify that this failure stems from their reliance on token-level entropy, which fundamentally conflates perceptual ambiguity (e.g., unclear visual details) with logical uncertainty (e.g., complex reasoning steps). To overcome this bottleneck, we present a novel training-free inference strategy for MLLMs that explicitly decouples perception and reasoning. We propose a novel metric, the vision-to-text attention ratio, to dynamically gauge the model's cognitive focus. Guided by this metric, our proposed framework, Attention-Guided Switching (AGS), adaptively triggers latent reasoning for perceptual tokens to preserve high-fidelity visual information in the continuous space, while enforcing explicit text generation for logical tokens to maintain structural anchoring. Extensive experiments demonstrate that our method achieves state-of-the-art performance, significantly improving both accuracy and inference efficiency by reducing autoregressive steps and latency. Code is released at https://github.com/swordAndSnow/MM26-AGS.

Problem

• Multimodal Large Language Models
• Chain-of-Thought Reasoning
• Perception-Reasoning Decoupling
• Visual Hallucination
• Training-Free Inference

Innovation

• training-free
• attention-guided switching
• multimodal reasoning
• vision-to-text attention ratio
• latent reasoning