Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQA

2026-07-30

AI Summary

This work addresses the inefficiency of existing high-resolution visual question answering (HR-VQA) methods, which suffer from redundant image cropping or re-encoding and neglect the dilution of fine-grained intermediate evidence in subsequent processing. The authors propose Thinking-Once, a training-free, single-pass framework that preserves critical entities and compact background context through question-conditioned attention reshaping and token selection within a single visual forward pass. Crucially, it routes intermediate-layer evidence directly to higher layers without additional training or repeated visual processing. This approach reveals, for the first time, that the performance bottleneck in HR-VQA stems from evidence dilution rather than insufficient input resolution. Evaluated across five multimodal large language models, Thinking-Once improves average scores by 3.1, 3.0, and 2.7 points on V*Bench, HRBench-4K, and HRBench-8K, respectively, reduces peak memory usage by approximately 4 GB, accelerates inference by 97.2% over DeepScan, and achieves an average cross-benchmark score of 82.7.

Abstract

High-resolution visual question answering (HR-VQA) is often treated as a problem of insufficient evidence acquisition, where failing multimodal large language models must inspect images again through cropping, re-encoding, or multi-round search. We show that this view is incomplete: in many cases, fine-grained evidence has already survived visual encoding and become identifiable and influential within an intermediate-layer routing window, but is later diluted before answer generation. We propose Thinking-Once, a \textbf{training-free, single-visual-pass} evidence-routing method that reconstructs question-conditioned attention at this window, preserves core entity tokens and compact background context, and routes this evidence to later layers without extra visual encoding. Across five base models, Thinking-Once consistently improves or matches the corresponding base setting, increasing the average scores on V$^*$Bench, HRBench-4K, and HRBench-8K by \textit{+3.1}, \textit{+3.0}, and \textit{+2.7} points while reducing the average peak memory by about 4,GB. On Qwen2.5-VL-7B, it improves the three benchmarks by \textit{+9.9}, \textit{+4.6}, and \textit{+5.5} points, raising the cross-benchmark mean from 72.5 to 79.1. With the ZwZ-8B base model, Thinking-Once reaches a mean score of 82.7. Against 11 open-source HR-VQA baselines, it obtains the best or tied-best score on all three benchmark averages and the best overall mean; for example, compared with DeepScan, it reduces V$^*$Bench inference time by \textbf{97.2\%} while improving the cross-benchmark mean from 77.8 to 79.1. These results show that HR-VQA can be improved by routing already encoded evidence rather than repeatedly acquiring new visual inputs. Code is available in the appendix.

Problem

• High-Resolution Visual Question Answering
• Evidence Acquisition
• Intermediate-Layer Representation
• Multimodal Large Language Models
• Visual Encoding

Innovation

• evidence routing
• single-visual-pass
• training-free
• intermediate-layer attention
• high-resolution VQA