EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception

2026-09-29

AI Summary

This study addresses the fine-grained perception challenges in high-resolution scenarios, namely visual token redundancy and context loss caused by local cropping. We propose a plug-and-play, lightweight evidence-adaptive framework that pioneers the use of human visual search trajectories to supervise evidence density, thereby guiding region re-reading and dynamic token allocation. Furthermore, a sparse coordinate bridging network is designed to integrate local features with the global scene. The proposed method can be deployed without retraining the host backbone network, ensuring strong compatibility. Experimental results demonstrate that our approach significantly improves average fine-grained accuracy across nine backbone models, outperforming purely global methods under identical token budgets.

Abstract

Fine-grained visual perception enables vision-language models to distinguish subtle attributes and ground their answers in visual evidence. In high-resolution scenes, processing the whole image at greater resolution spends visual tokens on irrelevant content, while isolated crops can lose the context needed to interpret the selected evidence. We introduce EviViT, a lightweight attachment that learns where a pretrained vision transformer should acquire detail. Human visual-search traces supervise a question-conditioned evidence density, which guides regional re-reading from the original pixels and the allocation of visual tokens. A sparse, coordinate-aware bridge then connects the regional features to the global scene, allowing the host to interpret precise evidence in context. Learned with the host backbone frozen, the attachment serves both the base model and compatible post-trained descendants without refitting. Experiments across nine hosts show consistent gains in average fine-grained accuracy. Matched-budget comparisons further show that EviViT outperforms global-only processing at every tested token ceiling while using fewer visual tokens.

Problem

• fine-grained visual perception
• vision-language models
• visual token allocation
• high-resolution scenes
• context preservation

Innovation

• Evidence-Adaptive Vision Transformer
• Fine-Grained Perception
• Human Visual-Search Traces
• Coordinate-Aware Bridge
• Token-Efficient