ET-Prune: Evidence-Aware Dynamic Budgeting for Visual Token Pruning in Text-Rich MLLMs

2026-08-03

AI Summary

This work addresses the vulnerability of existing fixed-ratio visual token pruning methods to inadvertently discard critical OCR evidence in text-heavy multimodal tasks. The authors propose a training-free dynamic pruning framework that formulates token reduction as an evidence allocation problem. By leveraging partial query-key blocks from the decoder side to extract question-conditioned cues, the method dynamically sets a token retention floor based on evidence uncertainty and density, and employs a three-stage progressive intermediate mechanism for sample-adaptive pruning. Notably, it introduces an evidence-aware dynamic budgeting scheme alongside a text-region preservation strategy. Evaluated across six backbone-benchmark combinations, the approach consistently outperforms current methods—achieving gains of 1.80 and 0.68 percentage points over the strongest baseline on OCRBench-v2 while retaining approximately 50% of tokens, and attaining an accuracy of 0.8467 on MMBench v1.1, surpassing the original unpruned model.

Abstract

Visual token pruning reduces the inference cost of multimodal large language models, but a fixed token ratio is poorly matched to text-rich inputs. In OCR-centric tasks, decisive evidence can be a small number, label, or field whose relevance is specified by the question; indiscriminate pruning can erase that evidence while retaining visually salient but irrelevant regions. We present ET-Prune, a training-free framework that casts pruning as evidence allocation. It derives question-conditioned evidence from a decoder-side partial query-key block, safeguards text-like spatial regions, and converts evidence uncertainty and density into a sample-specific token floor. Three progressive middle-layer events then move the sequence toward this budget, retaining more tokens for diffuse or text-dense evidence and pruning concentrated evidence more aggressively. At the observed point estimates from one deterministic pass per configuration, ET-Prune leads or ties among pruned methods in all six backbone-benchmark comparisons at roughly half tokens. On OCRBench-v2, it leads the strongest pruned baselines by 1.80 and 0.68 percentage points on Qwen3-VL-8B and InternVL3.5-8B, respectively, while retaining about half of the visual tokens; on MMBench v1.1, it reaches 0.8467 circular exact-matching accuracy versus 0.8437 for Vanilla at 54.45% average visual-token retention. These results show a favorable observed quality-cost trade-off for evidence-aware dynamic budgeting in text-rich multimodal inference.

Problem

• visual token pruning
• text-rich MLLMs
• evidence preservation
• OCR-centric tasks
• dynamic budgeting

Innovation

• evidence-aware pruning
• dynamic token budgeting
• text-rich MLLMs
• training-free framework
• visual token pruning