🤖 AI Summary
To address the high computational cost and input redundancy of vision-language models (VLMs) in document understanding, this paper proposes an index-preserving lightweight token pruning framework. Methodologically, it introduces a novel integration of a binarized block-level text classifier with a max-pooling refinement mechanism: first performing coarse-grained filtering to remove non-text background regions, then recovering fragmented text blocks via max-pooling—thereby preserving spatial coherence and maintaining original positional indices. The approach requires no fine-tuning of the backbone model and performs efficient token reduction solely during preprocessing. Experiments on real-world document datasets demonstrate significant inference acceleration (≈40% FLOPs reduction) while retaining baseline accuracy on key tasks such as OCR-enhanced visual question answering. The core contribution is the first realization of high-fidelity, index-aligned token pruning for document images—establishing a practical new paradigm for deploying VLMs under resource constraints.
📝 Abstract
Recent progress in vision-language models (VLMs) has led to impressive results in document understanding tasks, but their high computational demands remain a challenge. To mitigate the compute burdens, we propose a lightweight token pruning framework that filters out non-informative background regions from document images prior to VLM processing. A binary patch-level classifier removes non-text areas, and a max-pooling refinement step recovers fragmented text regions to enhance spatial coherence. Experiments on real-world document datasets demonstrate that our approach substantially lowers computational costs, while maintaining comparable accuracy.