SepPrune:A Separator-based Pruning Framework for Efficient Multimodal Large Language Models

2026-07-28

AI Summary

This work addresses the high computational cost incurred by the large number of visual tokens generated from high-resolution image inputs in multimodal large language models (MLLMs). Existing pruning methods either rely on cross-modal attention—preventing early-stage pruning—or require prohibitively expensive diversity evaluations. To overcome these limitations, the authors propose SepPrune, a training-free, plug-and-play pre-pruning framework that leverages the modality separator token’s role as an attentional bridge. By using the separator token’s attention scores as a unified query to rank and select visual tokens, and reusing the LLM’s built-in projection parameters for lightweight pruning, SepPrune achieves state-of-the-art performance. On Qwen2.5-VL-7B, it removes 80.2% of visual tokens while retaining 96.3% of the original accuracy.

Abstract

Recent multimodal large language models (MLLMs), such as Qwen2.5-VL and InternVL3, generate large numbers of vision tokens for high-resolution inputs, leading to substantial computational cost. Existing vision token pruning methods either depend on cross-modal attention and cannot prune before the prefill stage, or rely on diversity estimation with high computational overhead. We observe that attention scores from both vision and text tokens peak at modality separator tokens, suggesting that these separators bridge the two modalities. Based on this observation, we propose SepPrune, an efficient, training-free, plug-and-play pruning method that uses the separator token as a unified query to rank and select informative vision tokens. SepPrune reuses the LLM's built-in projection parameters and requires no architectural changes. Experiments on Qwen2.5-VL-7B show that SepPrune achieves state-of-the-art performance, retaining 96.3% of the original accuracy while removing 80.2% of vision tokens.

Problem

• vision token pruning
• multimodal large language models
• computational cost
• separator tokens
• efficient inference

Innovation

• separator token
• vision token pruning
• multimodal LLMs
• training-free
• efficient inference