Allocation Before Ranking: Decoupled Token Compression for OmniLLMs

2026-08-02

AI Summary

This work addresses the imbalance in cross-modal resource allocation inherent in conventional multimodal large language models, which typically employ a unified top-K token compression strategy that disproportionately favors audio at the expense of video information retention. To overcome this limitation, the authors propose Macer, a novel approach that decouples resource allocation from token ranking for the first time. Macer assigns explicit, modality-specific budgets to audio and video prior to compression and performs normalized ranking on their respective shallow features, enabling modality-aware, decoupled token pruning. Notably, the method requires no additional training. At a 25% token retention rate, it preserves 98.7% and 97.3% of the original performance on Qwen2.5-Omni-7B and -3B, respectively, and achieves up to a 12.9-point improvement over the baseline on OmniVinci-9B.

Abstract

Token compression in OmniLLMs is typically posed as a single saliency-ranking problem: score each multimodal token, keep the top-K. We argue this abstraction is mis-specified. The same attention score simultaneously decides two things: how much retained capacity each modality receives, and which tokens within a modality are kept. A shared top-K rule therefore inherits this audio-favoring allocation prior, spending retained capacity on audio before video tokens have a chance to compete. We propose Macer, a training-free compressor that first assigns explicit audio and video budgets, then performs allocation-normalized ranking within each modality at modality-specific shallow layers. Macer significantly reduces token cost while preserving accuracy across audio-grounded, audio--video joint, visual-dominant, and video-centric benchmarks. At 25 % retention, Macer preserves 98.7 % of full-token performance on Qwen2.5-Omni-7B and 97.3 % on Qwen2.5-Omni-3B. On Qwen2.5-Omni-7B, this 25 % setting reaches OmniZip-level performance at 45 % retention while using lower FLOPs. On OmniVinci-9B, the same allocation-before-ranking principle improves over shared top-K ranking by up to 12.9 points.

Problem

• token compression
• multimodal allocation
• OmniLLMs
• modality bias
• saliency ranking

Innovation

• token compression
• modality allocation
• OmniLLMs
• training-free
• decoupled ranking