OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models

2026-07-25

AI Summary

This work addresses the critical limitation of existing multimodal large language models, whose token compression methods often discard essential cross-modal temporal misalignment information due to reliance on single-modality guidance. To overcome this, we propose OmniScope, a training-free token compression framework that, for the first time, decouples audio and visual modalities during compression. OmniScope employs queries as shared semantic anchors to independently assess modality-specific relevance and allocate dedicated token budgets. On the visual side, it adopts an anchor-difference pruning strategy to preserve both global context and dynamic changes; on the audio side, it merges tokens per second to reduce redundancy while maintaining temporal continuity. Evaluated across four audio-visual benchmarks and two Qwen2.5-Omni model scales, OmniScope achieves up to 3.53× prefill acceleration and over 15% memory savings at a 25% token retention rate, with only a 0.35-point average accuracy drop—significantly outperforming current approaches.

Abstract

Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and video relevance often peaks at different moments. This cross-modal salience mismatch makes unidirectional guidance prone to discarding answer-critical cues under aggressive compression. We propose OmniScope, a training-free token compression framework that uses the query as a shared semantic anchor while estimating relevance separately for audio and video. OmniScope allocates modality-specific token budgets, prunes visual tokens with an anchor-delta strategy that preserves both global context and temporal changes, and merges audio tokens within each second to reduce redundancy while maintaining temporal continuity. Across four audio-video benchmarks and two Qwen2.5-Omni model scales, OmniScope achieves the best average accuracy across all compression settings. At 25% overall token retention, it delivers up to 3.53x prefill speedup and more than 15% GPU memory reduction, with only a 0.35-point drop in average accuracy. These results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates. The code is available at https://github.com/MAC-AutoML/OmniScope.

Problem

• token compression
• omnimodal large language models
• cross-modal salience mismatch
• modality-decoupled
• audio-video relevance

Innovation

• modality-decoupled
• token compression
• omnimodal LLM
• cross-modal salience
• training-free