WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation

2026-07-25

AI Summary

This work addresses the inefficiency of large vision-language models in long video understanding, where computational complexity grows quadratically with sequence length due to visual token processing. Existing compression methods are largely confined to the spatial domain and struggle to disentangle structural redundancy from semantic details. To overcome this limitation, we propose WaveZip, a novel framework that introduces discrete wavelet transform (DWT) into video token compression, enabling spatiotemporally adaptive compression in both signal and frequency domains. Specifically, one-dimensional DWT handles inter-frame temporal redundancy, while two-dimensional DWT, modulated by query-aware saliency, preserves critical spatial information and dynamically allocates the token budget. WaveZip is plug-and-play and requires no task-specific training; it retains 99.6% of the original model performance under a 10× compression ratio, significantly outperforming current state-of-the-art approaches.

Abstract

Existing Large Vision-Language Models (LVLMs) struggle with long-form video understanding due to the quadratic computational cost of visual tokens. While recent efficient methods attempt to compress tokens via hard pruning or uniform merging, they operate strictly in the spatial feature domain, where robust structural context and discriminative semantic details are inherently entangled. In this work, we propose WaveZip, a joint signal-frequency-domain framework for efficient video inference. Driven by the insight that temporal redundancy resides in low-pass approximation scales while spatial saliency strongly correlates with high-frequency components, WaveZip leverages Discrete Wavelet Transforms (DWT) to disentangle these signals. Temporally, it employs 1D DWT to analyze query-frame relevance, and the resulting high-frequency coefficients are further gated by inter-frame differences, with both signals jointly driving the dynamic allocation of a precise frame-level token budget. Spatially, a 2D DWT decomposes features into low-frequency approximations and high-frequency detail components, where the high-frequency coefficients are modulated within query-salient regions to regulate spatial reconstruction. Importantly, WaveZip requires no task-specific training and can be seamlessly integrated into off-the-shelf LVLMs to boost inference efficiency. Extensive experiments on long video understanding benchmarks demonstrate that WaveZip retains 99.6% of the full performance under an extreme 10x compression ratio, consistently outperforming state-of-the-art methods.

Problem

• video token condensation
• long-form video understanding
• computational efficiency
• spatial-temporal redundancy
• vision-language models

Innovation

• Wavelet Transform
• Token Condensation
• Space-Time Decoupling
• Video Understanding
• Efficient Inference