Q-Fold: Query-Aware Focus-Context Spatio-Temporal Folding for Long Video Understanding

2026-06-10

AI Summary

Long-form video understanding faces significant challenges due to high computational costs and the inherent trade-off between temporal coverage and fine-grained detail preservation. This work proposes a training-free, query-guided input construction framework that departs from conventional frame-centric paradigms. By leveraging query-aware temporal segmentation, the method retains semantically relevant segments as high-fidelity Focus frames while compressing irrelevant portions into order-preserving contextual layouts. This strategy simultaneously preserves critical visual details, broad temporal scope, and local temporal continuity under a fixed visual budget. Integrating heterogeneous Focus-Context representations with multimodal large language models, the approach achieves substantial performance gains across four long-video benchmarks, with improvements of up to 9.1 percentage points, without increasing input token overhead.

Abstract

Long-video understanding remains challenging for multimodal large language models, because temporally extended videos often contain thousands of frames and are therefore expensive to process exhaustively. Existing methods usually construct compact visual inputs from long videos under a limited visual budget. However, most of them still follow a frame-centric paradigm and apply similar representations to retained content regardless of its importance. This makes it difficult to preserve both high-fidelity visual evidence and broad temporal coverage. To address this issue, we propose Q-Fold, a training-free input construction framework for long-video understanding. Instead of treating isolated frames as the basic modeling unit, Q-Fold operates on contiguous temporal segments and constructs a heterogeneous Focus--Context representation under query guidance. Query-relevant segments are preserved as high-fidelity Focus Frames, while less relevant segments are folded into chronology-preserving contextual layouts. In this way, Q-Fold preserves critical visual evidence and broad temporal coverage, while better maintaining local temporal continuity within short segments. Experiments on four long-video benchmarks with multiple Video-MLLMs show that Q-Fold consistently improves performance without increasing the input budget. Notably, it achieves gains of up to 9.1 percentage points on an ultra-long video benchmark. Code will be made publicly available.

Problem

• long-video understanding
• multimodal large language models
• visual input compression
• temporal coverage
• frame-centric paradigm

Innovation

• query-aware
• focus-context representation
• spatio-temporal folding
• long-video understanding
• training-free framework