MetaSampling: Making Frame Samplers Efficient for Long-Video Question Answering

2026-09-27

AI Summary

This study addresses the inefficiency of frame sampling in long video question answering caused by fixed global budgets. To overcome this limitation, we propose a training-free, plug-and-play dynamic frame selection strategy. Specifically, this work pioneers the integration of a dynamic budget adjustment mechanism atop existing selectors to transcend fixed constraints, coupled with a training-free meta-sampling algorithm that optimizes the frame sequences fed into multimodal large language models. Experimental results demonstrate that the proposed strategy reduces the number of input frames by an average of 8.9% while simultaneously improving accuracy across most configurations, thereby significantly enhancing computational efficiency.

Abstract

Frame selection is an important component of long-video question answering (VQA) with Multimodal Large Language Models (MLLMs). Existing frame-selection methods improve over simple top-$k$ embedding retrieval and uniform sampling, but are typically applied under a fixed global selection budget. We introduce \textbf{MetaSampling}, a training-free, plug-and-play sampling strategy that can be applied on top of existing frame selectors. MetaSampling improves downstream VQA efficiency by dynamically reducing the number of frames passed to the MLLM while preserving, and in some cases improving, answer accuracy. We evaluate MetaSampling across 36 paired frame-selector--MLLM-backbone--VQA-benchmark configurations. MetaSampling reduces the number of selected frames in all 36 configurations and improves accuracy in 25 of them, yielding an average frame reduction of $8.9\%$ while slightly improving accuracy overall.

Problem

• Long-Video Question Answering
• Frame Selection
• Multimodal Large Language Models
• Efficiency

Innovation

• MetaSampling
• Long-Video Question Answering
• Frame Selection
• Multimodal Large Language Models
• Training-free