VCIFBench: Evaluating Complex Instruction Following for Video Understanding

2026-06-03

AI Summary

Existing video understanding benchmarks predominantly rely on simple prompts, which are insufficient for evaluating the instruction-following capabilities of multimodal large language models under complex output constraints. To address this gap, this work presents the first systematic benchmark for complex instruction following in video understanding, encompassing multidimensional constraints on content, format, style, and structure. We introduce a novel instruction generation approach that integrates benchmark-aware adaptation with direct video anchoring. Through a hybrid validation pipeline combining automatic and human evaluation, we construct high-quality DPO preference data along with a conflict-diagnostic subset. Experiments across ten mainstream models demonstrate that jointly satisfying multiple constraints remains highly challenging, yet DPO training based on our benchmark substantially enhances models’ ability to follow complex instructions.

Abstract

Multimodal large language models have made rapid progress in video understanding, yet existing benchmarks largely rely on simple prompts and provide limited evidence about whether models can satisfy explicit output constraints. We introduce VCIFBench, a benchmark for evaluating complex instruction following in video understanding. VCIFBench constructs constraint-rich instructions from both benchmark-adapted and directly video-grounded prompts, covering content, format, style, and structure requirements, and evaluates model outputs with a hybrid verification pipeline. The benchmark contains 306 satisfiable test instructions, a 540-pair DPO preference dataset, and a 30-item conflict diagnostic subset. Experiments on 10 MLLMs show that joint constraint satisfaction remains challenging. We further show that DPO training on VCIFBench data can improve instruction-following performance.

Problem

• complex instruction following
• video understanding
• multimodal large language models
• output constraints
• benchmark evaluation

Innovation

• complex instruction following
• video understanding
• multimodal large language models
• constraint satisfaction
• DPO training