VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understanding

2026-06-03

AI Summary

This work addresses the limitations of existing video understanding datasets in supporting knowledge-intensive and deep reasoning tasks by introducing VideoKR, the first large-scale video corpus specifically designed for knowledge-driven multi-step reasoning. VideoKR comprises 145K domain-specific videos and 315K reasoning instances, accompanied by a dedicated evaluation benchmark, VideoKR-Eval. The dataset is constructed through a human-AI collaborative, skill-oriented generation pipeline that integrates human feedback, chain-of-thought annotations, and a training paradigm combining supervised fine-tuning (SFT) with GRPO to ensure high difficulty, diversity, and reliability. Experimental results demonstrate that models trained on VideoKR significantly outperform current approaches on knowledge-intensive video reasoning tasks while preserving strong general video understanding capabilities.

Abstract

We introduce VideoKR, the first large-scale training corpus specifically designed to strengthen knowledge- and reasoning-intensive video understanding. It comprises 315K video reasoning examples over 145K newly collected, CC-licensed, expert-domain videos. We develop a human-in-the-loop, skill-oriented example generation pipeline that targets progressively deeper video reasoning capabilities while ensuring the difficulty, diversity, and reliability of both the examples and their CoT rationales. We also curate VideoKR-Eval, a new expert-annotated benchmark where questions require genuine video understanding and knowledge-intensive reasoning rather than textual shortcuts. Our experiments show that, under a standard SFT$\rightarrow$GRPO pipeline, models post-trained on VideoKR outperform prior post-training approaches on knowledge-intensive video reasoning while remaining competitive on general video reasoning, highlighting data design as a key driver of progress in video reasoning. We further conduct comprehensive ablations to isolate the contributions of VideoKR, providing actionable insights for future work.

Problem

• video understanding
• knowledge-intensive reasoning
• reasoning-intensive
• video reasoning
• expert-domain videos

Innovation

• knowledge-intensive reasoning
• video understanding
• chain-of-thought (CoT)
• human-in-the-loop data generation
• video reasoning benchmark