Video-Skill-CoT: Skill-based Chain-of-Thoughts for Domain-Adaptive Video Reasoning

2025-06-04

AI Summary

Existing video reasoning methods struggle to adapt generic chain-of-thought (CoT) prompting to domain-specific capabilities—such as event detection, spatial relation understanding, and sentiment analysis—due to their inherent heterogeneity and lack of skill specialization. Method: We propose a skill-aware CoT reasoning framework: (1) constructing a fine-grained skill taxonomy via clustering-based categorization; (2) designing a multi-step, skill-customized CoT annotation paradigm; and (3) implementing a lightweight adapter-driven collaborative learning architecture with skill-specific experts to enable modular, interpretable, and domain-adaptive reasoning. Contribution/Results: Evaluated on three mainstream video understanding benchmarks, our approach significantly outperforms strong baselines. It is the first to empirically validate that skill-decoupled modeling enhances both effectiveness and generalizability in video chain-of-thought reasoning.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Knowledge Representation and Reasoning: Common-Sense Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Recent advances in Chain-of-Thought (CoT) reasoning have improved complex video understanding, but existing methods often struggle to adapt to domain-specific skills (e.g., event detection, spatial relation understanding, emotion understanding) over various video content. To address this, we propose Video-Skill-CoT (a.k.a. Video-SKoT), a framework that automatically constructs and leverages skill-aware CoT supervisions for domain-adaptive video reasoning. First, we construct skill-based CoT annotations: we extract domain-relevant reasoning skills from training questions, cluster them into a shared skill taxonomy, and create detailed multi-step CoT rationale tailored to each video-question pair for training. Second, we introduce a skill-specific expert learning framework. Each expert module specializes in a subset of reasoning skills and is trained with lightweight adapters using the collected CoT supervision. We demonstrate the effectiveness of the proposed approach on three video understanding benchmarks, where Video-SKoT consistently outperforms strong baselines. We also provide in-depth analyses on comparing different CoT annotation pipelines and learned skills over multiple video domains.

Problem

• Adapting Chain-of-Thought reasoning to domain-specific video skills
• Automating skill-aware CoT supervision for video reasoning
• Improving video understanding across diverse domains with skill experts

Innovation

• Skill-based CoT annotations for video reasoning
• Skill-specific expert learning with adapters
• Domain-adaptive framework outperforms baselines