Reinforcing Temporal Answer Grounding in Instructional Video via Candidate-Aware Causal Reasoning

2026-06-06

AI Summary

This work addresses the semantic gap and length mismatch between natural language queries and long instructional video segments by proposing a candidate-aware causal reasoning framework. The method first leverages vision-language pretraining to generate high-quality candidate segments, then employs a temporally logical reasoning module augmented with a rejection-based reward mechanism for robust inference. To further enhance long-video reasoning and retrieval capabilities, the framework incorporates Group Relative Policy Optimization (GRPO). Evaluated across six benchmark datasets, the proposed approach achieves state-of-the-art performance in mean Intersection over Union (mIoU) by a significant margin, demonstrating its effectiveness and strong generalization ability.

Abstract

The task of temporal answer grounding in instructional video (TAGV), which aims to locate precise video segments that respond to natural language queries, is increasingly important for direct video answer retrieval. This task remains challenging due to the need to comprehend semantically complex questions and to address the significant length mismatch between untrimmed videos and short target moments. Existing methods often suffer from sensitivity to irrelevant content or insufficient visual reasoning capabilities. To tackle these limitations, we propose a Candidate-Aware Causal Reasoning (CACR) framework. Our approach first employs a Visual-Language Pre-training based Candidate Selection (VBCS) algorithm to efficiently generate K candidate segments, then applies a temporal logic reasoning module enhanced by a rejection reward mechanism and optimized via Group Relative Policy Optimization (GRPO) for robust inference. Extensive experiments on six benchmarks demonstrate that our method achieves state-of-the-art performance in terms of mean Intersection-over-Union (mIoU), providing a new perspective for reasoning-based retrieval in long videos.

Problem

• temporal answer grounding
• instructional video
• video retrieval
• natural language query
• video segment localization

Innovation

• Temporal Answer Grounding
• Candidate-Aware Causal Reasoning
• Visual-Language Pre-training
• Rejection Reward Mechanism
• Group Relative Policy Optimization