Perception First: A Frontier Native-Video Model with Self-Consistency for Implicit Video Question Answering

2026-05-31

AI Summary

This study addresses implicit video question answering, a task requiring models to reason about latent information—such as spatial layout, motion, depth, viewpoint, and social causality—across discontinuous frames, which cannot be resolved from single-frame observations alone. The authors systematically evaluate prominent video foundation models, including Qwen2.5-VL, InternVL3, and Video-R1, alongside test-time strategies like self-consistency and prompt engineering. Their findings reveal that performance is primarily constrained by low-level visual perception capabilities—particularly in depth estimation, viewpoint understanding, and object counting—rather than by the sophistication of reasoning mechanisms. Experiments demonstrate that strong perceptual baselines combined with lightweight test-time denoising yield optimal results, while incorporating monocular depth cues unexpectedly reduces accuracy by 5.8%, underscoring that enhancing perceptual fidelity is more effective than designing complex reasoning pipelines.

Abstract

We describe our submission to the VRR Challenge @ CVPR 2026, built on the \emph{ImplicitQA} / \emph{VRR-QA} benchmark~\cite{implicitqa}: multiple-choice video question answering in which answers are deliberately \emph{not} observable in any single frame and must be inferred from spatial layout, motion, depth, viewpoint, causality, and social context across discontinuous frames of creative video. We conduct a systematic, training-free study spanning open-source Video-LMMs (Qwen2.5-VL~\cite{qwen25vl}, Qwen3-VL~\cite{qwen3vl}, InternVL3, Gemma-3, and the RL-tuned video reasoners Video-R1~\cite{videor1} and VideoChat-R1.5~\cite{videochatr15}) and a battery of inference-time strategies (chain-of-thought, question decomposition, describe-then-reason cascades, audio transcripts, spatial state prompting, self-consistency~\cite{selfconsistency}, multi-model ensembling, and category routing). Our central finding is that this benchmark is \emph{perception-bound rather than reasoning-bound}: reasoning-side augmentations are neutral-to-harmful, whereas base-model perceptual capability and lightweight test-time denoising are the only reliable levers. A per-category error analysis localizes the difficulty to low-level perception -- relative depth, viewpoint, and counting are the hardest categories, while causal and social reasoning are nearly solved -- and a prompt that explicitly injects monocular depth cues to attack the weakest category \emph{lowers} test accuracy by $5.8$ points, confirming that the model needs a better \emph{percept}, not a better \emph{procedure}.

Problem

• Implicit Video Question Answering
• Perception-bound
• Video Reasoning
• Low-level Perception
• VRR-QA

Innovation

• Perception First
• Implicit Video Question Answering
• Self-Consistency
• Video-LMMs
• Perceptual Bottleneck