🤖 AI Summary
Current multimodal large language models frequently suffer from motion hallucination in cross-video action comparison, struggling to accurately capture genuine kinematic differences. This work presents the first systematic characterization of motion hallucination along three dimensions—direction, attributes, and temporal structure—and introduces MotionHalluc, a benchmark comprising 553 video pairs and 1,540 fine-grained questions, to enable quantitative evaluation. The authors propose Perceive-Parse-Verify, a training-free method that translates natural language instructions into executable measurement queries and mitigates hallucination through explicit motion verification. Experiments reveal that motion hallucination is prevalent across mainstream models, while integrating measurement injection yields an average performance gain of 10.6%, underscoring the critical role of explicit motion measurement in improving model reliability.
📝 Abstract
Motion instruction generation in cross-video comparison aims to produce corrective feedback that describes the differences between a query and a reference motion. However, existing models often generate instructions that exhibit motion hallucinations, failing to reflect actual kinematic differences between paired videos. To systematically investigate these hallucinations, we introduce MotionHalluc, a dedicated benchmark for evaluating motion hallucinations in paired-video comparison. MotionHalluc comprises 1540 fine-grained questions over 553 video pairs, evaluating hallucinations along three core dimensions: (1)directional hallucination, (2)attributional hallucination, and (3)temporal hallucination. Extensive evaluations of state-of-the-art large multimodal models demonstrate high susceptibility to these hallucinations. Furthermore, we provide Perceive-Parse-Verify (PPV) as a training-free measurements extraction and verification baseline that converts candidate instructions into executable measurement queries and supplies kinematic measurements at inference time. Our results show that this simple measurements injection yields an average 10.6% performance gain across models, suggesting that motion reasoning with explicit quantitative measurements is a key factor in reducing hallucinations in cross-video comparison. Our code and dataset will be made publicly available upon acceptance.