Stable Scores, Unstable Answers: Frame Phase and Option Order in Video Multiple-Choice Evaluation

2026-10-06

AI Summary

This study addresses the instability of answers caused by frame sampling phase shifts and option order bias in the evaluation of video multimodal models. It is the first to quantify the impact of half-phase shifts on answer consistency and proposes PHASEFUSION, a method that enhances stability through multi-phase grid decoding and posterior probability fusion. Additionally, a single-step answer marginality metric is introduced to detect option order sensitivity. Experimental results demonstrate that, while maintaining accuracy, the proposed approach reduces the answer variation rate induced by phase shifts from 18.2% to 10.1%, effectively improving the robustness and reliability of video question-answering evaluation.

Abstract

Video-language models are ranked by multiple-choice accuracy on frames from a uniform grid. The grid has two parameters, a rate and a phase, and benchmarks report only the rate. The phase moves answers: two deployed samplers differing only by a half-step phase offset answer 23.6% of questions differently while scoring within a point, and across four releases from two families shifting only the phase changes roughly one answer in five after controlling option order. PHASEFUSION decodes three offset grids and averages the option posteriors. The grids are the polyphase components of the dense grid. Fusion matches a 32-frame single pass in accuracy within a prespecified margin (logit-scored) and cuts the answers a half-step shift of all three grids changes from 18.2% to 10.1%. Option order, which changes only the presentation, is flagged instead by a one-pass answer margin. Report the phase convention with the budget, or marginalize it.

Problem

• video-language models
• multiple-choice evaluation
• frame sampling phase
• option order
• evaluation stability

Innovation

• Video-language models
• Phase offset
• PHASEFUSION
• Polyphase components
• Multiple-choice evaluation