Test-Time Noise Guided Adaptation for Realistic Autoregressive Video Generation

2026-07-17

AI Summary

This work addresses the challenge of error accumulation in autoregressive video diffusion models during long-horizon generation, which often causes trajectories to deviate from the true data manifold and terminate prematurely. To mitigate this issue, the authors propose TANGO, a novel test-time self-guidance mechanism that leverages the diffusion model itself as a discriminator. By dynamically aligning the predicted noise distribution through the forward noising process, TANGO optimizes the generation trajectory in real time. The method further incorporates isotropic Gaussian noise constraints and manifold anchoring to effectively avoid terminal failure points. Experimental results demonstrate that TANGO achieves an absolute improvement of 3.1% on VBench and reduces the Fréchet Video Distance by 28.3% on average for 15-second video generation.

Abstract

Autoregressive video diffusion models have enabled the generation of arbitrarily long videos by removing conditioning on future frames, thus greatly improving computational efficiency. Yet, they suffer from error accumulation over time, as the denoised sequence gradually drifts away from the conditioning distribution seen during training. Recent advances attempt to reduce this error by anchoring each generated frame to the learned manifold of real ones. However, even when all generated individual frames lie close to the real manifold, there are trajectories which the model lacks sufficient knowledge to continue without exiting it, thus reaching a terminal point. To prevent the model from being trapped in terminal points, we start from the hypothesis that for well-modeled future trajectories the distribution of the predicted noise should match the one of the forward noising process. To enforce such a prior at test time, we introduce Terminal points Avoidance through Noise Guided Optimization (TANGO), which uses the diffusion model as a critic of its own outputs, by predicting one step forward and requiring an isotropic Gaussian noise prediction. We use the deviation from this expected noise distribution to search for an alternative trajectory that does not lead to a terminal point. Our approach achieves a $3.1\%$ absolute improvement on VBench over state-of-the-art, while reducing Fréchet Video Distance by $28.3\%$ on average across $15$s videos. Our code is available on https://mever-team.github.io/tango.

Problem

• autoregressive video generation
• error accumulation
• terminal points
• diffusion models
• test-time adaptation

Innovation

• test-time adaptation
• noise guidance
• autoregressive video generation
• diffusion models
• trajectory optimization