Closing the Loop: Training-Free Revisit Consistency for Autoregressive Generative Rendering

2026-07-23

AI Summary

This work addresses the appearance inconsistency in autoregressive video generation when the camera revisits previously observed locations, a problem caused by overwriting of contextual cache that breaks 3D scene consistency. The authors propose a training-free method that leverages pose and depth information from a 3D engine to retrieve historical latent patches via spatiotemporal correspondences and inject them into the key-value (KV) cache. Additionally, they introduce a geometry-aware attention bias based on depth reprojection to align features geometrically. This approach is the first to integrate spatiotemporal correspondences without additional training, effectively closing the visual loop. It significantly outperforms existing training-free baselines on revisit trajectories in the TartanAir and TartanGround datasets while preserving high-quality video generation.

Abstract

Recent conditional video generation models have shown promising potentials to transform 3D engine renderings, such as depth maps and untextured geometry, into photorealistic videos for gaming and immersive content creation. These applications require long-horizon auto-regressive generation that continuously synthesizes new frames while preserving a persistent 3D world. Auto-regressive generators synthesize video chunk by chunk with a bounded KV cache, so when the camera revisits a location after its context has been evicted, the model often regenerates inconsistent appearance, even though the conditioning renderings (e.g., depth) remain perfectly aligned with the underlying geometry.We address this revisit inconsistency without any post-training by exploiting correspondences the 3D engine already provides: temporal correspondence retrieves pose-matched historical latent chunks into the KV cache as loop-closure memory, while spatial correspondence from camera pose and depth reprojection biases token-level attention toward geometrically corresponding regions of the retrieved chunks. We demonstrate our method on loop-closure trajectories mined from TartanAir and TartanGround dataset to mirror complicate real-world application scenarios, where it outperforms existing training-free baselines on revisit consistency without losing overall video quality. Project Page: https://wenchao-m.github.io/ClosetheLoop.github.io/

Problem

• revisit inconsistency
• autoregressive generation
• generative rendering
• video consistency
• 3D world persistence

Innovation

• revisit consistency
• training-free
• autoregressive video generation
• loop-closure memory
• 3D-aware attention