Envision4D: Envisioning Visual Futures via Feed-forward 4D Gaussian Splatting for Autonomous Driving

2026-06-09

AI Summary

Existing feedforward approaches for future view synthesis in autonomous driving often suffer from ghosting artifacts and rely on oversimplified motion assumptions or strong priors. This work proposes a fully self-supervised feedforward framework that reframes the ill-posed extrapolation problem as a robust relational mapping by predicting future camera poses, modeling nonlinear dynamics, and incorporating intra-layer temporal attention with a conditional motion refinement mechanism. Leveraging a 4D Gaussian splatting representation and a progressive training strategy, the method effectively mitigates error accumulation. Without requiring ground-truth pose supervision, it significantly outperforms existing approaches and achieves state-of-the-art performance.

Abstract

Forecasting the future evolution of dynamic scenes is crucial in autonomous driving. However, existing feed-forward paradigms are primarily designed for interpolation. When extended to future extrapolation, they suffer from ghosting artifacts under large displacements and are constrained by simplified motion assumptions or strict future priors. To overcome these challenges, we propose Envision4D, a fully self-supervised feed-forward framework for pose-free future extrapolation. Specifically, we introduce a Future Pose Prediction module that infers future camera parameters via an iterative denoising process. Furthermore, to capture non-linear dynamics, we propose In-layer Temporal Attention and employ Conditioned Motion Lifting, which transforms the highly uncertain extrapolation process into robust relational mappings. Finally, a Progressive Training Strategy is utilized to stabilize unsupervised motion learning against error accumulation. Extensive experiments demonstrate that Envision4D achieves state-of-the-art performance, significantly outperforming existing methods in future view synthesis.

Problem

• future extrapolation
• autonomous driving
• ghosting artifacts
• dynamic scene forecasting
• motion modeling

Innovation

• 4D Gaussian Splatting
• Future Extrapolation
• Self-supervised Learning
• Temporal Attention
• Autonomous Driving