WAVE: Warp-Based View Guidance for Consistent Novel View Synthesis Using a Single Image

2025-06-30

AI Summary

To address inter-view structural inconsistency in single-image novel view synthesis, this paper proposes a training-free, 3D-prior-free diffusion model framework for enhancing view consistency. The method operates directly on off-the-shelf diffusion models (e.g., Stable Diffusion) without multi-stage optimization or auxiliary 3D modules. Its core contributions are: (1) a view-guided differentiable warp operation that explicitly enforces cross-view geometric constraints in the latent space; and (2) an adaptive attention manipulation and noise reinitialization mechanism that dynamically refines spatial correspondences throughout the diffusion process. Evaluated on multiple novel view synthesis benchmarks, the approach achieves substantial improvements in view consistency—SSIM increases by 12.7% and LPIPS decreases by 28.4%—while preserving high-fidelity image quality. The method is highly generalizable and plug-and-play, requiring no architectural modification or retraining of the base diffusion model.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Generating high-quality novel views of a scene from a single image requires maintaining structural coherence across different views, referred to as view consistency. While diffusion models have driven advancements in novel view synthesis, they still struggle to preserve spatial continuity across views. Diffusion models have been combined with 3D models to address the issue, but such approaches lack efficiency due to their complex multi-step pipelines. This paper proposes a novel view-consistent image generation method which utilizes diffusion models without additional modules. Our key idea is to enhance diffusion models with a training-free method that enables adaptive attention manipulation and noise reinitialization by leveraging view-guided warping to ensure view consistency. Through our comprehensive metric framework suitable for novel-view datasets, we show that our method improves view consistency across various diffusion models, demonstrating its broader applicability.

Problem

• Ensuring view consistency in single-image novel view synthesis
• Improving spatial continuity across views in diffusion models
• Eliminating complex pipelines for efficient view-consistent generation

Innovation

• Training-free adaptive attention manipulation
• View-guided warping for consistency
• Noise reinitialization in diffusion models