LVS: Local View Synthesis from Relative Camera Pose by Reusing Previous Views

2026-10-08

AI Summary

This study addresses the redundant rendering and high latency caused by frequent viewpoint updates during interactive scene exploration. To this end, it proposes a local view synthesis framework based on relative camera poses. This work pioneers the decoupling of scene rendering from local view updates, leveraging relative poses to guide RGB-D image reuse. By integrating 3D Gaussian Splatting, geometric warping, a lightweight multi-scale residual network, and a feature caching mechanism, the method enables efficient generation of neighboring views. Experimental results demonstrate a 0.72 dB improvement in PSNR over pure warping approaches, alongside significantly reduced computational overhead and query latency. These advances effectively support real-time interaction in AR/VR applications.

Abstract

Interactive scene exploration requires frequent view updates, although small camera motions preserve much of the visible content. Conventional 3D Gaussian Splatting nevertheless renders each target view, leaving this image overlap unexploited. Reusing rendered images offers an alternative. Geometric warping alone cannot recover newly exposed content and remains sensitive to depth errors. We propose a per-scene framework that replaces repeated scene rendering for nearby views with relative-pose-guided RGB-D image reuse. Geometric warping uses depth and relative pose to transport source content, while a lightweight multiscale network predicts RGB residuals to correct artifacts and infer missing appearance. Cached source features further reduce repeated computation. On GS-render, residual refinement improves PSNR by 0.72~dB over pure warping; evaluations on captured and rendered scenes demonstrate low query latency. This separation of scene rendering from local view updates supports responsive scene exploration, with potential applications in augmented and virtual reality.

Problem

• Local View Synthesis
• 3D Gaussian Splatting
• Interactive Scene Exploration
• Relative Camera Pose
• View Rendering Latency

Innovation

• Local View Synthesis
• 3D Gaussian Splatting
• Relative Camera Pose
• RGB-D Image Reuse
• Residual Refinement