TriDF: Triplane-Accelerated Density Fields for Few-Shot Remote Sensing Novel View Synthesis

2025-03-17

AI Summary

To address overfitting and high computational cost in novel-view synthesis for remote sensing due to scarce multi-view imagery, this paper proposes TriDF—a framework that synthesizes high-fidelity novel views from only three input views. Methodologically, TriDF introduces a novel decoupled 3D representation: high-frequency color is modeled via learnable tri-planes, while density is encoded by a lightweight implicit field. To mitigate overfitting under few-shot conditions, we propose a point-cloud-driven depth-guided optimization strategy. Furthermore, TriDF integrates image-based rendering, reference-view feature fusion, and depth-aware constraints. Compared to NeRF, TriDF achieves a 30× speedup and improves PSNR, SSIM, and LPIPS by 7.4%, 12.2%, and 18.7%, respectively. Extensive experiments across diverse remote sensing scenarios demonstrate its strong robustness and generalization capability.

Technology Category

Computer Vision: 3D Computer VisionMachine Learning: Multi-instance/Multi-view LearningIntelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingGraph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Remote sensing novel view synthesis (NVS) offers significant potential for 3D interpretation of remote sensing scenes, with important applications in urban planning and environmental monitoring. However, remote sensing scenes frequently lack sufficient multi-view images due to acquisition constraints. While existing NVS methods tend to overfit when processing limited input views, advanced few-shot NVS methods are computationally intensive and perform sub-optimally in remote sensing scenes. This paper presents TriDF, an efficient hybrid 3D representation for fast remote sensing NVS from as few as 3 input views. Our approach decouples color and volume density information, modeling them independently to reduce the computational burden on implicit radiance fields and accelerate reconstruction. We explore the potential of the triplane representation in few-shot NVS tasks by mapping high-frequency color information onto this compact structure, and the direct optimization of feature planes significantly speeds up convergence. Volume density is modeled as continuous density fields, incorporating reference features from neighboring views through image-based rendering to compensate for limited input data. Additionally, we introduce depth-guided optimization based on point clouds, which effectively mitigates the overfitting problem in few-shot NVS. Comprehensive experiments across multiple remote sensing scenes demonstrate that our hybrid representation achieves a 30x speed increase compared to NeRF-based methods, while simultaneously improving rendering quality metrics over advanced few-shot methods (7.4% increase in PSNR, 12.2% in SSIM, and 18.7% in LPIPS). The code is publicly available at https://github.com/kanehub/TriDF

Problem

• Efficient few-shot novel view synthesis for remote sensing.
• Reduces computational burden in 3D scene reconstruction.
• Mitigates overfitting in limited input view scenarios.

Innovation

• Triplane representation accelerates few-shot NVS.
• Decouples color and density for efficient computation.
• Depth-guided optimization reduces overfitting in NVS.