Semantic Neural Radiance Fields for Multi-Date Satellite Data

2025-02-24

AI Summary

Dynamic objects (e.g., vehicles) in multi-temporal satellite imagery cause cross-temporal geometric inconsistencies and semantic label noise, degrading 3D semantic reconstruction quality. Method: We propose a semantics-enhanced Satellite NeRF framework: (i) the first integration of pixel-wise semantic supervision into satellite NeRF, enabling joint semantic-geometric optimization; (ii) a multi-temporal radiometric consistency modeling module to mitigate color drift; and (iii) a self-supervised label purification strategy to improve robustness against noisy semantic annotations. Contributions/Results: We release the first manually annotated multi-view satellite semantic dataset; achieve state-of-the-art performance in both semantic segmentation accuracy and 3D reconstruction PSNR; and significantly enhance reconstruction fidelity of dynamic objects while suppressing temporal artifacts. Our code and dataset are publicly available.

Technology Category

Computer Vision: Remote Sensing / Geospatial AIKnowledge Representation and Reasoning: Geometric, Spatial, and Temporal ReasoningConstraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSearch and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAGGraph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In this work we propose a satellite specific Neural Radiance Fields (NeRF) model capable to obtain a three-dimensional semantic representation (neural semantic field) of the scene. The model derives the output from a set of multi-date satellite images with corresponding pixel-wise semantic labels. We demonstrate the robustness of our approach and its capability to improve noisy input labels. We enhance the color prediction by utilizing the semantic information to address temporal image inconsistencies caused by non-stationary categories such as vehicles. To facilitate further research in this domain, we present a dataset comprising manually generated labels for popular multi-view satellite images. Our code and dataset are available at https://github.com/wagnva/semantic-nerf-for-satellite-data.

Problem

• 3D semantic representation from multi-date satellite images
• Improving noisy input labels using Neural Radiance Fields
• Addressing temporal inconsistencies in satellite image categories

Innovation

• NeRF for 3D semantic representation
• Improves noisy semantic labels
• Addresses temporal image inconsistencies