Physics Driven Image Simulation from Commercial Satellite Imagery

2025-04-21

AI Summary

Remote sensing simulation faces challenges including heavy reliance on LiDAR data, extensive manual intervention, and difficulty in cross-spectral band modeling. Method: This paper proposes an end-to-end, physically grounded simulation framework driven solely by commercial satellite imagery. It automatically constructs 3D geometry from digital surface models (DSMs), infers material properties by fusing multi-source satellite imagery, and integrates physics-based rendering with radiative transfer modeling across a broad spectral range (200 nm–20 μm) to achieve fully automated, high-fidelity reconstruction of terrain, buildings, vegetation, and dynamic vehicles. Contribution/Results: To our knowledge, this is the first framework to eliminate LiDAR dependence and generate novel regional scenes without any manual intervention. Experiments demonstrate substantial reduction in modeling cost, full-spectrum support—from ultraviolet (UV) to long-wave infrared (LWIR)—for algorithm development and processing pipeline validation, and significant improvements in realism, generalizability, and scalability of geospatial scene simulation.

Technology Category

Computer Vision: Remote Sensing / Geospatial AISearch and Optimization: Sampling/Simulation-based SearchIntelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systemsWeb Mining and Content Analysis: Web data generation and simulationSearch and Retrieval-Augmented AI: Large language models for search

Abstract

Physics driven image simulation allows for the modeling and creation of realistic imagery beyond what is afforded by typical rendering pipelines. We aim to automatically generate a physically realistic scene for simulation of a given region using satellite imagery to model the scene geometry, drive material estimates, and populate the scene with dynamic elements. We present automated techniques to utilize satellite imagery throughout the simulated scene to expedite scene construction and decrease manual overhead. Our technique does not use lidar, enabling simulations that could not be constructed previously. To develop a 3D scene, we model the various components of the real location, addressing the terrain, modelling man-made structures, and populating the scene with smaller elements such as vegetation and vehicles. To create the scene we begin with a Digital Surface Model, which serves as the basis for scene geometry, and allows us to reason about the real location in a common 3D frame of reference. These simulated scenes can provide increased fidelity with less manual intervention for novel locations on earth, and can facilitate algorithm development, and processing pipelines for imagery ranging from UV to LWIR $(200nm-20mu m)$.

Problem

• Automate physically realistic scene generation from satellite imagery
• Model terrain, structures, and dynamic elements without lidar
• Enhance simulation fidelity for UV to LWIR imagery processing

Innovation

• Physics driven simulation from satellite imagery
• Automated scene construction without lidar
• Digital Surface Model based 3D scene