🤖 AI Summary
Existing feedforward 3D foundation models, constrained by central perspective projection, struggle to accommodate the push-broom imaging geometry of satellites, limiting their applicability in multi-view satellite 3D reconstruction. This work proposes a lightweight adaptation framework that requires no fine-tuning of the backbone network: it selects an optimal view sequence through geometric consistency constraints, parameterizes push-broom rays into geometric tokens using rational function models, and introduces a ray-direction-aware adapter to inject these tokens into a frozen Transformer backbone. This approach achieves, for the first time, an effective integration of physical imaging geometry with deep feedforward architectures, significantly enhancing accuracy and robustness in digital surface model (DSM) generation and demonstrating the critical role of explicit geometric embedding and optimized view selection.
📝 Abstract
In the era of satellite constellations, multi-view optical satellite imagery is pivotal for Earth Observation (EO) and high-quality Digital Surface Model (DSM) reconstruction. Although feed-forward 3D foundation models have transformed computer vision, their deployment in satellite remote sensing is inherently constrained by the structural discrepancy between implicit perspective assumptions and explicit orbital pushbroom geometry. This geometric incongruity is further compounded by pronounced view-set heterogeneity. We present EO-VGGT, a framework that adapts a frozen perspective-driven model to orbital observations via explicit physical geometry embedding.First, the Geometry-Correlation Constrained Selection (GCCS) strategy prunes sub-optimal observations by balancing geometric diversity and radiometric consistency to optimize the input sequence. Second, a Sensor-Ray Encoder (SRE) parameterizes pixel-level pushbroom lines of sight derived from the Rational Function Model (RFM) into high-dimensional space-geometric tokens, reconciling the mathematical discrepancy between central projection and orbital kinematics. Third, a lightweight Ray-Pointing-Aware Adapter (RPAA) employs gated residual blocks to integrate these tokens directly into the frozen transformer backbone. Our findings underscore that integrating explicit physical geometry with optimized view selection is essential for robust feed-forward satellite 3D reconstruction.