FieldFormer: Physics-Informed Transformers for Spatio-Temporal Field Reconstruction from Sparse Sensors

2025-10-03

AI Summary

This work addresses the challenge of reconstructing physical fields from sparse, noisy, and irregularly distributed spatiotemporal sensor data. Methodologically, it proposes a mesh-free physics-informed field reconstruction framework that introduces a learnable anisotropic distance metric to capture complex propagation characteristics, coupled with a dynamic local neighborhood construction mechanism optimized via an EM-style iterative procedure. Spatial dependencies are modeled using a local Transformer encoder, while physics consistency is enforced through automatic differentiation—enabling exact PDE residual constraints and boundary-specific penalty terms. Experiments on three benchmark tasks demonstrate superior performance: the method outperforms strong baselines by over 40%, achieves high-accuracy reconstruction (RMSE < 10⁻²) with only 0.4%–2% sampling density and up to 10% noise, and exhibits both computational efficiency and strong generalization across diverse physical domains.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Spatio-temporal sensor data is often sparse, noisy, and irregular, and existing interpolation or learning methods struggle here because they either ignore governing PDEs or do not scale. We introduce FieldFormer, a transformer-based framework for mesh-free spatio-temporal field reconstruction that combines data-driven flexibility with physics-based structure. For each query, FieldFormer gathers a local neighborhood using a learnable velocity-scaled distance metric, enabling anisotropic adaptation to different propagation regimes. Neighborhoods are built efficiently via per-batch offset recomputation, and refined in an expectation-maximization style as the velocity scales evolve. Predictions are made by a local transformer encoder, and physics consistency is enforced through autograd-based PDE residuals and boundary-specific penalties. Across three benchmarks--a scalar anisotropic heat equation, a vector-valued shallow-water system, and a realistic advection-diffusion pollution simulation--FieldFormer consistently outperforms strong baselines by more than 40%. Our results demonstrate that FieldFormer enables accurate (RMSE$<10^{-2}$), efficient, and physically consistent field reconstruction from sparse (0.4%-2%) and noisy(10%) data.

Problem

• Reconstructs spatio-temporal fields from sparse noisy sensor data
• Incorporates physics constraints through PDE residuals and boundary penalties
• Handles anisotropic propagation regimes via velocity-scaled local neighborhoods

Innovation

• Transformer-based mesh-free spatio-temporal field reconstruction
• Learnable velocity-scaled anisotropic neighborhood gathering
• Autograd-based PDE residuals enforce physics consistency