Latent attention on masked patches for flow reconstruction

2026-03-02

AI Summary

This work addresses the challenges of flow field reconstruction under high masking ratios and noisy conditions, where existing methods often lack both efficiency and interpretability. To this end, the authors propose the LAMP model, a modular reconstruction architecture that integrates patch-based proper orthogonal decomposition (POD) for dimensionality reduction, a single-layer Transformer, and closed-form linear regression. The model incorporates an interpretable attention mechanism to generate multi-fidelity sensor placement maps, enabling nonlinear compression and deep attention. Evaluated on two-dimensional unsteady wake reconstruction with 90% masking and signal-to-noise ratios between 10–30 dB, LAMP significantly outperforms current benchmarks. Notably, incorporating nonlinear measurement states reduces prediction error by nearly an order of magnitude.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Vision transformers have demonstrated outstanding performance on image generation applications, but their adoption in scientific disciplines, like fluid dynamics, has been limited. We introduce the Latent Attention on Masked Patches (LAMP) model, an interpretable regression-based modified vision transformer designed for masked flow reconstruction. LAMP follows a three-fold strategy: (i) partition of each flow snapshot into patches, (ii) dimensionality reduction of each patch via patch-wise proper orthogonal decomposition, and (iii) reconstruction of the full field from a masked input using a single-layer transformer trained via closed-form linear regression. We test the method on two canonical 2D unsteady wakes: a wake past a bluff body, and a chaotic wake past a flat plate. We show that the LAMP accurately reconstructs the full flow field from a 90\%-masked and noisy input, across signal-to-noise ratios between 10 and 30\,dB. Incorporating nonlinear measurement states can reduce the prediction error by up to an order of magnitude. The learned attention matrix yields physically interpretable multi-fidelity optimal sensor-placement maps. The modularity of the framework enables nonlinear compression and deep attention blocks, thereby providing an efficient baseline for nonlinear and high-dimensional masked flow reconstruction.

Problem

• masked flow reconstruction
• fluid dynamics
• vision transformers
• flow field reconstruction
• noisy input

Innovation

• masked flow reconstruction
• vision transformer
• proper orthogonal decomposition
• interpretable attention
• sensor placement