EddyFormer: Accelerated Neural Simulations of Three-Dimensional Turbulence at Scale

2025-10-28

AI Summary

High computational cost and poor scalability of conventional direct numerical simulation (DNS) hinder efficient modeling of multiscale interactions in three-dimensional turbulent flows. To address this, we propose a novel data-driven modeling framework that integrates the spectral element method (SEM) with the Transformer architecture. Our approach introduces an innovative SEM tokenization scheme that explicitly decouples resolved-grid-scale and subgrid-scale features, and—uniquely—embeds self-attention mechanisms directly into spectral space, coupled with a frequency-domain loss function to enforce cross-scale physical consistency. The model achieves DNS-level accuracy at 256³ resolution while accelerating inference by 30×. It demonstrates robust generalization to unseen domains four times larger than the training domain and successfully resolves strongly nonlinear turbulent flows where existing machine learning models fail to converge. This work significantly advances the frontier of high-fidelity, rapid turbulent flow simulation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Computationally resolving turbulence remains a central challenge in fluid dynamics due to its multi-scale interactions. Fully resolving large-scale turbulence through direct numerical simulation (DNS) is computationally prohibitive, motivating data-driven machine learning alternatives. In this work, we propose EddyFormer, a Transformer-based spectral-element (SEM) architecture for large-scale turbulence simulation that combines the accuracy of spectral methods with the scalability of the attention mechanism. We introduce an SEM tokenization that decomposes the flow into grid-scale and subgrid-scale components, enabling capture of both local and global features. We create a new three-dimensional isotropic turbulence dataset and train EddyFormer to achieves DNS-level accuracy at 256^3 resolution, providing a 30x speedup over DNS. When applied to unseen domains up to 4x larger than in training, EddyFormer preserves accuracy on physics-invariant metrics-energy spectra, correlation functions, and structure functions-showing domain generalization. On The Well benchmark suite of diverse turbulent flows, EddyFormer resolves cases where prior ML models fail to converge, accurately reproducing complex dynamics across a wide range of physical conditions.

Problem

• Accelerating large-scale 3D turbulence simulations with neural networks
• Overcoming computational prohibitions of direct numerical simulation methods
• Achieving DNS-level accuracy while providing significant speedup

Innovation

• Transformer-based spectral-element architecture for turbulence simulation
• SEM tokenization captures local and global flow features
• Achieves DNS-level accuracy with 30x speedup