How Much Memory Do We Need? Adaptive Memory Gate for Neural Operators

2026-06-11

AI Summary

Existing memory-augmented neural operators employ fixed memory weights across varying observational conditions—such as resolution and viscosity—lacking adaptability and thereby limiting their performance in diverse scenarios. This work proposes the Adaptive Memory-Gated Fourier Neural Operator (AMGFNO), which introduces, for the first time within the Fourier neural operator framework, a learnable memory gating mechanism that dynamically modulates the weights of historical states. This enables the model to adaptively adjust the intensity of memory utilization based on input conditions. The approach substantially enhances modeling accuracy and generalization under low-resolution settings, achieving 55%–79% reductions in normalized root mean square error on the Kuramoto–Sivashinsky and Burgers’ equations. Notably, the learned memory gate values automatically decay from approximately 0.7 at low resolutions to nearly zero as resolution increases.

Abstract

Neural operators have emerged as a powerful data-driven approach for solving time-dependent PDEs. Among recent advances, memory-augmented neural operators explicitly incorporate past states and have achieved remarkable performance under low-resolution observation settings. However, existing approaches apply a fixed memory weight regardless of observation conditions, such as resolution or physical parameters, limiting their adaptability. Our preliminary experiments reveal that optimal memory weight varies with resolution and viscosity, implying that a fixed memory weight cannot simultaneously optimize performance across diverse settings. We propose AMGFNO, which dynamically modulates memory weight through a learnable gate. On the Kuramoto-Sivashinsky and Burgers' equations, AMGFNO achieves 55-79% nRMSE reduction over at low resolution, with the learned gate value automatically decreasing from $\bar{g} \approx 0.7$ to near-zero as resolution increases.

Problem

• neural operators
• memory weight
• adaptability
• resolution
• time-dependent PDEs

Innovation

• adaptive memory gate
• neural operators
• time-dependent PDEs
• memory-augmented models
• dynamic modulation