Beyond Compression: Training Latent Representations for Stable Long-Horizon Rollout in Neural Surrogate Solvers

2026-09-24

AI Summary

This study addresses the error accumulation and prediction instability in neural surrogate solvers during long-horizon autoregressive rollouts, which stem from training objective mismatches. To overcome this, we propose a latent space reconstruction strategy tailored for long-term dynamical evolution. The method integrates Koopman operator learning with Hamming noise injection to optimize latent representations, combined with multi-step rollout fine-tuning that shifts the model from static reconstruction toward dynamic evolution adaptation. Experimental results demonstrate that the proposed approach reduces long-rollout errors by 40%, achieving accuracy comparable to full-resolution models while decreasing computational overhead by two orders of magnitude. Consequently, this work enables efficient and stable long-horizon extrapolation of complex dynamics.

Abstract

Latent neural surrogate solvers, or latent dynamics models, accelerate simulations of time-dependent physical systems by evolving a compressed latent space rather than resolving full-resolution fields directly. In principle this reduces computational cost and simplifies learning, but in practice errors often accumulate rapidly during long autoregressive rollouts, limiting predictive utility. We show that this instability does not stem from the latent representation itself, but arises when it is trained solely for reconstruction, producing representations poorly suited to long-horizon forecasting. We systematically evaluate training-level interventions that align latent representations with long-horizon rollout: Koopman operator learning and Hamming noise injection during autoencoder training to improve compression, together with noise injection and multi-step rollout fine-tuning to improve dynamics. Interventions that improve long-horizon rollout stability often degrade conventional training metrics, including reconstruction and one-step prediction accuracy. Collectively, these interventions reduce long-rollout error by approximately 40\% and match or exceed the accuracy of full-resolution models on two physics benchmarks, while requiring 2 orders of magnitude fewer floating point operations and half the GPU memory. Applied to mesoscale crystal-plasticity simulations of high-cycle fatigue, the resulting surrogate achieves stable extrapolation over horizons orders of magnitude beyond those observed during training. More broadly, these results show that neural compression should be designed not merely to reduce dimensionality, but to restructure the solution space for stable dynamical evolution, a key requirement for reliable, efficient neural surrogates in scientific applications.

Problem

• latent neural surrogate solvers
• long-horizon rollout
• error accumulation
• autoregressive instability
• latent dynamics models

Innovation

• Latent dynamics models
• Koopman operator learning
• Long-horizon rollout
• Neural surrogate solvers
• Noise injection