Understanding Latent-Dimension Scaling in Dynamical-System Learning through Spectral Reliability

2026-10-08

AI Summary

This study addresses the issue of spurious eigenvalues arising from increased latent dimensions in autoregressive dynamics learning, which constrains prediction accuracy. To mitigate this, we propose a spectral residual loss function and a two-stage alternating training strategy based on Koopman autoencoders. The method employs relative residual analysis to identify high-residual regions, and the spectral residual loss effectively suppresses the generation of spurious eigenvalues, thereby enhancing the reliability of the Koopman operator's spectral decomposition. Experimental results demonstrate that the proposed approach significantly reduces prediction errors during long-term rollout across multiple chaotic systems. Moreover, the error decreases monotonically with increasing latent dimensionality, substantially extending the valid prediction horizon and improving both the stability and generalization capability of dynamical modeling.

Abstract

In deep learning, approximation theory motivates increasing representation size. We ask whether this benefit extends to dynamics learning through autoregressive prediction. We analyze the learned time evolution through the eigenstructure of Koopman operators, using relative residuals to detect spurious eigenpairs arising even as one-step error falls. For bounded Koopman operators, we show that minimal residuals over learned dictionary spaces converge pointwise to their full-space counterparts as these spaces approximate the observable space in $L^2$. Our hypothesis is that Koopman spectral reliability helps explain how consistently rollout error decreases with increasing dimension. We compare two models of a shared Koopman autoencoder trained alternately for reconstruction and latent evolution, using latent-prediction loss (one-step prediction errors in latent coordinates) or spectral-residual loss (relative residuals of candidate eigenpairs). Across six chaotic systems, both models reduced median windowed rollout error from smallest to largest dimension. The spectral-residual model achieved lower medians than the latent-prediction model for all systems and dimensions, and its median fell by a larger factor in every system. Its median decreased monotonically with dimension in four systems, against one for latent prediction. Against four baseline families, its mean valid prediction times were nearly always longer. At the largest dimension under two-stage training, we compared eigenvalue positions with each learned dictionary's residual contours. Spectral-residual eigenvalues concentrated in low-residual regions, whereas latent-prediction eigenvalues also appeared in high-residual regions, consistent with the hypothesis.

Problem

• dynamical-system learning
• latent-dimension scaling
• Koopman operator
• autoregressive prediction
• spectral reliability

Innovation

• Koopman operator
• spectral reliability
• autoregressive prediction
• latent-dimension scaling
• chaotic systems