translation-equivariant design

This work addresses the well-known difficulty of neural networks in achieving length extrapolation on tasks such as addition, a capability humans handle effortlessly across arbitrary sequence lengths. Drawing inspiration from physical principles, the authors identify three necessary conditions for successful generalization: locality, symmetry, and stability. Building upon these, they derive SEAD—an iterative neural cellular automaton governed by local convolutional rules that converge to a fixed point. Notably, SEAD embeds logical reasoning directly into its dynamical architecture rather than relying on scaling model parameters. Experiments demonstrate perfect length extrapolation and divergence-free training on parity, addition (generalizing from length L=16 to L=10⁶ with 100% accuracy), and Rule 110 tasks.

Data-driven modeling of dynamical systems often suffers from poor generalization—particularly in the absence of structural priors, where extrapolation to unexplored regions of state space (e.g., unseen basins of attraction) remains challenging. Method: We propose a multi-trajectory joint training paradigm for reservoir computing (RC), enabling, for the first time, strong generalization across distinct basins of attraction without relying on explicit dynamical assumptions or prior knowledge; training requires trajectories solely from a single basin. Contribution/Results: Evaluated on multistable systems, our approach achieves high-accuracy prediction of dynamics in entirely unseen basins. It fundamentally extends RC’s generalization capability beyond its traditional limits—achieving prior-free, cross-basin, and data-efficient extrapolation. This establishes a new paradigm for interpretable modeling and long-term forecasting of complex nonlinear systems.

This work addresses the lack of statistical fidelity in time-series modeling of dynamical systems, where conventional generalization error fails to guarantee accurate reproduction of physical invariants—such as invariant measures and Lyapunov exponents. To this end, we establish, for the first time, an ergodic-theoretic statistical generalization framework, explicitly identifying preservation of the physical measure as the central learning objective. We uncover a fundamental cause of statistical distortion in Neural ODEs applied to dynamical system regression: their neglect of Jacobian structural constraints. We theoretically prove—and empirically validate—that Jacobian regularization substantially enhances statistical generalization. On benchmark chaotic systems (Lorenz, Rössler), incorporating Jacobian information reduces estimation errors in invariant measures, statistical moments, and Lyapunov exponents by an average of 47%, outperforming MLPs, ResNets, Fourier Neural Networks, and RNNs.

Existing neural dynamical modeling approaches rely on single-dataset training and struggle with statistical heterogeneity across recordings. To address this, we propose the Meta Dynamical State-Space Model (Meta-DSSM), the first framework to integrate meta-learning into neural dynamical modeling. Meta-DSSM parameterizes task-specific dynamical families via a low-dimensional manifold and learns a shared dynamical solution space across multi-task neural activities. It unifies variational inference, deep state-space modeling, and meta-learning to enable rapid adaptation, reconstruction, and long-horizon prediction of latent dynamics from few-shot data. Evaluated on synthetic dynamical systems and multi-arm reaching datasets from primate motor cortex, Meta-DSSM significantly improves few-shot reconstruction accuracy and trajectory prediction stability. It establishes a generalizable modeling paradigm for cross-subject and cross-session neural decoding, advancing robustness and transferability in neural dynamical inference.

This work addresses physics-constrained dynamical modeling by proposing the first metriplectic neural network system provably satisfying both energy conservation and entropy stability. To handle both full-state and latent-entropy-variable settings, the method integrates geometric constraint embedding, low-rank parameterization, and error-controllable approximation theory—reducing computational complexity to quadratic in state dimension and rank. Compared to baseline models, it achieves significant gains in computational efficiency and representational capacity while maintaining high accuracy and strong robustness on multiscale generalization tasks; theoretical error bounds guarantee reliable extrapolation. The core contributions are: (i) the first verifiably stable metriplectic neural modeling framework, and (ii) a co-optimization framework that jointly enforces low-rank structure and fundamental physical conservation laws.

Traditional approaches struggle to generalize neural dynamical mechanisms across tasks and contexts from limited, high-dimensional, and noisy neural recordings. This work proposes a hierarchical RNN embedding model that jointly models neural dynamics across multiple tasks by learning a shared embedding in weight space. The method achieves, for the first time, joint embedding of neural dynamics across behavioral conditions, enabling direct inference of generalizable dynamical mechanisms from data. By integrating dynamical systems analysis—such as fixed-point identification and eigenvalue spectrum inversion—the model reveals the underlying structure of these dynamics. Experiments on both simulated data and real macaque motor cortex recordings demonstrate that the model accurately recovers ground-truth dynamics and extracts core neural mechanisms underlying motor control.

This work addresses the limited generalization of existing dynamical modeling approaches, which typically require system-specific modeling. To overcome this limitation, the paper introduces PDEDER, a pre-trained dynamics encoder that, for the first time, adapts the pre-training paradigm to dynamical system modeling. PDEDER employs neural ordinary differential equations (Neural ODEs) to construct a latent representation of system dynamics and jointly optimizes reconstruction, prediction objectives, and Lyapunov exponent constraints to ensure a well-structured and stable latent space. Evaluated across twelve diverse dynamical systems, PDEDER significantly outperforms baseline methods, demonstrating exceptional performance and strong generalization capabilities in both short- and long-term predictions within and across domains.

This study investigates how recurrent neural networks (RNNs) preserve the topological structure of invariant manifolds—such as those on tori or circles—in regular dynamical systems when trained for time-series prediction. By treating the invariant manifold of the input system as a driving signal, the work posits that the RNN’s hidden state encodes a finite history window rather than instantaneous inputs. A unified theoretical framework is developed by integrating generalized synchronization theory, differential embedding theorems, and contraction analysis. The analysis reveals that under regular dynamical driving, RNNs naturally satisfy smooth embedding conditions, circumventing the stringent requirements typical in chaotic systems. Furthermore, verifiable criteria are established that clarify the relationship between the dimensionality of the hidden state and the intrinsic dimension of the driving system, thereby elucidating the mechanism underlying topologically faithful representations.

Deep learning training often suffers sudden collapses due to minute perturbations, undermining reproducibility and scalability. This work reframes training stability as an intrinsic property of the learning system through the lens of dynamical systems theory, proposing a unified analytical framework that integrates optimization dynamics, data structure, parameter evolution, and learning signals. The authors introduce a controlled perturbation auditing method to quantify how training trajectories respond to structured disturbances. Their analysis reveals three key principles: high performance and stability are frequently decoupled; controlled randomness generally enhances robustness; and low-dimensional latent meta-state deviations consistently precede performance collapse. These findings are validated across both reinforcement learning and large language models, offering a measurable, comparable, and actionable theoretical foundation for understanding learning dynamics beyond final performance metrics.

This study investigates how unsupervised autoencoders learn macroscopic physical variables from microscopic spin configurations of the Ising model. By integrating multiscale coarse-graining analysis, recurrent modeling of dynamical trajectories, and nonequilibrium dynamics, the work reveals—for the first time—the flow-field topology in the representation space of autoencoders, driven by prediction error during training, and identifies two distinct learning regimes: magnetization-dominated and energy-dominated. The authors find that models trained with moderate to high learning rates tend to stall in transitional states, yet learning trajectories across diverse hyperparameter settings share universal topological features. This research establishes an interpretable bridge between unsupervised learning and statistical physics, offering a novel perspective on the physical underpinnings of deep learning representations.