From Redundancy to Minimality: Fixed-Point-Guided Hierarchical Reduction of Learned Piecewise-Linear Dynamics

2026-10-01

AI Summary

This study addresses the difficulty of reliably obtaining minimal dynamical representations by directly training augmented linear recurrent neural networks (AL-RNNs) with few ReLU units. To overcome this, we propose a "learn–reduce–retrain" framework that adopts an overparameterize-then-prune strategy. Through a fixed-point-guided hierarchical reduction procedure, multi-ReLU networks are progressively linearized and merged while preserving fixed-point signs to achieve minimal representations. We further prove that reproducing Q fixed points requires at least Q signs, thereby establishing a certificate for sign-level minimality. Evaluated on the 3-scroll Chua system under equal capacity constraints, the method increases the seed success rate from 20% to approximately 71%, validating the effectiveness of redundant capacity as scaffolding for discovering minimal structures.

Abstract

Understanding a nonlinear dynamical system from time series requires not only reproducing its trajectories, but also identifying a simple representation that preserves its essential dynamical structure. Almost-linear recurrent neural networks (AL-RNNs) are piecewise-linear RNNs in which only a subset of units use ReLU nonlinearities, so that nonlinear capacity is explicitly controlled by the number of ReLU units. Their activation patterns define linear regions, represented as symbols, whose observed transitions form a symbolic transition graph. However, directly training AL-RNNs with few ReLU units to realize minimal dynamical representations can be unreliable. We ask whether an AL-RNN with more ReLU units can instead be trained first and systematically reduced to a minimal dynamical representation. We introduce a fixed-point-guided hierarchical reduction procedure that progressively linearizes selected ReLU units, merging neighboring linear regions and graph nodes while preserving distinct symbols containing fixed points (FPs). The resulting reduction tree defines a hierarchy of progressively simpler candidates. Each reduced candidate is initialized from the parent parameters and retrained under guidance from the parent dynamics. We also prove that reproducing $Q$ distinct fixed points requires at least $Q$ FP-containing symbols, providing a certificate of symbol-level minimality when this bound is attained. On the 3-scroll Chua system, direct training with the theoretical minimum of three ReLU units achieves high-fidelity minimal realizations in only 20% of seeds, whereas our learn-reduce-retrain strategy increases the seed-macro success rate to approximately 71% at the same final nonlinear capacity. These results show that redundant nonlinear capacity can serve as a scaffold for discovering and realizing minimal dynamical representations.

Problem

• nonlinear dynamical systems
• minimal representation
• almost-linear recurrent neural networks
• piecewise-linear dynamics
• fixed points

Innovation

• Almost-linear RNNs
• Hierarchical reduction
• Fixed-point guidance
• Piecewise-linear dynamics
• Minimal representation