Dynamics as Code: On Model Compression via Dynamic System

2026-10-07

AI Summary

This study addresses the deployment challenges of large models under resource constraints by proposing a dynamic system compression paradigm that encodes high-dimensional weights as indices of system trajectories, enabling training-free compact representation with controllable error recovery. Theoretically, it proves that finite trajectories constitute an ε-net under Diophantine conditions, unifying four classes of dynamical system families and establishing formal relationships among resolution, error, and compression ratio. Methodologically, the approach integrates space-filling curves, chaotic systems, and low-discrepancy sequences, enhanced by KD-tree acceleration and coordinate template optimization for improved efficiency. Experiments demonstrate that this method achieves competitive compression ratios and flexible state-space design on both ResNet and Qwen architectures.

Abstract

The escalating size of pretrained neural networks has rendered model compression a prerequisite for deployment under stringent memory and compute constraints. With the irrational winding as an example, earlier work introduced a dynamic system (DS) paradigm that reconceptualizes compression as compact weight representation: high-dimensional parameters are encoded by the index of a trajectory produced by a dynamic system, from which the vector is recovered during decompression. This mechanism is fundamentally distinct from pruning, quantization, knowledge distillation, and low-rank decomposition. Along this direction, we prove that under a Diophantine condition, a finite trajectory of \(M = O(ε^{-(d+ν)})\) states in the irrational winding constitutes an \(ε\)-net over the \(d\)-dimensional weight space, thereby linking state resolution, decompression error, and compression ratio in a predictable manner. Furthermore, we propose a generalized DS-based model compression framework by unifying four DS families---space-filling curves (Hilbert, Peano, Morton/Z-order, Snake), chaotic systems (Lorenz), congruential and pseudo-random generators (LCG, PCG), and low-discrepancy sequences (Halton). Also, we introduce the KD-tree and coordinate-template acceleration to scale to large models as well as outlier identification to control the error. Experiments on ResNet-18 and Qwen2.5-1.5B/Qwen1.5-7B validate that DS-based compression achieves competitive compression ratios without post-hoc retraining, with controllable decompression error and flexible state-space design, establishing it as a principled and practical compression approach.

Problem

• Model Compression
• Dynamic System
• Neural Network Deployment
• Weight Representation

Innovation

• Model Compression
• Dynamic System
• Space-filling Curves
• Weight Representation
• Large Language Models