LLN: Learnable Lens Networks for Parameter-Efficient Long-Horizon Dynamical Prediction

2026-09-28

AI Summary

This study addresses the limited constraint of residual connections on feature geometry evolution and the low parameter efficiency in long-term prediction within deep networks. To this end, we propose an optics-inspired learnable lens network that replaces conventional residual accumulation with learnable optical transport in phase space, integrating free propagation with lens field transformations while introducing a globally invertible, volume-preserving symplectic structure. Furthermore, coordinate Gaussian transport and nonlinear trajectory focusing techniques are incorporated to enhance position-angle modeling. Experimental results demonstrate that this framework significantly improves prediction accuracy and gradient stability across diverse dynamical systems for long-horizon forecasting, while substantially reducing parameter counts. The proposed approach effectively combines universal approximation capabilities with physical interpretability.

Abstract

Explicit residual connections of the form (x+f(x)), often combined with normalization layers, have become a standard strategy for training very deep neural networks. However, residual addition primarily provides an algebraic shortcut for gradient propagation, while leaving the evolution of feature geometry across layers largely unconstrained. We introduce Learnable Lens Networks (LLN), a physics-inspired architecture that replaces direct feature-space residual accumulation with learnable optical transport in an augmented position-angle phase space. Each layer alternates between free propagation, which provides an implicit transport path, and a learnable lens field that performs nonlinear trajectory transformation and focusing. Theoretically, we establish that LLN transport is globally invertible and volume-preserving for any differentiable lens field, with the implemented coordinate-wise Gaussian transport further satisfying symplecticity. Importantly, these structural constraints do not limit expressivity: with unrestricted embeddings and readouts, LLN retain universal approximation of continuous end-to-end maps. Experiments across diverse dynamical systems demonstrate that LLN improves long-horizon prediction while using substantially fewer parameters than same-depth comparators. Further analysis reveals stable depth-wise gradient transport and interpretable learned dynamics under the coupled propagation and refraction design.

Problem

• Long-horizon dynamical prediction
• Residual connections
• Parameter efficiency
• Feature geometry evolution
• Dynamical systems

Innovation

• Learnable Lens Networks
• Parameter-Efficient
• Long-Horizon Dynamical Prediction
• Symplecticity
• Universal Approximation