Technology Category
Machine Learning: Representation LearningComputer Vision: Representation Learning for VisionNatural Language Processing: Learning & Optimization for NLP
Koopman Autoencoders Learn Neural Representation Dynamics
🤖 AI Summary
This work investigates whether the evolution of neural network hidden-layer representations can be modeled using dynamical systems theory. Method: We propose the first framework integrating the Koopman operator with variational autoencoders (VAEs), treating layer-wise activations as states of a dynamical system and learning interpretable, editable representation dynamics in a lifted linear space. Topological regularization jointly enforces dynamical linearization and preservation of the original representation topology, naturally reproducing the progressive topological simplification observed in deep networks. Contribution/Results: On Yin-Yang and MNIST benchmarks, our method enables targeted class forgetting and significantly enhances interpretability and controllability of representation dynamics. Experiments demonstrate high-fidelity modeling of deep neural network representation evolution via this linearized dynamical model, establishing a novel paradigm for neural representation analysis.
Technology Category
Application Category
📝 Abstract
This paper explores a simple question: can we model the internal transformations of a neural network using dynamical systems theory? We introduce Koopman autoencoders to capture how neural representations evolve through network layers, treating these representations as states in a dynamical system. Our approach learns a surrogate model that predicts how neural representations transform from input to output, with two key advantages. First, by way of lifting the original states via an autoencoder, it operates in a linear space, making editing the dynamics straightforward. Second, it preserves the topologies of the original representations by regularizing the autoencoding objective. We demonstrate that these surrogate models naturally replicate the progressive topological simplification observed in neural networks. As a practical application, we show how our approach enables targeted class unlearning in the Yin-Yang and MNIST classification tasks.
Problem
Research questions and friction points this paper is trying to address.
Model neural network transformations using dynamical systems theory
Predict neural representation evolution via Koopman autoencoders
Enable targeted class unlearning in classification tasks
Innovation
Methods, ideas, or system contributions that make the work stand out.
Uses Koopman autoencoders for neural dynamics
Learns linear surrogate models for representation editing
Preserves topology via regularized autoencoding objective
🔎 Similar Papers
