seqKAN: Sequence processing with Kolmogorov-Arnold Networks

2025-02-20

AI Summary

This work addresses the weak interpretability and poor extrapolation capability of Kolmogorov–Arnold Networks (KANs) in sequence modeling. To this end, we propose seqKAN—the first strictly KAN-compliant, interpretable sequence model grounded in the Kolmogorov–Arnold representation theorem. Methodologically, seqKAN introduces learnable grid-based spline activations, a sequenced tensor expansion architecture, and end-to-end differentiable symbolic parameterization, ensuring full analytical tractability of functional paths. Evaluated on complex physics-generated time series, seqKAN substantially outperforms existing KAN variants, RNNs, and symbolic regression methods: extrapolation error decreases by over 40%, while interpolation accuracy improves concurrently—demonstrating both theoretical consistency and empirical robustness. The core contribution lies in systematically extending the mathematical foundations of KANs to sequential data, thereby unifying high transparency with strong generalization capacity.

Technology Category

• Machine Learning: Kernel Methods
• Cognitive Modeling & Cognitive Systems: Symbolic Representations
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Kolmogorov-Arnold Networks (KANs) have been recently proposed as a machine learning framework that is more interpretable and controllable than the multi-layer perceptron. Various network architectures have been proposed within the KAN framework targeting different tasks and application domains, including sequence processing. This paper proposes seqKAN, a new KAN architecture for sequence processing. Although multiple sequence processing KAN architectures have already been proposed, we argue that seqKAN is more faithful to the core concept of the KAN framework. Furthermore, we empirically demonstrate that it achieves better results. The empirical evaluation is performed on generated data from a complex physics problem on an interpolation and an extrapolation task. Using this dataset we compared seqKAN against a prior KAN network for timeseries prediction, recurrent deep networks, and symbolic regression. seqKAN substantially outperforms all architectures, particularly on the extrapolation dataset, while also being the most transparent.

Problem

• Proposes seqKAN for sequence processing
• Enhances interpretability and controllability in KANs
• Outperforms existing architectures in extrapolation tasks

Innovation

• seqKAN for sequence processing
• More faithful KAN framework
• Outperforms in extrapolation tasks