TKAN: Temporal Kolmogorov-Arnold Networks

2024-05-12

Social Science Research Network

AI Summary

To address the limitations of RNNs and LSTMs in multistep time-series forecasting—including insufficient capacity to model complex nonlinear patterns, poor interpretability, and low computational efficiency—this paper proposes the TKAN architecture. Its core innovation is the Recurrent Kolmogorov–Arnold Network (RKAN) layer, which integrates learnable spline-based activations from Kolmogorov–Arnold Networks (KANs) into a gated recurrent structure for the first time. This design unifies dynamic weight evolution with long-term dependency modeling while ensuring end-to-end differentiability and strong universal approximation capability grounded in the Kolmogorov–Arnold representation theorem. Extensive experiments demonstrate that TKAN achieves significantly higher prediction accuracy and faster inference speed than state-of-the-art MLP, RNN, and LSTM baselines, without sacrificing model interpretability.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Recurrent Neural Networks (RNNs) have revolutionized many areas of machine learning, particularly in natural language and data sequence processing. Long Short-Term Memory (LSTM) has demonstrated its ability to capture long-term dependencies in sequential data. Inspired by the Kolmogorov-Arnold Networks (KANs) a promising alternatives to Multi-Layer Perceptrons (MLPs), we proposed a new neural networks architecture inspired by KAN and the LSTM, the Temporal Kolomogorov-Arnold Networks (TKANs). TKANs combined the strenght of both networks, it is composed of Recurring Kolmogorov-Arnold Networks (RKANs) Layers embedding memory management. This innovation enables us to perform multi-step time series forecasting with enhanced accuracy and efficiency. By addressing the limitations of traditional models in handling complex sequential patterns, the TKAN architecture offers significant potential for advancements in fields requiring more than one step ahead forecasting.

Problem

• Enhance multi-step time series forecasting accuracy
• Combine strengths of KAN and LSTM networks
• Address limitations in complex sequential pattern handling

Innovation

• Combines KAN and LSTM for time series
• Uses Recurring KAN Layers with memory
• Enhances multi-step forecasting accuracy