SigGate: Enhancing Recurrent Neural Networks with Signature-Based Gating Mechanisms

2025-02-13

AI Summary

This paper addresses the limitation of recurrent neural networks (RNNs) in capturing the global geometric structure of temporal paths. To overcome this, we propose integrating learnable path signatures into gating mechanisms—replacing the forget gate in LSTM and the reset gate in GRU—so that gating decisions depend on geometric features of the entire historical path rather than local hidden states. Our approach achieves the first end-to-end differentiable, path-signature-driven gating mechanism, combining theoretical rigor with computational tractability. The resulting architectures, SigLSTM and SigGRU, consistently outperform standard LSTM/GRU and state-of-the-art baselines across diverse time-series forecasting and classification tasks. Notably, they exhibit superior long-range dependency modeling and enhanced robustness to noise, empirically validating that geometry-aware gating fundamentally improves temporal representation learning.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Temporal Planning
• Machine Learning: Neuro-Symbolic Learning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this paper, we propose a novel approach that enhances recurrent neural networks (RNNs) by incorporating path signatures into their gating mechanisms. Our method modifies both Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) architectures by replacing their forget and reset gates, respectively, with learnable path signatures. These signatures, which capture the geometric features of the entire path history, provide a richer context for controlling information flow through the network's memory. This modification allows the networks to make memory decisions based on the full historical context rather than just the current input and state. Through experimental studies, we demonstrate that our Signature-LSTM (SigLSTM) and Signature-GRU (SigGRU) models outperform their traditional counterparts across various sequential learning tasks. By leveraging path signatures in recurrent architectures, this method offers new opportunities to enhance performance in time series analysis and forecasting applications.

Problem

• Enhance RNNs with signature-based gating
• Modify LSTM and GRU using path signatures
• Improve time series analysis performance

Innovation

• Incorporates path signatures into RNNs
• Modifies LSTM and GRU architectures
• Enhances memory decisions with historical context