SigMA: Path Signatures and Multi-head Attention for Learning Parameters in fBm-driven SDEs

2025-12-17

AI Summary

This paper addresses the challenge of parameter estimation for stochastic differential equations driven by fractional Brownian motion (fBm), whose non-Markovianity, non-semimartingale structure, and long-range dependence severely limit conventional methods. To this end, we propose SigMA—a novel framework that innovatively integrates path signatures’ invariant feature extraction with multi-head self-attention, augmented by convolutional preprocessing and an end-to-end differentiable architecture. SigMA enables joint, robust estimation of the Hurst exponent and drift/diffusion coefficients. Evaluated on synthetic data as well as real-world financial volatility and lithium-ion battery degradation time series, SigMA consistently outperforms CNN, LSTM, Transformer, and Deep Signature baselines. It achieves simultaneous improvements in estimation accuracy, cross-scale generalization, and model compactness—demonstrating superior performance across all three key metrics.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Stochastic differential equations (SDEs) driven by fractional Brownian motion (fBm) are increasingly used to model systems with rough dynamics and long-range dependence, such as those arising in quantitative finance and reliability engineering. However, these processes are non-Markovian and lack a semimartingale structure, rendering many classical parameter estimation techniques inapplicable or computationally intractable beyond very specific cases. This work investigates two central questions: (i) whether integrating path signatures into deep learning architectures can improve the trade-off between estimation accuracy and model complexity, and (ii) what constitutes an effective architecture for leveraging signatures as feature maps. We introduce SigMA (Signature Multi-head Attention), a neural architecture that integrates path signatures with multi-head self-attention, supported by a convolutional preprocessing layer and a multilayer perceptron for effective feature encoding. SigMA learns model parameters from synthetically generated paths of fBm-driven SDEs, including fractional Brownian motion, fractional Ornstein-Uhlenbeck, and rough Heston models, with a particular focus on estimating the Hurst parameter and on joint multi-parameter inference, and it generalizes robustly to unseen trajectories. Extensive experiments on synthetic data and two real-world datasets (i.e., equity-index realized volatility and Li-ion battery degradation) show that SigMA consistently outperforms CNN, LSTM, vanilla Transformer, and Deep Signature baselines in accuracy, robustness, and model compactness. These results demonstrate that combining signature transforms with attention-based architectures provides an effective and scalable framework for parameter inference in stochastic systems with rough or persistent temporal structure.

Problem

• Estimates parameters in fBm-driven SDEs using path signatures and attention
• Addresses non-Markovian processes lacking classical estimation techniques
• Focuses on Hurst parameter and multi-parameter inference from trajectories

Innovation

• Integrates path signatures with multi-head attention
• Uses convolutional preprocessing and MLP for encoding
• Learns parameters from fBm-driven SDEs robustly