Neural Variable-Order Fractional Differential Equation Networks

2025-03-20

AI Summary

Traditional integer-order and constant-order fractional differential equations struggle to capture dynamic memory evolution. To address this, we propose Neural Variable-Order Fractional Differential Equation Networks (NvoFDE). Our method introduces a learnable, hidden-state-dependent variable-order Caputo fractional derivative into the neural differential equation framework—enabling the differentiation order to adapt dynamically to evolving hidden features and explicitly modeling non-stationary long-range dependencies. NvoFDE integrates an implicit neural ODE solver with graph neural networks, supporting end-to-end training. Extensive experiments on diverse graph learning tasks demonstrate that NvoFDE consistently outperforms both integer-order and constant-order fractional baselines, achieving significant gains in memory modeling fidelity and generalization performance. This work establishes a more expressive fractional-order neural dynamical paradigm for modeling complex dynamic systems.

Technology Category

Machine Learning: Graph-based Machine LearningNatural Language Processing: Learning & Optimization for NLPComputer Vision: Learning & Optimization for CV

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Neural differential equation models have garnered significant attention in recent years for their effectiveness in machine learning applications.Among these, fractional differential equations (FDEs) have emerged as a promising tool due to their ability to capture memory-dependent dynamics, which are often challenging to model with traditional integer-order approaches.While existing models have primarily focused on constant-order fractional derivatives, variable-order fractional operators offer a more flexible and expressive framework for modeling complex memory patterns. In this work, we introduce the Neural Variable-Order Fractional Differential Equation network (NvoFDE), a novel neural network framework that integrates variable-order fractional derivatives with learnable neural networks.Our framework allows for the modeling of adaptive derivative orders dependent on hidden features, capturing more complex feature-updating dynamics and providing enhanced flexibility. We conduct extensive experiments across multiple graph datasets to validate the effectiveness of our approach.Our results demonstrate that NvoFDE outperforms traditional constant-order fractional and integer models across a range of tasks, showcasing its superior adaptability and performance.

Problem

• Modeling memory-dependent dynamics in machine learning
• Introducing variable-order fractional derivatives for flexibility
• Enhancing adaptability and performance in graph datasets

Innovation

• Integrates variable-order fractional derivatives with neural networks
• Models adaptive derivative orders based on hidden features
• Outperforms traditional constant-order and integer models