Efficient Neural Controlled Differential Equations via Attentive Kernel Smoothing

2026-02-02

AI Summary

This work addresses the high computational cost of Neural Controlled Differential Equations (Neural CDEs), which arises from excessively small solver step sizes due to roughness in the control path. To mitigate this, the authors propose a novel path construction method based on kernel functions and Gaussian processes, replacing conventional spline interpolation to effectively suppress high-frequency noise while preserving essential temporal details. Furthermore, they introduce an attention-driven, multi-view CDE architecture that enables controllable modeling of trajectory smoothness and facilitates multi-scale dynamic feature fusion. The resulting model, termed MVC-CDE with GP, achieves state-of-the-art accuracy while significantly reducing the number of function evaluations and inference time, thereby offering an improved balance between model efficiency and performance.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Neural Controlled Differential Equations (Neural CDEs) provide a powerful continuous-time framework for sequence modeling, yet the roughness of the driving control path often restricts their efficiency. Standard splines introduce high-frequency variations that force adaptive solvers to take excessively small steps, driving up the Number of Function Evaluations (NFE). We propose a novel approach to Neural CDE path construction that replaces exact interpolation with Kernel and Gaussian Process (GP) smoothing, enabling explicit control over trajectory regularity. To recover details lost during smoothing, we propose an attention-based Multi-View CDE (MV-CDE) and its convolutional extension (MVC-CDE), which employ learnable queries to inform path reconstruction. This framework allows the model to distribute representational capacity across multiple trajectories, each capturing distinct temporal patterns. Empirical results demonstrate that our method, MVC-CDE with GP, achieves state-of-the-art accuracy while significantly reducing NFEs and total inference time compared to spline-based baselines.

Problem

• Neural Controlled Differential Equations
• path roughness
• computational efficiency
• function evaluations
• sequence modeling

Innovation

• Neural Controlled Differential Equations
• Kernel Smoothing
• Gaussian Process
• Attention Mechanism
• Multi-View CDE