Technology Category
Search and Optimization: Mixed Discrete/Continuous SearchMachine Learning: Deep Neural Architectures and Foundation ModelsPlanning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
Neural Predictive Control to Coordinate Discrete- and Continuous-Time Models for Time-Series Analysis with Control-Theoretical Improvements
🤖 AI Summary
Existing time-series models rely on unconstrained neural networks to model dynamics, rendering them vulnerable to distribution shifts and limiting generalization and stability. This paper reformulates time-series modeling as an optimal control problem governed by ordinary differential equations (ODEs). We propose a discrete-continuous co-architectural framework: a discrete module (e.g., RNN) generates control inputs, while a continuous module (Neural ODE) governs latent state evolution; model predictive control (MPC) enables multi-step trajectory planning and task-aware long-horizon cost minimization. By embedding control-theoretic principles, the framework guarantees system convergence—specifically, exponential convergence to the infinite-horizon optimal solution. Evaluated across multiple benchmark datasets, our method consistently outperforms state-of-the-art approaches, demonstrating superior robust generalization, enhanced out-of-distribution adaptability, and improved runtime stability.
Technology Category
Application Category
📝 Abstract
Deep sequence models have achieved notable success in time-series analysis, such as interpolation and forecasting. Recent advances move beyond discrete-time architectures like Recurrent Neural Networks (RNNs) toward continuous-time formulations such as the family of Neural Ordinary Differential Equations (Neural ODEs). Generally, they have shown that capturing the underlying dynamics is beneficial for generic tasks like interpolation, extrapolation, and classification. However, existing methods approximate the dynamics using unconstrained neural networks, which struggle to adapt reliably under distributional shifts. In this paper, we recast time-series problems as the continuous ODE-based optimal control problem. Rather than learning dynamics solely from data, we optimize control actions that steer ODE trajectories toward task objectives, bringing control-theoretical performance guarantees. To achieve this goal, we need to (1) design the appropriate control actions and (2) apply effective optimal control algorithms. As the actions should contain rich context information, we propose to employ the discrete-time model to process past sequences and generate actions, leading to a coordinate model to extract long-term temporal features to modulate short-term continuous dynamics. During training, we apply model predictive control to plan multi-step future trajectories, minimize a task-specific cost, and greedily select the optimal current action. We show that, under mild assumptions, this multi-horizon optimization leads to exponential convergence to infinite-horizon solutions, indicating that the coordinate model can gain robust and generalizable performance. Extensive experiments on diverse time-series datasets validate our method's superior generalization and adaptability compared to state-of-the-art baselines.
Problem
Research questions and friction points this paper is trying to address.
Coordinate discrete- and continuous-time models for time-series analysis
Improve robustness under distributional shifts in dynamics
Optimize control actions for ODE-based optimal control problem
Innovation
Methods, ideas, or system contributions that make the work stand out.
Combines discrete- and continuous-time models for dynamics
Uses Neural Predictive Control for optimal actions
Ensures exponential convergence with multi-horizon optimization
🔎 Similar Papers
2024-09-20arXiv.orgCitations: 0
