Physiome-ODE: A Benchmark for Irregularly Sampled Multivariate Time Series Forecasting Based on Biological ODEs

2025-02-11

AI Summary

Existing benchmarks for irregularly sampled multivariate time series (IMTS) forecasting are inadequate: mainstream evaluation relies on only four small-scale datasets, and on three of them, a naive constant-value baseline outperforms state-of-the-art ordinary differential equation (ODE)-based models proposed within the last five years—severely underestimating ODE modeling potential. Method: We propose Physiome-ODE, the first large-scale IMTS forecasting benchmark grounded in real biological physiology, synthesizing data from 50 physiologically grounded ODE systems and employing dynamics-aware rejection sampling to select high-challenge instances. Contribution/Results: Physiome-ODE establishes a novel “biological ODE modeling + dynamics-aware rejection sampling” paradigm for IMTS generation. Experiments demonstrate that it substantially amplifies the relative advantage of ODE-based models, inducing qualitative shifts in method ranking; its model discriminability exceeds that of the conventional four-dataset benchmark by an order of magnitude, thereby catalyzing the resurgence of ODE-driven temporal modeling.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Time-Series/Data Streams

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

State-of-the-art methods for forecasting irregularly sampled time series with missing values predominantly rely on just four datasets and a few small toy examples for evaluation. While ordinary differential equations (ODE) are the prevalent models in science and engineering, a baseline model that forecasts a constant value outperforms ODE-based models from the last five years on three of these existing datasets. This unintuitive finding hampers further research on ODE-based models, a more plausible model family. In this paper, we develop a methodology to generate irregularly sampled multivariate time series (IMTS) datasets from ordinary differential equations and to select challenging instances via rejection sampling. Using this methodology, we create Physiome-ODE, a large and sophisticated benchmark of IMTS datasets consisting of 50 individual datasets, derived from real-world ordinary differential equations from research in biology. Physiome-ODE is the first benchmark for IMTS forecasting that we are aware of and an order of magnitude larger than the current evaluation setting of four datasets. Using our benchmark Physiome-ODE, we show qualitatively completely different results than those derived from the current four datasets: on Physiome-ODE ODE-based models can play to their strength and our benchmark can differentiate in a meaningful way between different IMTS forecasting models. This way, we expect to give a new impulse to research on ODE-based time series modeling.

Problem

• Develops benchmark for irregular multivariate time series
• Generates datasets from biological ordinary differential equations
• Enhances evaluation of ODE-based forecasting models

Innovation

• Generates IMTS from biological ODEs
• Uses rejection sampling for challenge
• Creates large benchmark Physiome-ODE