Technology Category
Machine Learning: Graph-based Machine LearningPlanning, Routing, and Scheduling: Optimization of Spatio-temporal SystemsKnowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting
🤖 AI Summary
Real-world spatiotemporal data arrive in streams, and the underlying graph structure dynamically expands—posing dual challenges for online forecasting: inefficient model retraining and catastrophic forgetting. To address this, we propose the first prompt-tuning framework for dynamic spatiotemporal graphs, grounded in two principles—*expansion* and *compression*. Our method employs a reusable, continuous prompt pool to retain historical knowledge while enabling rapid adaptation to newly added nodes or time intervals. It integrates lightweight prompt fine-tuning of spatiotemporal graph neural networks, dynamic prompt pool management, and joint optimization. Evaluated on multiple real-world traffic and environmental datasets, our approach consistently outperforms state-of-the-art methods. Crucially, it achieves superior prediction accuracy, training efficiency, and cross-scenario generalization while introducing fewer than 0.5% additional parameters.
Technology Category
Application Category
📝 Abstract
The widespread deployment of sensing devices leads to a surge in data for spatio-temporal forecasting applications such as traffic flow, air quality, and wind energy. Although spatio-temporal graph neural networks have achieved success in modeling various static spatio-temporal forecasting scenarios, real-world spatio-temporal data are typically received in a streaming manner, and the network continuously expands with the installation of new sensors. Thus, spatio-temporal forecasting in streaming scenarios faces dual challenges: the inefficiency of retraining models over newly arrived data and the detrimental effects of catastrophic forgetting over long-term history. To address these challenges, we propose a novel prompt tuning-based continuous forecasting method, following two fundamental tuning principles guided by empirical and theoretical analysis: expand and compress, which effectively resolve the aforementioned problems with lightweight tuning parameters. Specifically, we integrate the base spatio-temporal graph neural network with a continuous prompt pool, utilizing stored prompts (i.e., few learnable parameters) in memory, and jointly optimize them with the base spatio-temporal graph neural network. This method ensures that the model sequentially learns from the spatio-temporal data stream to accomplish tasks for corresponding periods. Extensive experimental results on multiple real-world datasets demonstrate the multi-faceted superiority of our method over the state-of-the-art baselines, including effectiveness, efficiency, universality, etc.
Problem
Research questions and friction points this paper is trying to address.
Address inefficiency of retraining models with new data
Mitigate catastrophic forgetting in long-term history
Enable continuous spatio-temporal forecasting with lightweight tuning
Innovation
Methods, ideas, or system contributions that make the work stand out.
Prompt tuning-based continuous forecasting method
Expand and compress tuning principles
Continuous prompt pool integration
🔎 Similar Papers
No similar papers found.
