IN-Flow: Instance Normalization Flow for Non-stationary Time Series Forecasting

2024-01-30

AI Summary

To address performance degradation in non-stationary time series forecasting caused by distributional shift, this paper proposes a decoupled modeling framework that separates distribution correction from forecasting and introduces a bilevel optimization paradigm for joint learning. Its core innovation is Instance Normalization Flow (IN-Flow)—a reversible, bidirectional, and highly expressive temporal distribution transformation network explicitly designed for forecasting, overcoming the limitation of conventional normalizing flows restricted to generative tasks. IN-Flow integrates instance normalization layers with stacked invertible neural networks: the outer level optimizes distribution transformation, while the inner level optimizes forecasting, ensuring compatibility with arbitrary forecasting architectures and eliminating reliance on statistical assumptions. Extensive experiments on synthetic and diverse real-world datasets demonstrate significant improvements over state-of-the-art methods, strong robustness to unseen distribution shifts, and simultaneous gains in both predictive accuracy and generalization capability.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Distributed Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Due to the non-stationarity of time series, the distribution shift problem largely hinders the performance of time series forecasting. Existing solutions either rely on using certain statistics to specify the shift, or developing specific mechanisms for certain network architectures. However, the former would fail for the unknown shift beyond simple statistics, while the latter has limited compatibility on different forecasting models. To overcome these problems, we first propose a decoupled formulation for time series forecasting, with no reliance on fixed statistics and no restriction on forecasting architectures. This formulation regards the removing-shift procedure as a special transformation between a raw distribution and a desired target distribution and separates it from the forecasting. Such a formulation is further formalized into a bi-level optimization problem, to enable the joint learning of the transformation (outer loop) and forecasting (inner loop). Moreover, the special requirements of expressiveness and bi-direction for the transformation motivate us to propose instance normalization flow (IN-Flow), a novel invertible network for time series transformation. Different from the classic"normalizing flow"models, IN-Flow does not aim for normalizing input to the prior distribution (e.g., Gaussian distribution) for generation, but creatively transforms time series distribution by stacking normalization layers and flow-based invertible networks, which is thus named"normalization"flow. Finally, we have conducted extensive experiments on both synthetic data and real-world data, which demonstrate the superiority of our method.

Problem

• Addresses non-stationarity in time series forecasting.
• Proposes decoupled formulation for distribution shift.
• Introduces IN-Flow for effective time series transformation.

Innovation

• Decoupled formulation for forecasting
• Bi-level optimization for joint learning
• Instance Normalization Flow transformation