Probing the Robustness of Time-series Forecasting Models with CounterfacTS

2024-03-06

arXiv.org

AI Summary

Real-world time-series forecasting models often suffer from reduced robustness when encountering out-of-distribution (OOD) data induced by concept drift. To address this, we propose CounterfacTS—the first interpretable counterfactual probing framework specifically designed for time-series forecasting, enabling interactive time-series transformation, predictive contrast, and feature-space visualization. Our contributions are threefold: (1) directionally guided counterfactual generation via time-series feature embedding and projection, facilitating proactive OOD scenario detection; (2) precise localization of critical temporal features and quantitative assessment of model sensitivity to them; and (3) an open-source, modular evaluation architecture supporting plug-and-play integration with diverse forecasting models. Extensive experiments demonstrate that CounterfacTS significantly enhances prediction stability and interpretability under concept drift, offering practitioners a principled tool for diagnosing and improving model resilience.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Computer Vision: Interpretability, Explainability, and Transparency
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

A common issue for machine learning models applied to time-series forecasting is the temporal evolution of the data distributions (i.e., concept drift). Because most of the training data does not reflect such changes, the models present poor performance on the new out-of-distribution scenarios and, therefore, the impact of such events cannot be reliably anticipated ahead of time. We present and publicly release CounterfacTS, a tool to probe the robustness of deep learning models in time-series forecasting tasks via counterfactuals. CounterfacTS has a user-friendly interface that allows the user to visualize, compare and quantify time series data and their forecasts, for a number of datasets and deep learning models. Furthermore, the user can apply various transformations to the time series and explore the resulting changes in the forecasts in an interpretable manner. Through example cases, we illustrate how CounterfacTS can be used to i) identify the main features characterizing and differentiating sets of time series, ii) assess how the model performance depends on these characateristics, and iii) guide transformations of the original time series to create counterfactuals with desired properties for training and increasing the forecasting performance in new regions of the data distribution. We discuss the importance of visualizing and considering the location of the data in a projected feature space to transform time-series and create effective counterfactuals for training the models. Overall, CounterfacTS aids at creating counterfactuals to efficiently explore the impact of hypothetical scenarios not covered by the original data in time-series forecasting tasks.

Problem

• Addressing robustness of time series models under distribution drift
• Providing interpretable time series generation for data augmentation
• Enhancing model evaluation with visual analytics for OOD scenarios

Innovation

• Combines time-series decompositions with analytic functions
• Generates interpretable in- and out-of-distribution time series
• Visual analytics tool for exploring and evaluating robustness