A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks

2026-03-23

AI Summary

This work addresses the limited flexibility of existing time series foundation models in performing multi-task reasoning through explicit instructions and contextual examples. To overcome this, we propose an instruction-conditioned in-context learning framework for time series modeling that jointly integrates structured instructions with contextual examples for the first time. Built upon a quantile regression T5 architecture, our model employs a hierarchical Transformer to separately handle intra-example encoding, inter-example fusion, and cross-example attention. Trained via a hybrid self-supervised and supervised multi-task objective—encompassing forecasting, imputation, reconstruction, classification, and anomaly detection—the model achieves general-purpose inference without task-specific fine-tuning. Experimental results on benchmarks such as FEV-Bench and GIFT-Eval demonstrate superior performance in both point and probabilistic forecasting compared to strong baselines, while maintaining competitive accuracy in classification and anomaly detection tasks.

Technology Category

Machine Learning: Time-Series/Data StreamsReasoning under Uncertainty: Relational Probabilistic ModelsComputer Vision: Multi-modal Vision

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingGraph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In-context learning (ICL) allows a model to adapt at inference time by conditioning on examples rather than updating parameters. Existing time-series foundation models use implicit positional context, retrieval, or task-specific objectives, but rarely explicit instruction-conditioned demonstrations. We present a foundation model for instruction-conditioned in-context time-series tasks based on a quantile-regression T5 encoder-decoder. Historical examples and queries are encoded with a structured tokenization scheme that marks target series, covariates, context, and task-specific future information. A hierarchical Transformer with per-example encoding, example-level fusion, and cross-example attention conditions decoding on demonstration pairs, enabling forecasting and related tasks without task-specific fine-tuning. We train on large-scale real and synthetic time series using supervised forecasting plus self-supervised tasks, including imputation, reconstruction, classification, anomaly detection, and source demixing. This multi-task training learns a distribution over task mappings and improves adaptation to local structure at inference time. Across diverse datasets, frequencies, and horizons, our method outperforms strong foundation baselines on point and probabilistic forecasting benchmarks, including fev-bench and GIFT-Eval, while remaining competitive on classification and anomaly detection.

Problem

• instruction-conditioned
• in-context learning
• time series
• foundation model
• multi-task adaptation

Innovation

• instruction-conditioned learning
• in-context learning
• time series foundation model
• hierarchical Transformer
• multi-task self-supervision