Rethinking Tabular Foundation Models On Data Streams

2026-10-04

AI Summary

This study addresses the challenges of memory management, computational overhead, and concept drift adaptation faced by tabular foundation models in data stream scenarios. Adopting an in-context learning paradigm, the proposed approach adapts to streaming data variations by dynamically maintaining an exemplar set rather than updating model parameters. The findings reveal that retaining only the most recent samples suffices to handle drift efficiently, significantly outperforming more complex memory strategies, while also uncovering a trade-off between predictive accuracy and inference cost. Experimental results demonstrate that this method achieves optimal predictive performance, recovers more rapidly following concept drift, and remains robust under label delay. These insights provide valuable guidance for future architectural optimization of tabular foundation models in dynamic environments.

Abstract

Tabular foundation models (TFMs) outperform established machine learning models on tabular benchmarks through in-context learning. Building on this success, interest is growing in applying them to data streams, where data arrive continuously and evolve over time. On a stream, a TFM adapts by updating its context rather than its parameters, so its accuracy and cost depend on which examples it keeps and how often it rebuilds its context. We therefore present a systematic study of TFMs on data streams, covering memory management, computational cost, and stream-specific challenges such as concept drift and delayed labels. We find that TFMs achieve the highest predictive performance and that simply retaining the most recent examples is as effective as existing memory management techniques. They also recover faster than streaming learners after drift and keep the highest accuracy under label delay. This accuracy, however, comes at a high serving cost, since a nearly unchanged context is re-encoded at every prediction. These results point to architectural efficiency as the way forward for in-context stream learning.

Problem

• Tabular Foundation Models
• Data Streams
• Concept Drift
• Delayed Labels
• Computational Cost

Innovation

• Tabular Foundation Models
• Data Streams
• In-context Learning
• Concept Drift
• Memory Management