TabDPT-Turbo: Efficient In-Context Learning for Tabular Prediction

2026-08-02

AI Summary

This work addresses the trade-off between predictive performance and inference efficiency in existing tabular prediction models, which often sacrifice speed for accuracy, hindering deployment in resource-constrained or latency-sensitive settings. The authors propose an efficient foundation model for tabular data that eschews retrieval mechanisms and instead introduces row-wise attention, combined with long-context pretraining, architectural optimizations, and self-supervised learning on large-scale real-world tabular datasets. The resulting model achieves prediction performance comparable to TabDPT v1.1 on the TabArena-Lite, CC18, and CTR23 benchmarks while accelerating inference by several orders of magnitude. This approach strikingly balances effectiveness and efficiency, establishing a new state-of-the-art as the fastest general-purpose tabular prediction model to date.

Abstract

Tabular foundation models, driven by in-context learning, have rapidly grown in quality and popularity. However, recent approaches with either cell-based architectures or retrieval have sacrificed efficiency for raw performance, restricting their utility in situations where compute is limited or inference speed is crucial. We adopt an alternate approach, sticking with row-based attention while incorporating long context pre-training to eliminate the need for retrieval. By combining this with architectural improvements and SSL pre-training on a newly-sourced, larger corpus of real data results, we present TabDPT-Turbo, a model that provides comparable default performance to TabDPT v1.1 on TabArena-Lite, CC18, and CTR23, at orders of magnitude faster. In our experiments, TabDPT-Turbo is the fastest model overall among leading foundation models. We have released the new model as TabDPT v1.2 at https://github.com/layer6ai-labs/TabDPT-inference.

Problem

• tabular prediction
• in-context learning
• computational efficiency
• foundation models
• inference speed

Innovation

• in-context learning
• row-based attention
• long context pre-training
• self-supervised learning
• tabular foundation model