TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models

2025-11-11

AI Summary

To address the scalability limitations and accuracy bottlenecks in tabular data modeling, this paper introduces TabPFN-2.5, a next-generation foundation model. It is the first neural process architecture scaled to handle up to 50,000 samples and 2,000 features—achieving a 20× capacity increase over prior models. We propose a novel knowledge distillation engine that generates compact, high-performance surrogate models—including lightweight MLPs or tree ensembles—balancing accuracy and inference latency. On the TabArena benchmark, TabPFN-2.5 matches AutoGluon 1.4 for the first time and consistently outperforms XGBoost. It achieves a 100% win rate against default XGBoost on small-to-medium classification datasets, and maintains strong performance on large-scale data with 87% (classification) and 85% (regression) win rates—significantly surpassing tuned tree-based models.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Generation
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The first tabular foundation model, TabPFN, and its successor TabPFNv2 have impacted tabular AI substantially, with dozens of methods building on it and hundreds of applications across different use cases. This report introduces TabPFN-2.5, the next generation of our tabular foundation model, built for datasets with up to 50,000 data points and 2,000 features, a 20x increase in data cells compared to TabPFNv2. TabPFN-2.5 is now the leading method for the industry standard benchmark TabArena (which contains datasets with up to 100,000 training data points), substantially outperforming tuned tree-based models and matching the accuracy of AutoGluon 1.4, a complex four-hour tuned ensemble that even includes the previous TabPFNv2. Remarkably, default TabPFN-2.5 has a 100% win rate against default XGBoost on small to medium-sized classification datasets (<=10,000 data points, 500 features) and a 87% win rate on larger datasets up to 100K samples and 2K features (85% for regression). For production use cases, we introduce a new distillation engine that converts TabPFN-2.5 into a compact MLP or tree ensemble, preserving most of its accuracy while delivering orders-of-magnitude lower latency and plug-and-play deployment. This new release will immediately strengthen the performance of the many applications and methods already built on the TabPFN ecosystem.

Problem

• Advancing tabular foundation models for larger datasets and features
• Outperforming traditional tree-based models on industry benchmarks
• Enabling efficient deployment through distillation for production use

Innovation

• Handles 50,000 data points and 2,000 features
• Outperforms tree models and matches AutoGluon accuracy
• Distills model into compact MLP or tree ensemble