🤖 AI Summary
This work investigates the generalization capabilities of tabular foundation models in cross-modal settings and introduces a unified evaluation framework accompanied by a standardized classification pipeline. The approach leverages equiangular tight frame (ETF) preprocessing, in-context learning, and probability calibration to systematically assess model performance across 95 datasets spanning seven distinct modalities. A novel validation-free ETF-based training stopping criterion is proposed, along with a lightweight baseline built upon frozen features. The method achieves performance comparable to task-specific fine-tuned models on most benchmarks while accelerating inference by 4–200× and producing well-calibrated confidence estimates, thereby substantially enhancing practical deployability.
📝 Abstract
We present a single classification pipeline that combines an Equiangular Tight Frame (ETF) preprocessing stage with a tabular foundation model for in-context inference, applied identically across modalities once data is mapped to fixed vector representations. We evaluate it on 95 datasets spanning seven signal modalities -- vision, audio, speech, text, molecular, time-series, and tabular. The main methodological contribution is to fix the comparison object: throughout the paper, performance is judged against the strongest lightweight tuned baseline on the same frozen features, while oracle selection, deployed selection, and specialized fine-tuning are reported separately.
The pipeline is broadly competitive with strong lightweight tuned baselines on the same frozen features. It does not match the very best specialized models or heavily tuned pipelines on every task, but it stays close, and it runs much faster -- typically 4 to 200 times faster than full backbone fine-tuning, often at comparable quality.
We describe how to deploy the pipeline in practice: when to apply ETF preprocessing, how to stop its training without a validation split, how to set up the in-context classifier, and how to calibrate the resulting probabilities. The calibration step is non-cosmetic: TabICL produces well-calibrated probabilities by construction, ETF preprocessing initially disrupts that calibration, and the post-hoc rescaling restores it -- yielding a per-prediction confidence signal that practitioners can use as a trust threshold for confidence-gated deployment. We also report where the pipeline should not be expected to help, and how to identify those cases in advance.