Support-Compiled Feature Folding: More Evidence at Lower Memory Across Tabular Foundation Models

2026-09-23

AI Summary

本文提出支持编译特征折叠方法，解决表格基础模型中特征侧扩展问题，通过线性工作量处理特征交互，提高准确性和降低内存消耗。

Abstract

Tabular foundation models face a feature-side scaling dilemma: full-width pairwise mixing grows quadratically with the number of columns, whereas feature selection saves memory by discarding evidence. We introduce Support-Compiled Feature Folding (SCFF), a training-free inference framework that resolves this dilemma without changing the frozen backbone. SCFF routes support-ranked features through bounded leaves of the native feature encoder, support-checks the residual evidence, and merges the encoded messages before a single contextual prediction. It thereby converts quadratic feature-interaction work into linear-in-width work with a bounded local working set, without ensembling predictions or training new parameters. On the exhaustive 18-dataset wide-table slice of fixed AMLB-29, TabZilla, and TabArena snapshots, SCFF improves dataset-macro accuracy and NLL on all six evaluated backbones. All four matched-width comparisons retain favorable 95 percent dataset-bootstrap intervals on locked folds, with relative error reductions up to 26.1 percent. Median paired GPU-memory savings are 2.09x to 2.36x, and the ratio of separately observed maximum peaks reaches 34.3x. Under a measured peak-memory ceiling, SCFF uses the saved budget to preserve more support-selected evidence, improving accuracy by 4.06 and 3.72 points over the widest feasible single leaf on predeclared wide-Core strata of TabICLv2 and TabPFN-3.

Problem

• Tabular Foundation Models
• Feature Scaling
• Memory Efficiency
• Evidence Preservation

Innovation

• Support-Compiled Feature Folding
• linear-in-width work
• memory savings
• feature interaction
• contextual prediction