🤖 AI Summary
This work proposes a unified geometric framework for the joint analysis of prediction, interpretability, robustness, and generalization in tree ensemble models. By leveraging node-weighted feature mappings and path-isometric embeddings, the framework constructs a non-diagonal Gram kernel representation in a squared Euclidean space, thereby realizing a metric embedding of tree ensembles. For the first time, it integrates exact additive attributions, deterministic Lipschitz robustness radii (based on the KPP metric), and Rademacher generalization bounds within a single kernel structure. Under honesty and cross-fitting conditions, the authors derive a unified risk bound applicable to both regression and classification. The study further establishes the existence of fast convergence rates and formulates related open problems.
📝 Abstract
A recent line of work has reframed individual decision trees as linear models on engineered features associated with their splits, opening routes for oracle inequalities and feature-importance reinterpretation, but leaving open the question of what unified geometric object a forest induces when one indexes its feature map by nodes rather than by splits. The present paper studies that object. KPP indexes the feature map by the nodes of the forest, weighted by a path metric that turns each coordinate into a component of a squared-Euclidean path-isometric embedding. KPP unifies four pillars under a single non-diagonal Gram that carries a metric: prediction, exact additive attribution, deterministic Lipschitz robust radius in the KPP metric, and uniform Rademacher risk bounds for regression and classification under fixed, honest, or cross-fit conditioning. All probabilistic guarantees are conditional on the representation and are stated under three explicit conditioning regimes; the robust-radius guarantee is deterministic in the KPP metric rather than in a norm on the raw input. Conjectured fast-rate refinements for both regression and classification are stated as open problems and are not claimed as theorems.