Mixed-Curvature Decision Trees and Random Forests

2024-06-07

arXiv.org

AI Summary

This work addresses the absence of decision tree and random forest models for heterogeneous curvature data—specifically, product manifolds combining hyperbolic, spherical, and Euclidean spaces. We present the first generalization of these models to mixed-curvature product manifolds, supporting both classification and regression. Methodologically, we introduce a manifold-geometric nonlinear splitting criterion, curvature-adaptive partitioning, a product-space distance metric, and an embedding optimization strategy—overcoming the limitations of linear decision boundaries inherent in Euclidean or tangent-space baselines. Our key contributions are: (1) the first tree-based framework capable of joint modeling across multiple constant curvatures; (2) the first interpretable ensemble method for non-Euclidean regression; and (3) state-of-the-art performance on benchmarks spanning single- and multi-curvature manifolds, with significantly reduced metric distortion and improved classification/regression accuracy over Euclidean and tangent-space counterparts.

Technology Category

Machine Learning: Learning with ManifoldsSearch and Optimization: Mixed Discrete/Continuous SearchConstraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsUser Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalizationEconomics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

We extend decision tree and random forest algorithms to product space manifolds: Cartesian products of Euclidean, hyperspherical, and hyperbolic manifolds. Such spaces have extremely expressive geometries capable of representing many arrangements of distances with low metric distortion. To date, all classifiers for product spaces fit a single linear decision boundary, and no regressor has been described. Our method enables a simple, expressive method for classification and regression in product manifolds. We demonstrate the superior accuracy of our tool compared to Euclidean methods operating in the ambient space or the tangent plane of the manifold across a range of constant-curvature and product manifolds. Code for our implementation and experiments is available at https://github.com/pchlenski/embedders.

Problem

• Extend decision trees to product manifolds
• Handle heterogeneous curvature in data
• Benchmark product random forests on various tasks

Innovation

• Extends DT and RF to product manifolds
• Angular reformulation respects manifold geometry
• Benchmarks show superior performance in tasks