Fréchet Geodesic Boosting

2025-09-22

AI Summary

Traditional gradient boosting struggles with non-Euclidean structured outputs—such as probability distributions, graphs, and manifolds—due to the absence of additive and scalar-multiplicative algebraic structures in their underlying spaces. To address this, we propose Fréchet Geodesic Boosting, the first generalization of gradient boosting to arbitrary metric spaces. Our method replaces Euclidean residuals with geodesic directions, constructs the ensemble path via Fréchet means and geodesic interpolation, and introduces a weighted minimization optimization framework. Crucially, it intrinsically respects the geometric structure of the output space without requiring embedding or linearization. Theoretical analysis establishes convergence and consistency under mild regularity conditions on the metric space and base learners. Experiments on synthetic and real-world tasks—including distributional forecasting and network evolution modeling—demonstrate substantial improvements over state-of-the-art baselines, achieving both superior generalization performance and strict geometric fidelity.

Technology Category

• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Gradient boosting has become a cornerstone of machine learning, enabling base learners such as decision trees to achieve exceptional predictive performance. While existing algorithms primarily handle scalar or Euclidean outputs, increasingly prevalent complex-structured data, such as distributions, networks, and manifold-valued outputs, present challenges for traditional methods. Such non-Euclidean data lack algebraic structures such as addition, subtraction, or scalar multiplication required by standard gradient boosting frameworks. To address these challenges, we introduce Fréchet geodesic boosting (FGBoost), a novel approach tailored for outputs residing in geodesic metric spaces. FGBoost leverages geodesics as proxies for residuals and constructs ensembles in a way that respects the intrinsic geometry of the output space. Through theoretical analysis, extensive simulations, and real-world applications, we demonstrate the strong performance and adaptability of FGBoost, showcasing its potential for modeling complex data.

Problem

• Handling complex-structured data lacking Euclidean algebraic operations
• Modeling outputs in geodesic metric spaces like distributions and manifolds
• Extending gradient boosting to respect intrinsic geometry of output spaces

Innovation

• FGBoost uses geodesics as proxies for residuals
• FGBoost constructs ensembles respecting intrinsic output geometry
• FGBoost handles outputs in geodesic metric spaces