The Implicit Bias of Hyperbolic Representation Learning for Multiclass Data: A Busemann Risk Perspective

2026-10-05

AI Summary

This study investigates the implicit bias mechanism of Riemannian gradient flow in multi-class classification within hyperbolic space. Leveraging a Busemann function decomposition, it analyzes the asymptotic behavior of the PERM loss under fixed prototypes and introduces a radial dichotomy to establish a theoretical framework from a Busemann risk perspective. This framework rigorously explains boundary saturation and near-boundary clustering phenomena. Furthermore, the analysis demonstrates that the radius either grows logarithmically or returns in finite time, while the boundary direction converges to critical points of the Busemann risk. These findings provide rigorous theoretical support for understanding the optimization dynamics underlying hyperbolic representation learning.

Abstract

We study the implicit bias of Riemannian gradient flow for hyperbolic multiclass classification with fixed class prototypes in hyperbolic space $\mathbb{H}^n$. Our framework accommodates general permutation invariant relative margin (PERM) losses, a class that includes cross entropy and other standard multiclass losses. Our analysis is based on a decomposition: at large radius, the distance to each prototype splits into a radial term and a direction-dependent term described by the Busemann function. This yields two main results. First, we prove a radial dichotomy: the sign of a drift coefficient $μ$ determines whether the radius is pushed toward the ideal boundary or back toward the interior; if the positive drift persists, then $r(t)=\frac{1}{2}\log t+O(1)$, while persistent negative drift returns the trajectory to the large-radius threshold in finite time. Second, we show that the boundary direction converges to a critical point of the Busemann risk on $\partial\mathbb{H}^n$. These results provide a rigorous asymptotic perspective on two phenomena we refer to as boundary saturation and near-boundary clustering in hyperbolic representation learning.

Problem

• Hyperbolic representation learning
• Implicit bias
• Multiclass classification
• Busemann risk
• Riemannian gradient flow

Innovation

• Hyperbolic Representation Learning
• Implicit Bias
• Busemann Risk
• Riemannian Gradient Flow
• Radial Dichotomy