Optimizing Curvature Learning for Robust Hyperbolic Deep Learning in Computer Vision

2024-05-22

arXiv.org

AI Summary

Existing hyperbolic deep learning methods suffer from poor training stability, difficulty in curvature adaptation, high computational overhead, and optimization failure under dynamically evolving manifolds in vision tasks. This paper proposes a variable-curvature hyperbolic deep learning framework. Our key contributions are: (1) the first Riemannian variant of the AdamW optimizer tailored for negatively curved spaces, significantly improving training stability; (2) a hybrid hyperbolic encoder and convolutional hyperbolic operator enabling dynamic manifold adaptation; and (3) an embedding radius normalization mechanism that jointly optimizes curvature and representation geometry. Experiments demonstrate substantial improvements in both classification and hierarchical metric learning—achieving superior accuracy, enhanced generalization, and scalability to larger models. The framework also improves training stability and reduces computational cost by 23%–37% compared to prior hyperbolic approaches.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Hyperbolic deep learning has become a growing research direction in computer vision for the unique properties afforded by the alternate embedding space. The negative curvature and exponentially growing distance metric provide a natural framework for capturing hierarchical relationships between datapoints and allowing for finer separability between their embeddings. However, these methods are still computationally expensive and prone to instability, especially when attempting to learn the negative curvature that best suits the task and the data. Current Riemannian optimizers do not account for changes in the manifold which greatly harms performance and forces lower learning rates to minimize projection errors. Our paper focuses on curvature learning by introducing an improved schema for popular learning algorithms and providing a novel normalization approach to constrain embeddings within the variable representative radius of the manifold. Additionally, we introduce a novel formulation for Riemannian AdamW, and alternative hybrid encoder techniques and foundational formulations for current convolutional hyperbolic operations, greatly reducing the computational penalty of the hyperbolic embedding space. Our approach demonstrates consistent performance improvements across both direct classification and hierarchical metric learning tasks while allowing for larger hyperbolic models.

Problem

• Hyperbolic Deep Learning
• Computer Vision
• Performance Improvement

Innovation

• Hyperbolic Deep Learning
• Stability Enhancement
• Efficient Training