Neural networks on Symmetric Spaces of Noncompact Type

2026-01-03

International Conference on Learning Representations

AI Summary

This work addresses the challenge of designing efficient neural networks on non-compact symmetric spaces, such as hyperbolic space and the manifold of symmetric positive definite (SPD) matrices. The authors propose a unified framework grounded in G-invariant Riemannian metrics and differential geometry, which naturally subsumes existing models as special cases. Central to this framework is the first closed-form solution for the distance between a point and a hyperplane in high-rank non-compact symmetric spaces, enabling the construction of tailored fully connected layers and attention mechanisms. Extensive experiments demonstrate that the proposed architecture achieves significant performance gains across diverse tasks, including image classification, EEG signal processing, image generation, and natural language inference.

Technology Category

Machine Learning: Learning with ManifoldsSearch and Optimization: Non-convex OptimizationComputer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsEconomics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystemsSystems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Recent works have demonstrated promising performances of neural networks on hyperbolic spaces and symmetric positive definite (SPD) manifolds. These spaces belong to a family of Riemannian manifolds referred to as symmetric spaces of noncompact type. In this paper, we propose a novel approach for developing neural networks on such spaces. Our approach relies on a unified formulation of the distance from a point to a hyperplane on the considered spaces. We show that some existing formulations of the point-to-hyperplane distance can be recovered by our approach under specific settings. Furthermore, we derive a closed-form expression for the point-to-hyperplane distance in higher-rank symmetric spaces of noncompact type equipped with G-invariant Riemannian metrics. The derived distance then serves as a tool to design fully-connected (FC) layers and an attention mechanism for neural networks on the considered spaces. Our approach is validated on challenging benchmarks for image classification, electroencephalogram (EEG) signal classification, image generation, and natural language inference.

Problem

• symmetric spaces
• noncompact type
• neural networks
• point-to-hyperplane distance
• Riemannian manifolds

Innovation

• symmetric spaces
• noncompact type
• point-to-hyperplane distance
• Riemannian geometry
• geometric deep learning