Hyperbolic Multimodal Continual Learning: A Closest-Admissible Solution

2026-09-24

AI Summary

This study addresses the vulnerability of existing hyperbolic multimodal continual learning methods to Lorentz geometric structure degradation, which distorts semantic hierarchies and cross-modal relationships. We propose the HMCL framework, which for the first time reformulates old-task geometry preservation as a shared hyperbolic isometric rotation constraint. To achieve near-optimal parameter updates while ensuring task adaptability, we design Closest Admissible (CA) and Minimal Rotation (MR) correction strategies. Experiments demonstrate that this framework significantly outperforms baselines across 16-task streams, reducing radial and angular drift by 81.2%–95.5%. Consequently, HMCL effectively enhances overall performance, backward transfer, and semantic hierarchy fidelity in multimodal continual learning scenarios.

Abstract

Existing continual-learning methods protect parameters, replayed examples, or Euclidean feature subspaces. When applied to hyperbolic multimodal models, they do not explicitly preserve the Lorentz geometry that jointly encodes within-modality similarity, cross-modal correspondence, and semantic hierarchy; sequential updates can therefore retain task scores while still distorting previously learned relations. We address this gap with Hyperbolic Multimodal Continual Learning (HMCL). We show that preserving the old multimodal geometry amounts to restricting all modalities to one shared hyperbolic isometry, which induces a family of admissible first-order parameter changes. We formulate a joint closest-admissible (CA) correction that retains the shared rotation best matching the candidate modal updates; its minimal-rotation (MR) special case fixes this rotation to zero. Both variants correct the displacement realized by AdamW, and task anchoring bounds within-task accumulation while preserving learning freedom. Across a unified 16-task classification-retrieval stream with three hyperbolic backbones, HMCL improves final performance and backward transfer over sequential fine-tuning and four continual-learning baselines; HMCL-CA gives the highest Overall score on every backbone. A modality-extended stream confirms the retrieval gains. Representation analyses find 81.2 to 95.5 percent less radial, angular, cross-modal, and paired-distance drift; ImageNet-WordNet results show better semantic ancestry and radial hierarchy.

Problem

• continual learning
• hyperbolic geometry
• multimodal models
• Lorentz geometry preservation
• catastrophic forgetting

Innovation

• Hyperbolic Continual Learning
• Multimodal Representation
• Lorentz Geometry Preservation
• Closest-Admissible Correction
• Shared Isometry