Hierarchical Prompt Learning for Hyperbolic Vision-Language Models

2026-09-21

AI Summary

本文提出了一种层次提示学习方法，通过增加父类提示学习器、父级监督等手段改进了双曲视觉-语言模型在下游任务中的表现，特别是在固定分类体系中对新类别的处理。

Abstract

Hyperbolic vision-language models (VLMs) represent image and text features in a geometry naturally suited to hierarchy, but their adaptation to downstream tasks has largely relied on fixed prompts. Existing prompt learning methods, meanwhile, treat class labels as a flat set and do not exploit available taxonomic structure. We address this gap with a hierarchical prompt learning plug-in for frozen hyperbolic VLMs. Given a fixed offline parent-class hierarchy, it augments a class prompt learner with a separate parent prompt learner, parent-level supervision, hyperbolic entailment regularization, and parent-feedback logit fusion. We instantiate the method with CoOp, CoCoOp and MaPLe, yielding HyPLO, CoHyPLO and MaHyPLO. Across the standard 11-dataset benchmark, all variants improve base-to-new generalization and cross-dataset transfer, and remain comparable to their prompt learning baselines under domain shift. Six hierarchical metrics and embedding analyses show that the method produces more taxonomically consistent predictions and induces a hierarchy-consistent organization of parent, class, and image embeddings in hyperbolic space. Its gains are largest when novel classes must be placed within a fixed taxonomy, and smallest for fine-grained confusions among sibling classes or shifts affecting only the image distribution.

Problem

• Hierarchical Prompt Learning
• Hyperbolic Vision-Language Models
• Taxonomic Structure

Innovation

• Hierarchical Prompt Learning
• Hyperbolic Vision-Language Models
• Parent Feedback Logit Fusion
• Taxonomic Structure