Learning Clustering-based Prototypes for Compositional Zero-shot Learning

2025-02-10

AI Summary

Conventional compositional zero-shot learning (CZSL) methods model primitive concepts (attributes/objects) in a monolithic manner, neglecting their semantic diversity across different compositions. Method: We propose ClusPro, a parameter-free framework that dynamically discovers diverse prototypes via adaptive clustering in the embedding space, thereby constructing a disentangled and structured representation space for primitives. It introduces an updateable prototype set to characterize concept boundaries and jointly optimizes prototype-driven contrastive learning and decorrelation learning to enforce intra-class compactness and inter-class disentanglement—without introducing any additional parameters. Contribution/Results: ClusPro achieves significant improvements over state-of-the-art methods on three standard benchmarks under both closed-world and open-world evaluation protocols, demonstrating superior generalization capability and principled modeling of semantic compositionality.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Clustering
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Learning primitive (i.e., attribute and object) concepts from seen compositions is the primary challenge of Compositional Zero-Shot Learning (CZSL). Existing CZSL solutions typically rely on oversimplified data assumptions, e.g., modeling each primitive with a single centroid primitive representation, ignoring the natural diversities of the attribute (resp. object) when coupled with different objects (resp. attribute). In this work, we develop ClusPro, a robust clustering-based prototype mining framework for CZSL that defines the conceptual boundaries of primitives through a set of diversified prototypes. Specifically, ClusPro conducts within-primitive clustering on the embedding space for automatically discovering and dynamically updating prototypes. These representative prototypes are subsequently used to repaint a well-structured and independent primitive embedding space, ensuring intra-primitive separation and inter-primitive decorrelation through prototype-based contrastive learning and decorrelation learning. Moreover, ClusPro efficiently performs prototype clustering in a non-parametric fashion without the introduction of additional learnable parameters or computational budget during testing. Experiments on three benchmarks demonstrate ClusPro outperforms various top-leading CZSL solutions under both closed-world and open-world settings.

Problem

• Challenges in learning primitive concepts for CZSL
• Overcomes oversimplified data assumptions in CZSL
• Develops robust clustering-based prototype mining framework

Innovation

• Clustering-based prototype mining
• Non-parametric prototype clustering
• Prototype-based contrastive learning