Bayesian Learning-driven Prototypical Contrastive Loss for Class-Incremental Learning

2024-05-17

arXiv.org

AI Summary

To address catastrophic forgetting in class-incremental learning, this paper proposes a Bayesian-driven prototype contrastive learning framework. The method introduces Bayesian uncertainty modeling into the prototype contrastive loss for the first time, enabling dynamic, adaptive weighting between cross-entropy and contrastive losses. It establishes a prototype-level contrastive learning paradigm tailored to incremental settings, jointly enforcing intra-class compactness and inter-class separability in the latent space to co-optimize representations of both old and new class prototypes. Evaluated on CIFAR-10, CIFAR-100, and a GNSS interference classification dataset, the approach consistently outperforms state-of-the-art methods, achieving simultaneous improvements in both final accuracy and forgetting rate. These results demonstrate its effectiveness in enhancing representation robustness and sustaining discriminative capability across incremental tasks.

Technology Category

• Machine Learning: Bayesian Learning
• Search and Optimization: Learning to Search
• Computer Vision: Representation Learning for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

The primary objective of methods in continual learning is to learn tasks in a sequential manner over time from a stream of data, while mitigating the detrimental phenomenon of catastrophic forgetting. In this paper, we focus on learning an optimal representation between previous class prototypes and newly encountered ones. We propose a prototypical network with a Bayesian learning-driven contrastive loss (BLCL) tailored specifically for class-incremental learning scenarios. Therefore, we introduce a contrastive loss that incorporates new classes into the latent representation by reducing the intra-class distance and increasing the inter-class distance. Our approach dynamically adapts the balance between the cross-entropy and contrastive loss functions with a Bayesian learning technique. Empirical evaluations conducted on both the CIFAR-10 and CIFAR-100 dataset for image classification and images of a GNSS-based dataset for interference classification validate the efficacy of our method, showcasing its superiority over existing state-of-the-art approaches.

Problem

• Mitigate catastrophic forgetting in sequential task learning
• Learn effective representation between old and new class prototypes
• Dynamically balance cross-entropy and contrastive loss functions

Innovation

• Bayesian learning-driven contrastive loss for representation
• Dynamic balance between cross-entropy and contrastive loss
• Prototypical network for class-incremental learning scenarios