Deep clustering using adversarial net based clustering loss

2024-12-12

AI Summary

Unsupervised deep clustering is often constrained by reliance on explicit cluster centroids and closed-form loss functions (e.g., KL divergence), limiting modeling flexibility and requiring prior knowledge of cluster count or structure. Method: This paper reformulates clustering as a minimax adversarial game between an encoder and a discriminator—integrating a deep autoencoder with a GAN framework to minimize the Jensen–Shannon (JS) divergence between the encoded sample distribution and a uniform prior in latent space, without pre-specifying cluster number, centroids, or hand-crafted loss functions. Contribution/Results: We theoretically prove that the adversarial optimization converges to JS divergence minimization, ensuring interpretability and generalizability. Extensive experiments on MNIST, SVHN, USPS, and CIFAR-10 demonstrate state-of-the-art or competitive clustering performance, significantly enhancing robustness and adaptability of unsupervised representation learning.

Technology Category

Machine Learning: ClusteringSearch and Optimization: Learning to SearchComputer Vision: Representation Learning for Vision

Application Category

Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphsWeb Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web textUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Deep clustering is a recent deep learning technique which combines deep learning with traditional unsupervised clustering. At the heart of deep clustering is a loss function which penalizes samples for being an outlier from their ground truth cluster centers in the latent space. The probabilistic variant of deep clustering reformulates the loss using KL divergence. Often, the main constraint of deep clustering is the necessity of a closed form loss function to make backpropagation tractable. Inspired by deep clustering and adversarial net, we reformulate deep clustering as an adversarial net over traditional closed form KL divergence. Training deep clustering becomes a task of minimizing the encoder and maximizing the discriminator. At optimality, this method theoretically approaches the JS divergence between the distribution assumption of the encoder and the discriminator. We demonstrated the performance of our proposed method on several well cited datasets such as SVHN, USPS, MNIST and CIFAR10, achieving on-par or better performance with some of the state-of-the-art deep clustering methods.

Problem

• Deep Clustering
• Unsupervised Learning
• Training Instability

Innovation

• Deep Clustering
• Adversarial Game
• Representation Learning