A mini-batch training strategy for deep subspace clustering networks

2025-07-26

AI Summary

Existing deep subspace clustering (DSC) methods rely on full-batch training, as the self-expression module requires global data to construct the affinity matrix—limiting scalability to high-resolution images and large-scale models. To address this, we propose the first mini-batch DSC framework: (1) a learnable, updateable memory bank caches global feature representations, decoupling self-expression computation from batch dependencies; and (2) a decoder-free contrastive learning objective jointly optimizes feature discriminability and subspace structure consistency. Our approach abandons the conventional full-batch autoencoder paradigm. On COIL100 and ORL, it achieves clustering performance on par with full-batch baselines while reducing GPU memory consumption and training time by significant margins. This enables efficient, scalable DSC for complex deep architectures—establishing a new pathway for large-scale subspace clustering.

Technology Category

• Machine Learning: Clustering
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Distributed Search

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Mini-batch training is a cornerstone of modern deep learning, offering computational efficiency and scalability for training complex architectures. However, existing deep subspace clustering (DSC) methods, which typically combine an autoencoder with a self-expressive layer, rely on full-batch processing. The bottleneck arises from the self-expressive module, which requires representations of the entire dataset to construct a self-representation coefficient matrix. In this work, we introduce a mini-batch training strategy for DSC by integrating a memory bank that preserves global feature representations. Our approach enables scalable training of deep architectures for subspace clustering with high-resolution images, overcoming previous limitations. Additionally, to efficiently fine-tune large-scale pre-trained encoders for subspace clustering, we propose a decoder-free framework that leverages contrastive learning instead of autoencoding for representation learning. This design not only eliminates the computational overhead of decoder training but also provides competitive performance. Extensive experiments demonstrate that our approach not only achieves performance comparable to full-batch methods, but outperforms other state-of-the-art subspace clustering methods on the COIL100 and ORL datasets by fine-tuning deep networks.

Problem

• Enables mini-batch training for deep subspace clustering networks
• Overcomes full-batch processing bottleneck in self-expressive layers
• Proposes decoder-free framework for efficient large-scale encoder fine-tuning

Innovation

• Mini-batch training with memory bank
• Decoder-free contrastive learning framework
• Fine-tuning pre-trained encoders efficiently