Regularized Sparse Optimal Discriminant Clustering

2025-01-17

AI Summary

To address the degraded clustering accuracy caused by weak discriminative structure and low inter-cluster separability in high-dimensional, small-sample data, this paper proposes an improved Sparse Optimal Discriminative Clustering (SODC) method. The core contribution is the first incorporation of a convex clustering regularizer into the SODC framework—preserving the orthogonality constraint on the scoring matrix while explicitly modeling intra-cluster compactness and inter-cluster separability. We design a Majorization-Minimization (MM)-based iterative optimization algorithm that jointly enforces orthogonality and clustering structure consistency. Theoretical analysis and extensive experiments on synthetic and multiple real-world high-dimensional datasets demonstrate that the proposed method significantly improves clustering accuracy and robustness, effectively mitigating overfitting and structural distortion.

Technology Category

Machine Learning: ClusteringSearch and Optimization: Non-convex OptimizationConstraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphsWeb Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web textSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We propose a new method based on sparse optimal discriminant clustering (SODC), by a penalty term to scoring matrix based on convex clustering. With the addition of this penalty term, it is expected to improve the accuracy of cluster identification by attaching points from the same cluster closer together and points from different clusters further apart. Moreover, we develop a novel algorithm to derive the updated formula of this scoring matrix using majorizing function. It solves the difficulty to satisfy both constraint and containing the clustering structure to the scoring matrix. We have demonstrated the numerical simulations and its an application to real data to assess the performance of the proposed method.

Problem

• Improving cluster identification accuracy through penalty regularization
• Developing algorithm to enforce orthogonal constraints on scoring matrix
• Enhancing visualization of clustering structure in data analysis

Innovation

• Incorporates penalty term into scoring matrix
• Uses ADMM algorithm for matrix updates
• Enforces orthogonal constraint with clustering structure