Discriminative Representation learning via Attention-Enhanced Contrastive Learning for Short Text Clustering

2025-01-07

AI Summary

In short-text clustering, contrastive learning often introduces false-negative samples, erroneously separating semantically similar instances. To address this, we propose Attention-Enhanced Contrastive Learning (AECL), a novel framework that jointly mitigates false negatives and improves representation discriminability. Our key contributions are: (1) a sample-level attention mechanism that explicitly models cross-text semantic similarity; (2) co-optimization of pseudo-label generation and positive-pair construction to suppress false negatives at their source; and (3) an integrated objective combining pseudo-label-guided self-supervision, attention-weighted feature aggregation, similarity-aware contrastive loss, and consistency-driven representation optimization. Extensive experiments on multiple benchmark short-text datasets demonstrate that AECL consistently outperforms state-of-the-art methods, achieving an average 3.2% improvement in clustering accuracy. The implementation will be made publicly available.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

Contrastive learning has gained significant attention in short text clustering, yet it has an inherent drawback of mistakenly identifying samples from the same category as negatives and then separating them in the feature space (false negative separation), which hinders the generation of superior representations. To generate more discriminative representations for efficient clustering, we propose a novel short text clustering method, called Discriminative Representation learning via extbf{A}ttention- extbf{E}nhanced extbf{C}ontrastive extbf{L}earning for Short Text Clustering ( extbf{AECL}). The extbf{AECL} consists of two modules which are the pseudo-label generation module and the contrastive learning module. Both modules build a sample-level attention mechanism to capture similarity relationships between samples and aggregate cross-sample features to generate consistent representations. Then, the former module uses the more discriminative consistent representation to produce reliable supervision information for assist clustering, while the latter module explores similarity relationships and consistent representations optimize the construction of positive samples to perform similarity-guided contrastive learning, effectively addressing the false negative separation issue. Experimental results demonstrate that the proposed extbf{AECL} outperforms state-of-the-art methods. If the paper is accepted, we will open-source the code.

Problem

• Contrastive Learning
• Short Text Classification
• Classification Error

Innovation

• Attention Mechanism
• Contrastive Learning
• Short Text Classification