Reducing Bias and Variance: Generative Semantic Guidance and Bi-Layer Ensemble for Image Clustering

2026-05-12

AI Summary

This work addresses the limitations of existing image clustering methods, which are constrained by the semantic expressiveness of predefined vocabularies and often neglect variance control, thereby compromising performance. To overcome these issues, the paper proposes a novel framework that integrates generative semantic guidance with a two-level ensemble learning strategy. At the inner level, a multimodal large language model generates rich semantic descriptions, and BatchEnsemble is employed to weight and fuse visual and semantic embeddings. At the outer level, an alignment mechanism integrates ensemble outputs to jointly reduce bias and variance. This approach represents the first integration of generative semantic guidance with a dual-layer ensemble architecture, transcending the constraints of conventional matching spaces and substantially enhancing both semantic adaptability and clustering robustness. Extensive experiments on six benchmark datasets demonstrate its superiority over 18 state-of-the-art methods.

Abstract

Image clustering aims to partition unlabeled image datasets into distinct groups. A core aspect of this task is constructing and leveraging prior knowledge to guide the clustering process. Recent approaches introduce semantic descriptions as prior information, most of which typically relying on matching-based techniques with predefined vocabularies. However, the limited matching space restricts their adaptability to downstream clustering tasks. Moreover, these methods primarily focus on reducing bias to improve performance, frequently overlooking the importance of variance reduction. To address these limitations, we propose GSEC (Image Clustering based on Generative Semantic Guidance and Bi-Layer Ensemble), a framework designed to reduce bias through generative semantic guidance and mitigate variance via ensemble learning. Our method employs Multimodal Large Language Models to generate semantic descriptions and derive image embeddings via weighted averaging. Additionally, a bi-layer ensemble strategy integrates cross-modal information through BatchEnsemble in the inner layer and aligns outputs via an alignment mechanism in the outer layer. Comparative experiments demonstrate that GSEC outperforms 18 state-of-the-art methods across six benchmark datasets, while further analysis confirms its effectiveness in simultaneously reducing both bias and variance. The code is available at https://github.com/2017LI/GSEC.git.

Problem

• image clustering
• bias reduction
• variance reduction
• semantic guidance
• prior knowledge

Innovation

• Generative Semantic Guidance
• Bi-Layer Ensemble
• Bias-Variance Tradeoff
• Multimodal Large Language Models
• Image Clustering