Community detection with the Bethe-Hessian

2024-11-05

AI Summary

This work addresses the challenges of estimating the number of communities and achieving weak recovery in sparse network community detection. We propose a spectral clustering method based on the Bethe–Hessian matrix. Theoretically, we prove that the number of negative outlier eigenvalues consistently estimates the true number of communities, and the method achieves the Kesten–Stigum detectability threshold for average degree $d geq 2$. It yields consistent community number estimation under both bounded-degree and growing-degree regimes. Moreover, as $d o infty$, the negative outliers concentrate asymptotically, enabling weak consistency in community recovery. This is the first rigorous quantitative characterization linking the spectral structure of the Bethe–Hessian matrix to the detectability threshold of the stochastic block model (SBM). Our results provide both theoretical foundations and practical tools for spectral methods on sparse graphs.

Technology Category

• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Social Networks and Social Media: Computational social science

Abstract

The Bethe-Hessian matrix, introduced by Saade, Krzakala, and Zdeborov'a (2014), is a Hermitian matrix designed for applying spectral clustering algorithms to sparse networks. Rather than employing a non-symmetric and high-dimensional non-backtracking operator, a spectral method based on the Bethe-Hessian matrix is conjectured to also reach the Kesten-Stigum detection threshold in the sparse stochastic block model (SBM). We provide the first rigorous analysis of the Bethe-Hessian spectral method in the SBM under both the bounded expected degree and the growing degree regimes. Specifically, we demonstrate that: (i) When the expected degree $dgeq 2$, the number of negative outliers of the Bethe-Hessian matrix can consistently estimate the number of blocks above the Kesten-Stigum threshold, thus confirming a conjecture from Saade, Krzakala, and Zdeborov'a (2014) for $dgeq 2$. (ii) For sufficiently large $d$, its eigenvectors can be used to achieve weak recovery. (iii) As $d oinfty$, we establish the concentration of the locations of its negative outlier eigenvalues, and weak consistency can be achieved via a spectral method based on the Bethe-Hessian matrix.

Problem

• Analyzing Bethe-Hessian matrix for community detection in sparse networks
• Rigorous evaluation of Bethe-Hessian spectral method in stochastic block model
• Confirming detection threshold and weak recovery via Bethe-Hessian eigenvalues

Innovation

• Uses Bethe-Hessian matrix for spectral clustering
• Estimates blocks via negative eigenvalue outliers
• Achieves weak recovery with eigenvectors