Machine Learning Informed by Micro and Mesoscopic Statistical Physics Methods for Community Detection

2025-04-18

AI Summary

Existing community detection methods—particularly those rooted in statistical physics—focus predominantly on mesoscopic structures and struggle to incorporate microscopic node-pair similarity, thereby limiting deep understanding of complex network organization. Method: We propose the first systematic, low-complexity, machine learning–enhanced framework that bridges microscopic similarity modeling and mesoscopic statistical physics modeling. Our approach constructs a community energy function based on the Potts model, jointly optimizes network embedding and modularity, and employs ensemble learning for cross-scale information fusion. Contribution/Results: Experiments on synthetic and real-world networks demonstrate significant improvements in modularity, normalized mutual information (NMI), and adjusted Rand index (ARI). With ground-truth labels, our method achieves state-of-the-art community recovery rates and the lowest misclassification rate. Notably, it is the first to reveal strong statistical correlations between node-pair similarity and mainstream evaluation metrics.

Technology Category

• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Machine Learning: Ensemble Methods
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Social Networks and Social Media: Computational social science

Abstract

Community detection plays a crucial role in understanding the structural organization of complex networks. Previous methods, particularly those from statistical physics, primarily focus on the analysis of mesoscopic network structures and often struggle to integrate fine-grained node similarities. To address this limitation, we propose a low-complexity framework that integrates machine learning to embed micro-level node-pair similarities into mesoscopic community structures. By leveraging ensemble learning models, our approach enhances both structural coherence and detection accuracy. Experimental evaluations on artificial and real-world networks demonstrate that our framework consistently outperforms conventional methods, achieving higher modularity and improved accuracy in NMI and ARI. Notably, when ground-truth labels are available, our approach yields the most accurate detection results, effectively recovering real-world community structures while minimizing misclassifications. To further explain our framework's performance, we analyze the correlation between node-pair similarity and evaluation metrics. The results reveal a strong and statistically significant correlation, underscoring the critical role of node-pair similarity in enhancing detection accuracy. Overall, our findings highlight the synergy between machine learning and statistical physics, demonstrating how machine learning techniques can enhance network analysis and uncover complex structural patterns.

Problem

• Integrating micro-level node similarities into mesoscopic community detection
• Improving community detection accuracy using machine learning and statistical physics
• Analyzing correlation between node-pair similarity and detection performance metrics

Innovation

• Integrates machine learning with statistical physics
• Embeds micro-level similarities into mesoscopic structures
• Uses ensemble learning to enhance detection accuracy