🤖 AI Summary
Existing graph generation models (GGMs) are commonly evaluated using Maximum Mean Discrepancy (MMD), which lacks sensitivity to domain-specific structural fidelity—particularly the preservation of semantically meaningful topological patterns across diverse graph domains.
Method: We propose Representation-aware Graph Generation Model evaluation (RGM), the first GGM evaluation framework incorporating geometric deep learning. RGM employs trained graph classifiers to assess intra-domain structural consistency between generated and real graphs, enabling fine-grained, semantically interpretable structural fidelity assessment.
Contribution/Results: Evaluated on synthetic–real hybrid benchmarks, RGM uncovers systematic structural preservation failures in state-of-the-art models (e.g., GRAN, EDGE) under cross-domain generalization. Empirical results demonstrate that RGM significantly outperforms MMD in sensitivity, robustness, and discriminative power, establishing a new, principled benchmark for rigorous GGM evaluation.
📝 Abstract
Graph generation is a crucial task in many fields, including network science and bioinformatics, as it enables the creation of synthetic graphs that mimic the properties of real-world networks for various applications. Graph Generative Models (GGMs) have emerged as a promising solution to this problem, leveraging deep learning techniques to learn the underlying distribution of real-world graphs and generate new samples that closely resemble them. Examples include approaches based on Variational Auto-Encoders, Recurrent Neural Networks, and more recently, diffusion-based models. However, the main limitation often lies in the evaluation process, which typically relies on Maximum Mean Discrepancy (MMD) as a metric to assess the distribution of graph properties in the generated ensemble. This paper introduces a novel methodology for evaluating GGMs that overcomes the limitations of MMD, which we call RGM (Representation-aware Graph-generation Model evaluation). As a practical demonstration of our methodology, we present a comprehensive evaluation of two state-of-the-art Graph Generative Models: Graph Recurrent Attention Networks (GRAN) and Efficient and Degree-guided graph GEnerative model (EDGE). We investigate their performance in generating realistic graphs and compare them using a Geometric Deep Learning model trained on a custom dataset of synthetic and real-world graphs, specifically designed for graph classification tasks. Our findings reveal that while both models can generate graphs with certain topological properties, they exhibit significant limitations in preserving the structural characteristics that distinguish different graph domains. We also highlight the inadequacy of Maximum Mean Discrepancy as an evaluation metric for GGMs and suggest alternative approaches for future research.