Graph is a Natural Regularization: Revisiting Vector Quantization for Graph Representation Learning

2025-08-08

AI Summary

Codebook collapse is a pervasive issue in graph vector quantization (VQ), severely limiting token diversity and representational capacity; yet it remains systematically uncharacterized and unresolved in the graph domain. Method: This paper formally defines and analyzes the codebook collapse mechanism specific to graphs, and proposes RGVQ—a structure-aware graph VQ framework. RGVQ introduces topology-guided structural contrastive regularization to constrain codebook learning and integrates Gumbel-Softmax-based soft assignment to enhance codebook utilization and token discriminability. Contribution/Results: By deeply unifying graph neural networks with discrete representation learning, RGVQ significantly outperforms existing graph VQ methods across multiple benchmark datasets. It improves generalization and transferability on downstream tasks—including node classification and graph classification—establishing a novel paradigm for discrete graph representation learning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Computer Vision: Representation Learning for Vision
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Vector Quantization (VQ) has recently emerged as a promising approach for learning discrete representations of graph-structured data. However, a fundamental challenge, i.e., codebook collapse, remains underexplored in the graph domain, significantly limiting the expressiveness and generalization of graph tokens.In this paper, we present the first empirical study showing that codebook collapse consistently occurs when applying VQ to graph data, even with mitigation strategies proposed in vision or language domains. To understand why graph VQ is particularly vulnerable to collapse, we provide a theoretical analysis and identify two key factors: early assignment imbalances caused by redundancy in graph features and structural patterns, and self-reinforcing optimization loops in deterministic VQ. To address these issues, we propose RGVQ, a novel framework that integrates graph topology and feature similarity as explicit regularization signals to enhance codebook utilization and promote token diversity. RGVQ introduces soft assignments via Gumbel-Softmax reparameterization, ensuring that all codewords receive gradient updates. In addition, RGVQ incorporates a structure-aware contrastive regularization to penalize the token co-assignments among similar node pairs. Extensive experiments demonstrate that RGVQ substantially improves codebook utilization and consistently boosts the performance of state-of-the-art graph VQ backbones across multiple downstream tasks, enabling more expressive and transferable graph token representations.

Problem

• Addresses codebook collapse in graph vector quantization
• Identifies redundancy and optimization loops causing collapse
• Proposes RGVQ to enhance codebook utilization and diversity

Innovation

• Soft assignments via Gumbel-Softmax reparameterization
• Structure-aware contrastive regularization for token diversity
• Graph topology and feature similarity as explicit regularization