Variational Flow Matching for Graph Generation

2024-06-07

Neural Information Processing Systems

AI Summary

Efficient generation of graph-structured data—particularly abstract graphs and molecules—remains challenging due to the discrete, non-Euclidean nature of graphs and limitations of existing generative paradigms. Method: This paper introduces Variational Flow Matching (VFM), the first framework unifying flow matching with variational inference. VFM models deterministic trajectories over discrete structures and class-conditional dynamics for categorical data, yielding CatFlow—a novel objective that generalizes both standard flow matching and score-based dynamics as a weighted evidence lower bound. Unlike prior methods, VFM enables end-to-end training without sampling or complex scheduling. Results: On abstract graph generation and two molecular generation benchmarks, VFM matches or surpasses state-of-the-art methods in performance while achieving significantly lower computational overhead and implementation simplicity.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

We present a formulation of flow matching as variational inference, which we refer to as variational flow matching (VFM). Based on this formulation we develop CatFlow, a flow matching method for categorical data. CatFlow is easy to implement, computationally efficient, and achieves strong results on graph generation tasks. In VFM, the objective is to approximate the posterior probability path, which is a distribution over possible end points of a trajectory. We show that VFM admits both the CatFlow objective and the original flow matching objective as special cases. We also relate VFM to score-based models, in which the dynamics are stochastic rather than deterministic, and derive a bound on the model likelihood based on a reweighted VFM objective. We evaluate CatFlow on one abstract graph generation task and two molecular generation tasks. In all cases, CatFlow exceeds or matches performance of the current state-of-the-art models.

Problem

• Generating graphs using variational flow matching
• Modeling categorical data with efficient flow methods
• Improving molecular and abstract graph generation performance

Innovation

• Variational flow matching for inference
• CatFlow method for categorical data
• Approximates posterior probability paths