Compositional Generative Modeling from Decentralized Data

2026-06-08

AI Summary

This work addresses the limitation of existing decentralized generative methods, which model only the union of data silos and struggle to capture novel combinations of generative factors across silos. To overcome this, we propose Decentralized Compositional Flow Matching (DCFM), a framework that coordinates generative models across silos through structural constraints without exchanging raw data, thereby enabling compositional modeling of global generative factors. DCFM is the first approach to support cross-silo compositional generation in decentralized settings, breaking the expressivity bottleneck imposed by single-source data on complex combinations. Experiments demonstrate that DCFM significantly outperforms federated learning and mixture-of-experts baselines in conditional image generation, robotic spatial planning, and modeling co-occurring medical attributes.

Abstract

Learning the compositional nature of the physical world requires joint observation of interacting factors. However, because practical data is often decentralized, these factors are fragmented across isolated silos. Existing decentralized generative approaches focus only on modeling the union of siloed data, overlooking novel combinations implied by the collective whole. To bridge this gap, we introduce Decentralized Compositional Flow Matching (DCFM), a framework that enforces structural constraints across the global set of generative factors, without exchanging any raw data. DCFM enables novel combinations to emerge through peer interactions, even when no single data source can independently support the composition. Empirically, DCFM substantially outperforms federated learning and mixture-of-experts baselines across conditional image generation, robotic spatial planning, and medical attribute co-occurrence modeling.

Problem

• compositional generative modeling
• decentralized data
• factor composition
• data silos
• novel combinations

Innovation

• Decentralized Learning
• Compositional Generative Modeling
• Flow Matching
• Data Silos
• Federated Generation