Learning Distributions from Multiple Data Providers

2026-07-27

AI Summary

This work addresses the problem of efficiently learning an unknown target distribution when access is limited to multiple heterogeneous data sources with overlapping samples and constrained conditional sampling. The core approach leverages the structure of a co-occurrence graph, establishing that its connectivity and completeness govern PAC learnability. The authors introduce a “hierarchical comparability” condition enabling near-linear sample complexity. Key contributions include proving an optimal sample complexity of Õ(n²/ε²) under a fully connected co-occurrence graph, which improves to Θ(n/ε²) when the entire universe is queryable. Furthermore, they construct a family of query mechanisms achieving Õ(n^α/ε²) sample complexity for any α ∈ (1, 2), thereby realizing any polynomial rate between linear and quadratic.

Abstract

Motivated by learning from heterogeneous and overlapping data providers, we study a stylized model of distribution learning from restricted conditional samples. The goal is to learn an unknown distribution $p$ on a finite domain $[n]$. The learner is given a fixed family of queryable sets $\mathscr{S} \subseteq 2^{[n]}$, and each query to $S \in \mathscr{S}$ returns an independent sample from the conditional distribution $p(\cdot \mid S)$. Learnability is governed by the co-occurrence graph associated with $\mathscr{S}$: two domain elements are adjacent if they appear together in some queryable set. Pointwise consistency is achievable when this graph is connected on the target support. PAC learning requires more: it is possible when the co-occurrence graph is complete. The optimal sample complexity of PAC learning ranges from nearly linear to quadratic. Every query family with complete co-occurrence graph admits sample complexity $\widetilde O(n^2/ε^2)$, and this bound is tight in the worst case. On the other hand, if $[n]$ is queryable then ordinary sampling improves the bound to $Θ(n/ε^2)$, and this cannot be improved further even if every set is queryable. More generally, we identify hierarchical comparabilityas a sufficient structural condition on $\mathscr S$ under which the optimal complexity is nearly linear, $\widetilde Θ(n/ε^2)$, with pairwise query families as a canonical example. Finally, the full range of polynomial rates between linear and quadratic is attainable: for every $α\in (1,2)$, there exists a query family with optimal PAC rate $\widetilde Θ(n^α/ε^2)$.

Problem

• distribution learning
• conditional sampling
• PAC learning
• co-occurrence graph
• sample complexity

Innovation

• conditional sampling
• distribution learning
• co-occurrence graph
• PAC learning
• sample complexity