Estimating Size of the Union of Sets in Streaming Model

2026-07-29

AI Summary

This work addresses the problem of efficiently estimating the size of the union of multiple implicitly defined sets in the streaming model, encompassing classical settings such as discrete Klee measure, combinatorial test coverage estimation, and DNF model counting. The authors propose a sampling-based streaming algorithm that achieves, for the first time, an update time linear in the dimension $d$, resolving a long-standing open question in the field. Their approach reveals a fundamental trade-off between near-optimal space complexity and $P^{NP}$-level update time in coverage estimation. The algorithm uses $O(R \log|\Omega|)$ space and $O(R \log R \cdot \log(M/\delta) \cdot \log|\Omega|)$ update time, where $R = O(\varepsilon^{-2} \log(M/\delta))$, and has been formally verified in Lean 4.

Abstract

We study estimating the size of the union of sets $S_1,\dots,S_M$, where each $S_i\subseteqΩ$ is presented implicitly and arrives in a stream. We introduce Delphic sets, a class of streaming problems in which membership, sampling, and counting queries to each set are efficient, and show that this notion captures three well-known problems: Klee's measure problem (discrete version), test coverage estimation in combinatorial testing, and model counting of DNF formulas. Our primary contribution is a simple and efficient sampling-based algorithm that outputs an $(\varepsilon,δ)$-approximation of the cardinality of the union of Delphic sets in the streaming setting. It has space complexity $O(R\log|Ω|)$ and update time $O(R\log R\cdot\log(M/δ)\cdot\log|Ω|)$, where $R=O(\log(M/δ)\cdot\varepsilon^{-2})$. For the streaming Klee's measure problem, this gives the first algorithm whose update time depends linearly on the dimension $d$ for $d>1$, settling an open problem of Tirthapura and Woodruff (PODS 2012), and it directly yields efficient streaming algorithms for coverage estimation and DNF model counting. We further show that the space for coverage estimation can be made near-optimal at the cost of an update procedure in $\mathrm{P}^{\mathrm{NP}}$, revealing a time-space trade-off. A key strength of our approach is the simplicity of both the algorithm and its analysis, which makes it amenable to practical implementation. In this revised version, the algorithm and its correctness analysis have additionally been formalized and machine-checked in Lean 4. (Shortened for Arxiv)

Problem

• streaming model
• union size estimation
• Delphic sets
• Klee's measure problem
• model counting

Innovation

• streaming algorithm
• Delphic sets
• union size estimation
• Klee's measure problem
• time-space trade-off