Distribution Testing Meets Sum Estimation

2025-04-21

AI Summary

This paper studies the problem of estimating the total weight (W = sum_i w(i)) under non-increasing (monotone) or unimodal weight distributions via non-adaptive conditional sampling, aiming for a ((1pmvarepsilon))-approximation using sublinear sample complexity. It pioneers the application of distribution testing techniques to summation estimation, proposing the first non-adaptive conditional sampling algorithm for monotone distributions and extending it to unimodal distributions and support size (k) estimation. Key technical components include weighted/uniform conditional sampling, hybrid sampling strategies, piecewise estimation, and binary search-based localization. For monotone distributions, the weighted-sample complexity is (O(1/varepsilon^3 log n + 1/varepsilon^6)), while uniform-sample complexity is (O(1/varepsilon^3 log n)), matching the (Omega(log n)) lower bound up to constant factors. Support size estimation achieves additive error (pm 2varepsilon n).

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We study the problem of estimating the sum of $n$ elements, each with weight $w(i)$, in a structured universe. Our goal is to estimate $W = sum_{i=1}^n w(i)$ within a $(1 pm epsilon)$ factor using a sublinear number of samples, assuming weights are non-increasing, i.e., $w(1) geq w(2) geq dots geq w(n)$. The sum estimation problem is well-studied under different access models to the universe $U$. However, to the best of our knowledge, nothing is known about the sum estimation problem using non-adaptive conditional sampling. In this work, we explore the sum estimation problem using non-adaptive conditional weighted and non-adaptive conditional uniform samples, assuming that the underlying distribution ($D(i)=w(i)/W$) is monotone. We also extend our approach to to the case where the underlying distribution of $U$ is unimodal. Additionally, we consider support size estimation when $w(i) = 0$ or $w(i) geq W/n$, using hybrid sampling (both weighted and uniform) to access $U$. We propose an algorithm to estimate $W$ under the non-increasing weight assumption, using $O(frac{1}{epsilon^3} log{n} + frac{1}{epsilon^6})$ non-adaptive weighted conditional samples and $O(frac{1}{epsilon^3} log{n})$ uniform conditional samples. Our algorithm matches the $Omega(log{n})$ lower bound by cite{ACK15}. For unimodal distributions, the sample complexity remains similar, with an additional $O(log{n})$ evaluation queries to locate the minimum weighted point in the domain. For estimating the support size $k$ of $U$, where weights are either $0$ or at least $W/n$, our algorithm uses $Oig( frac{log^3(n/epsilon)}{epsilon^8} cdot log^4 frac{log(n/epsilon)}{epsilon} ig)$ uniform samples and $Oig( frac{log(n/epsilon)}{epsilon^2} cdot log frac{log(n/epsilon)}{epsilon} ig)$ weighted samples to output $hat{k}$ satisfying $k - 2epsilon n leq hat{k} leq k + epsilon n$.

Problem

• Estimating sum of weighted elements with sublinear samples
• Exploring sum estimation using non-adaptive conditional sampling
• Support size estimation with hybrid sampling under constraints

Innovation

• Non-adaptive conditional weighted sampling for sum estimation
• Hybrid sampling for support size estimation
• Efficient algorithm with logarithmic sample complexity