Non-iid hypothesis testing: from classical to quantum

2025-10-07

AI Summary

This paper studies tolerant hypothesis testing for classical and quantum states under non-i.i.d. conditions: given a single sample from each of $T$ unknown distributions (or quantum states), we test whether their average is $varepsilon$-close to a known reference distribution $q$ (or state $sigma$). Methodologically, we introduce a quantum Efron–Stein inequality and state decomposition techniques to uniformly model non-i.i.d. statistical structures. Our key contributions are: (1) In the quantum setting, optimal discrimination is achieved using only one copy per $ ho_t$, breaking the classical lower bound requiring at least two samples per state—revealing a counterintuitive advantage of non-i.i.d. quantum testing; (2) We generalize identity testing to the case of unknown average states; (3) When $T gg d/varepsilon^2$, our algorithm attains an $O(d/varepsilon^2)$ sample complexity—matching the i.i.d. optimal bound—and yields tight sample-complexity improvements even in the classical regime.

Technology Category

• Machine Learning: Quantum Machine Learning
• Search and Optimization: Non-convex Optimization
• Intelligent Robots: State Estimation

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

We study hypothesis testing (aka state certification) in the non-identically distributed setting. A recent work (Garg et al. 2023) considered the classical case, in which one is given (independent) samples from $T$ unknown probability distributions $p_1, dots, p_T$ on $[d] = {1, 2, dots, d}$, and one wishes to accept/reject the hypothesis that their average $p_{mathrm{avg}}$ equals a known hypothesis distribution $q$. Garg et al. showed that if one has just $c = 2$ samples from each $p_i$, and provided $T gg frac{sqrt{d}}{epsilon^2} + frac{1}{epsilon^4}$, one can (whp) distinguish $p_{mathrm{avg}} = q$ from $d_{mathrm{TV}}(p_{mathrm{avg}},q)>epsilon$. This nearly matches the optimal result for the classical iid setting (namely, $T gg frac{sqrt{d}}{epsilon^2}$). Besides optimally improving this result (and generalizing to tolerant testing with more stringent distance measures), we study the analogous problem of hypothesis testing for non-identical quantum states. Here we uncover an unexpected phenomenon: for any $d$-dimensional hypothesis state $sigma$, and given just a single copy ($c = 1$) of each state $ ho_1, dots, ho_T$, one can distinguish $ ho_{mathrm{avg}} = sigma$ from $D_{mathrm{tr}}( ho_{mathrm{avg}},sigma)>epsilon$ provided $T gg d/epsilon^2$. (Again, we generalize to tolerant testing with more stringent distance measures.) This matches the optimal result for the iid case, which is surprising because doing this with $c = 1$ is provably impossible in the classical case. We also show that the analogous phenomenon happens for the non-iid extension of identity testing between unknown states. A technical tool we introduce may be of independent interest: an Efron-Stein inequality, and more generally an Efron-Stein decomposition, in the quantum setting.

Problem

• Extends classical non-iid hypothesis testing to quantum state certification
• Analyzes single-copy quantum testing achieving iid-optimal performance
• Establishes quantum advantage in distinguishing average states from hypothesis

Innovation

• Extends classical non-iid hypothesis testing to quantum states
• Uses single quantum copy per state for optimal testing
• Introduces quantum Efron-Stein inequality for decomposition analysis