exchangeable concentration bounds

This work addresses the quantification of uncertainty in accuracy estimation for AI benchmarks such as MMLU that exhibit exchangeable dependency structures. Leveraging de Finetti’s representation theorem, it decomposes the bias of infinitely exchangeable sequences into conditional sampling variability and latent mixture variability. The study reveals, for the first time, a precise cancellation mechanism of the mixture component under zero-sum and linear contrast conditions, yielding a tight Hoeffding-type concentration inequality free of mixture terms. This approach requires no distributional assumptions, applies naturally to both infinite exchangeable sequences and their finite counterparts, and provides domain-stratified upper bounds on uncertainty for composite AI benchmarks. Furthermore, it offers statistical guarantees for low-cost subset evaluations, enabling reliable performance assessment with reduced computational overhead.

This paper investigates statistical distance bounds between high-dimensional exchangeable mixture distributions (i.e., permutation mixtures) and their i.i.d. approximations. To overcome the challenge of controlling the χ²-divergence, we develop a novel analytical framework: (i) establishing Maclaurin-type inequalities for elementary symmetric polynomials of zero-mean variables; (ii) deriving tight upper bounds on the permanent of doubly stochastic positive semidefinite matrices; and (iii) integrating moment–cumulant methods, exchangeability structure, and asymptotic statistical theory. Key contributions include: (i) a strengthened de Finetti-type theorem with significantly improved convergence rates over classical versions; and (ii) a generalization of the Hannan–Robbins (1955) result on asymptotic optimality of compound decision rules to a broader class of exchangeable models, along with sharpened sufficient conditions and enhanced accuracy guarantees.

High-probability analysis of learning algorithms involving light-tailed (e.g., sub-exponential, sub-Gaussian) but possibly unbounded random variables poses significant technical challenges due to the lack of uniform concentration tools across distribution families. Method: We propose a generic black-box reduction that systematically transforms high-probability analysis of any algorithm relying on light-tailed randomness into the corresponding analysis under bounded-variable assumptions, incurring only controllable logarithmic-factor overheads. Contribution/Results: This is the first unified framework handling diverse light-tailed distributions without ad hoc concentration inequalities—greatly simplifying theoretical analysis. As applications, we reconstruct a generalized Azuma’s inequality and derive tight high-probability convergence bounds for stochastic optimization algorithms under light-tailed noise, demonstrating both the method’s effectiveness and broad applicability.

This paper addresses non-asymptotic statistical inference for high-dimensional linear and Poisson regression by systematically extending and refining concentration inequality theory. Methodologically, it unifies treatment of diverse light-tailed structures—from distribution-free settings to sub-Gaussian and sub-Weibull tails—via moment-generating function analysis and exponential-type tail control, yielding novel concentration bounds with explicit, tight constants. The contributions are threefold: (i) it introduces the first systematic concentration inequality framework tailored to inference in high-dimensional generalized linear models; (ii) it substantially improves bound tightness and verifiability under realistic model assumptions; and (iii) it delivers computationally tractable, theoretically rigorous statistical guarantees for finite-sample parameter estimation and hypothesis testing. These advances enhance both the accuracy and applicability of high-dimensional inference, particularly in settings where asymptotic approximations are unreliable.

This work establishes Rosenthal- and Bernstein-type concentration inequalities for additive functionals of geometrically ergodic Markov chains, explicitly characterizing the dependence of deviation bounds on mixing time. Methodologically, it pioneers the extension of the classical Rosenthal inequality to the Markov-dependent setting via a novel analytical framework based on Poisson equation decomposition, which precisely links mixing constants, martingale Rosenthal constants, and deviation bounds. Integrating martingale techniques, geometric ergodicity analysis, and quantitative mixing time estimation, the approach yields computable, explicit, and tight upper bounds—significantly improving the polynomial dependence on mixing time present in prior results. The derived inequalities provide a rigorous theoretical foundation for error control in MCMC algorithms, sequential Monte Carlo estimation, and large-sample inference for non-i.i.d. statistics.

This work addresses the challenge of deriving concentration inequalities for structured tensor and matrix data that are non-independent yet exchangeable—a setting poorly handled by existing methods. Under an exchangeability assumption, the paper establishes Hoeffding- and Bernstein-type tail bounds by introducing a novel framework for exchangeable dependence, which overcomes limitations of Chatterjee’s exchangeable pair approach. The resulting bounds are sharper for combinatorial matrix sums and unify classical results for both independent and exchangeable cases. The analysis integrates tools from exchangeable random variable theory, matrix concentration inequalities, and combinatorial techniques. The theoretical guarantees are validated through applications to average effect estimation in multi-factor response models and fixed-design sketching algorithms in federated learning, with numerical experiments showing excellent agreement with theoretical predictions.

This study addresses the asymptotic enumeration of dense binary matrices under dual constraints—fixed row sums and pairwise column sums—in the context of large-scale genetic data. By integrating saddle-point approximation, probabilistic methods, entropy inequalities, and combinatorial enumeration, we derive a refined asymptotic expansion for the logarithm of the count. Our key finding reveals that when the sample size \(N = \Theta(P)\) and marginal sums are uniform, the constraint ensemble satisfies an independence heuristic but with a correction factor of \(1/\sqrt[4]{e}\), deviating from the classical \(e^{\pm 1/2}\). We further provide the first explicit fourth-moment contribution and quantitative control of higher-order remainder terms. The work yields an exact counting formula for single-constraint ensembles, an asymptotic theory for dual constraints, and an open-source extended Miller–Harrison algorithm for numerical validation.

This work investigates improved probabilistic concentration inequalities under weak randomness models, including limited independence, random hashing, and Markov chains. To this end, it introduces a unified analytical framework based on elementary symmetric polynomials augmented with auxiliary randomization, recasting concentration problems as the control of product moments over uniformly sampled index sets—thereby replacing conventional exponential moment methods. This approach substantially weakens dependence on worst-case degrees and is successfully applied to three settings: read-Δ families, random binary linear hashing, and Markov chains. The framework not only recovers known spectral bounds and mixing time results but also yields tighter concentration inequalities across these scenarios.

This study addresses the construction of optimal lower confidence bounds for the maximum mean parameter of a vector of independent nonnegative random variables. By embedding Buehler’s classical approach within a purely probabilistic framework, the work establishes—for the first time—that the Gaffke bound possesses Buehler optimality under the sample ordering it induces in the case of independent components. This result confirms the inadmissibility of any improvement over the Gaffke bound within this setting, thereby extending classical confidence bound theory and providing a rigorous theoretical foundation for nonparametric lower-bound estimation.

研究通过最小化差异设计控制非参数函数类的不平衡，解决高维协变量下实验设计无法达到半参数效率边界的问题。