derive concentration inequalities

This work addresses the challenge of deriving concentration inequalities for structured tensor and matrix data that are non-independent yet exchangeable—a setting poorly handled by existing methods. Under an exchangeability assumption, the paper establishes Hoeffding- and Bernstein-type tail bounds by introducing a novel framework for exchangeable dependence, which overcomes limitations of Chatterjee’s exchangeable pair approach. The resulting bounds are sharper for combinatorial matrix sums and unify classical results for both independent and exchangeable cases. The analysis integrates tools from exchangeable random variable theory, matrix concentration inequalities, and combinatorial techniques. The theoretical guarantees are validated through applications to average effect estimation in multi-factor response models and fixed-design sketching algorithms in federated learning, with numerical experiments showing excellent agreement with theoretical predictions.

This work addresses the problem of establishing lower bounds on the smallest singular value of random matrices whose entries are low-degree polynomials of a small number of base random variables—bypassing fundamental limitations of classical smoothed analysis under strong anti-concentration assumptions. The method introduces a novel anti-concentration framework grounded in the well-conditionedness of polynomial mappings’ Jacobians, integrating hierarchical ε-nets, higher-order matrix lifting, and spectral analysis of linear operators. This yields the first singular-value criterion applicable to matrices with strong algebraic dependencies. The contribution resolves long-standing open problems—including power-sum decomposition and robust subspace entanglement certification—by providing the first smoothed-analysis guarantees for such settings. Crucially, it extends the scope of smoothed analysis from fully independent random matrices to non-robust, algebraically dependent ensembles, thereby furnishing new theoretical tools and foundations for algorithmic smoothed analysis.

High-probability analysis of learning algorithms involving light-tailed (e.g., sub-exponential, sub-Gaussian) but possibly unbounded random variables poses significant technical challenges due to the lack of uniform concentration tools across distribution families. Method: We propose a generic black-box reduction that systematically transforms high-probability analysis of any algorithm relying on light-tailed randomness into the corresponding analysis under bounded-variable assumptions, incurring only controllable logarithmic-factor overheads. Contribution/Results: This is the first unified framework handling diverse light-tailed distributions without ad hoc concentration inequalities—greatly simplifying theoretical analysis. As applications, we reconstruct a generalized Azuma’s inequality and derive tight high-probability convergence bounds for stochastic optimization algorithms under light-tailed noise, demonstrating both the method’s effectiveness and broad applicability.

This paper addresses non-asymptotic statistical inference for high-dimensional linear and Poisson regression by systematically extending and refining concentration inequality theory. Methodologically, it unifies treatment of diverse light-tailed structures—from distribution-free settings to sub-Gaussian and sub-Weibull tails—via moment-generating function analysis and exponential-type tail control, yielding novel concentration bounds with explicit, tight constants. The contributions are threefold: (i) it introduces the first systematic concentration inequality framework tailored to inference in high-dimensional generalized linear models; (ii) it substantially improves bound tightness and verifiability under realistic model assumptions; and (iii) it delivers computationally tractable, theoretically rigorous statistical guarantees for finite-sample parameter estimation and hypothesis testing. These advances enhance both the accuracy and applicability of high-dimensional inference, particularly in settings where asymptotic approximations are unreliable.

This work addresses the asymptotic convergence and non-asymptotic maximal inequalities for semidefinite matrix-valued martingales and reverse submartingales. Motivated by the lack of systematic convergence analysis and adaptive stopping-time control under the Loewner order in existing theory, we establish, for the first time, a Loewner-order-based martingale convergence theorem and adaptive maximal inequalities for stopping times—unifying light-tailed, heavy-tailed, and self-normalized settings. Our approach integrates matrix probability theory, random matrix theory, martingale analysis, and dependence modeling to derive novel matrix concentration inequalities. Crucially, our results overcome the limitations of classical Chernoff-type bounds—which require fixed sample sizes and independence—by accommodating arbitrary (possibly data-dependent) sample sizes and stopping times. This significantly broadens theoretical applicability and practical utility in high-dimensional statistical inference and online machine learning.

This work addresses the absence of finite-sample error bounds and concentration inequalities for nonlinear stochastic approximation algorithms under the Wasserstein-p distance. By coupling the discrete-time iterative process with its Ornstein–Uhlenbeck diffusion limit, the paper establishes the first non-asymptotic distributional convergence rates in Wasserstein distance under general noise conditions—such as martingale differences and ergodic Markov chains. The main contributions include proving that the last iterate converges to a Gaussian distribution at a rate of γₙ^{1/6}, while the Polyak–Ruppert averaged iterate achieves a rate of n^{-1/6}. Moreover, the analysis yields high-probability concentration inequalities that improve upon those derived via classical moment-based methods. The proposed framework applies broadly to canonical algorithms, including linear stochastic approximation and stochastic gradient descent.

This paper studies learning unknown operators between separable Hilbert spaces from noisy samples, focusing on minimax risk bounds for uniformly bounded Lipschitz operators. Methodologically, it integrates minimax analysis, information-theoretic lower bound construction, functional estimation in random processes, and operator spectral theory to derive the first tight (matching or nearly matching) information-theoretic upper and lower bounds for this setting. Key contributions include: (1) revealing a “sample complexity curse” that precludes algebraic convergence rates; (2) achieving essentially optimal characterization under exponential decay of the covariance operator’s eigenvalues; and (3) establishing that the risk rate is governed solely by the spectrum of the covariance operator induced by the error metric—valid for both fixed and random designs, and extending to distributions with unbounded support. The results provide a unified theoretical benchmark for operator learning.

This study investigates high-confidence non-asymptotic upper and lower bounds for the minimal risk in statistical learning, circumventing the conventional reliance on boundedness assumptions of the empirical risk function. By integrating sharp forms of Talagrand’s concentration inequality—specifically the Bousquet and Klein–Rio refinements—with transport-entropy inequalities and empirical process theory, the authors derive a lower bound independent of both the number of parameters and input dimension, under Gaussian or exponential integrability assumptions. The corresponding upper bound is characterized by the interplay between sample size and the box-counting dimension of the parameter set measured in an Orlicz norm. This work thus provides a more general and non-asymptotic theoretical framework for evaluating learning algorithm performance without resorting to asymptotic approximations.

This work addresses the challenge of transforming sharp matrix concentration inequalities derived from free probability theory into deterministic algorithms, thereby replacing traditional constructions that rely on randomness. By systematically introducing core concepts and techniques from free probability into the design of efficient deterministic algorithms for the first time, we develop polynomial-time algorithms that successfully achieve deterministic constructions for both the matrix Spencer problem and near-Ramanujan graphs. This breakthrough overcomes the limitations inherent in prior randomized approaches and establishes a new paradigm for the deterministic realization of high-dimensional probabilistic and combinatorial structures.

This work addresses stochastic approximation problems involving multiplicative noise and unbounded iterates under compression, proposing a unified and elementary analytical framework. By directly constructing a first-order Lyapunov drift inequality for the error norm, the approach avoids sophisticated tools such as smoothing or Moreau envelopes. Combining averaged noise sequences with auxiliary iterates, the method employs inductive expectation arguments to derive mean-square error bounds and, for the first time under multiplicative noise, establishes a full-trajectory maximal concentration bound with sub-Gaussian tails via probabilistic induction and the Azuma–Hoeffding inequality. The stepsize schedule depends only logarithmically on the confidence parameter, eliminating the need for multi-stage warm-up procedures or smooth Lyapunov functions, thereby significantly simplifying the analysis. The framework is successfully applied to ℓ∞-contractive operators in reinforcement learning and exhibits strong generalizability.