apply measure concentration

This paper addresses non-asymptotic statistical inference for high-dimensional linear and Poisson regression by systematically extending and refining concentration inequality theory. Methodologically, it unifies treatment of diverse light-tailed structures—from distribution-free settings to sub-Gaussian and sub-Weibull tails—via moment-generating function analysis and exponential-type tail control, yielding novel concentration bounds with explicit, tight constants. The contributions are threefold: (i) it introduces the first systematic concentration inequality framework tailored to inference in high-dimensional generalized linear models; (ii) it substantially improves bound tightness and verifiability under realistic model assumptions; and (iii) it delivers computationally tractable, theoretically rigorous statistical guarantees for finite-sample parameter estimation and hypothesis testing. These advances enhance both the accuracy and applicability of high-dimensional inference, particularly in settings where asymptotic approximations are unreliable.

This work establishes the first global concentration bound—valid for all $ n geq n_0 $—on the finite-sample behavior of online single-trajectory TD(0) with linear function approximation under Markovian noise. Unlike offline settings relying on i.i.d. samples or ergodic state visits, it directly addresses non-independent, non-stationary, temporally dependent data. Methodologically, it integrates stochastic approximation theory with Poisson equation analysis to characterize Markovian bias and employs a relaxed concentration inequality to circumvent the lack of a priori boundedness in iterates. The result delivers an explicit exponential decay probability bound, rigorously quantifying the convergence rate of estimation error with respect to iteration count. This provides the first finite-sample theoretical guarantee for online reinforcement learning with fully explicit constant dependencies.

This paper addresses statistical error control in stochastic optimization under unbounded objective functions—particularly relevant to denoising score matching (DSM), where the objective may be unbounded even when the data distribution has bounded support. Methodologically, we develop the first generalization theory framework for unbounded functions by introducing a sample-dependent McDiarmid-type inequality and deriving a tight upper bound on the Rademacher complexity for locally Lipschitz function classes, thereby unifying law-of-large-numbers guarantees with sharp statistical error bounds. Theoretical contributions include: (1) the first rigorous $O(1/sqrt{n})$ statistical error bound for DSM; and (2) a quantitative characterization of how Gaussian auxiliary variable resampling improves estimation accuracy. Our results break the conventional boundedness assumption, significantly broadening the theoretical foundations of score-based generative modeling and related stochastic optimization methods.

Traditional statistical model checking (SMC) is limited to mean-type statistics—such as expected reward and probability—and thus struggles to reliably estimate distribution-sensitive risk measures, including quantiles, conditional value-at-risk (CVaR), and entropy-based risk. This work introduces, for the first time, the Dvoretzky–Kiefer–Wolfowitz–Massart (DKW) inequality into SMC, enabling non-asymptotic confidence bands around the empirical cumulative distribution function (ECDF) to perform rigorous, distribution-wide inference over system trajectories. The resulting framework supports statistical verification of key risk metrics—quantiles, CVaR, and entropy risk—with formal guarantees. It is efficiently implemented in the MODEST Toolset’s modes simulator. Experimental evaluation across multiple quantitative verification benchmarks demonstrates high accuracy and strong robustness, significantly extending the risk modeling capabilities and expressive power of SMC for reliability assessment.

High-probability analysis of learning algorithms involving light-tailed (e.g., sub-exponential, sub-Gaussian) but possibly unbounded random variables poses significant technical challenges due to the lack of uniform concentration tools across distribution families. Method: We propose a generic black-box reduction that systematically transforms high-probability analysis of any algorithm relying on light-tailed randomness into the corresponding analysis under bounded-variable assumptions, incurring only controllable logarithmic-factor overheads. Contribution/Results: This is the first unified framework handling diverse light-tailed distributions without ad hoc concentration inequalities—greatly simplifying theoretical analysis. As applications, we reconstruct a generalized Azuma’s inequality and derive tight high-probability convergence bounds for stochastic optimization algorithms under light-tailed noise, demonstrating both the method’s effectiveness and broad applicability.

This work investigates the concentration of iteration errors in stochastic approximation algorithms driven by heavy-tailed Markov noise, covering both expansive and non-expansive operator settings. Under a framework involving a finite-state Markov component and martingale difference noise, the authors construct a novel Lyapunov function via the moment-generating function of the solution to the Poisson equation, complemented by auxiliary projection and black-box truncation techniques to reduce unbounded noise to a bounded setting. The study provides the first systematic characterization of the fine structure of error tails: under bounded noise, tails can be sub-Gaussian, sub-Weibull, or intermediate between Pareto and Weibull; under unbounded noise, if the operator is almost surely non-expansive, the error tail is at most three times heavier than that of the noise, whereas if the operator is expansive with positive probability, significantly heavier tails may arise, with sharp worst-case examples demonstrating the tightness of these bounds.

This study investigates high-confidence non-asymptotic upper and lower bounds for the minimal risk in statistical learning, circumventing the conventional reliance on boundedness assumptions of the empirical risk function. By integrating sharp forms of Talagrand’s concentration inequality—specifically the Bousquet and Klein–Rio refinements—with transport-entropy inequalities and empirical process theory, the authors derive a lower bound independent of both the number of parameters and input dimension, under Gaussian or exponential integrability assumptions. The corresponding upper bound is characterized by the interplay between sample size and the box-counting dimension of the parameter set measured in an Orlicz norm. This work thus provides a more general and non-asymptotic theoretical framework for evaluating learning algorithm performance without resorting to asymptotic approximations.

This work investigates the fundamental performance limits of learning and estimation tasks within an information-theoretic framework, independent of the computational capabilities of specific algorithms. By integrating tools from information theory and statistical learning theory—including metric entropy, VC dimension, Rademacher complexity, mutual information, and relative entropy—it systematically derives multiple upper bounds on generalization error. Simultaneously, leveraging Fano’s inequality together with covering and packing numbers, the study establishes information-theoretic lower bounds on minimax risk. The analysis unifies two complementary paradigms: one grounded in the geometric structure of metric spaces and the other based on information-theoretic measures. This synthesis yields a rigorous and broadly applicable theoretical framework for characterizing the optimal performance boundaries inherent to learning and estimation problems.

This study addresses the critical challenge of reliably estimating sharp lower bounds for the standard errors of moment condition estimators when cross-sample correlation information is either absent or only partially available. By leveraging geometric inequalities, the authors derive explicit and tight lower bounds on standard errors and show that the general problem can be reformulated as a semidefinite programming (SDP) problem amenable to efficient computation. This approach yields the first sharp error bounds in settings with no knowledge of cross-sample correlations. Integrating insights from moment condition estimation and statistical inference theory, the method demonstrates both validity and practical utility across several applications, including menu cost models, heterogeneous-agent New Keynesian frameworks, and two-sample instrumental variable settings.

This work addresses stochastic approximation problems involving multiplicative noise and unbounded iterates under compression, proposing a unified and elementary analytical framework. By directly constructing a first-order Lyapunov drift inequality for the error norm, the approach avoids sophisticated tools such as smoothing or Moreau envelopes. Combining averaged noise sequences with auxiliary iterates, the method employs inductive expectation arguments to derive mean-square error bounds and, for the first time under multiplicative noise, establishes a full-trajectory maximal concentration bound with sub-Gaussian tails via probabilistic induction and the Azuma–Hoeffding inequality. The stepsize schedule depends only logarithmically on the confidence parameter, eliminating the need for multi-stage warm-up procedures or smooth Lyapunov functions, thereby significantly simplifying the analysis. The framework is successfully applied to ℓ∞-contractive operators in reinforcement learning and exhibits strong generalizability.