derive minimax bounds

This work studies the cumulative regret of one-dimensional noisy Bayesian optimization (BO) under Gaussian process priors and Gaussian observation noise. Methodologically, it employs information-theoretic analysis, RKHS complexity estimation, and confidence interval construction. The key contribution is the first nontrivial lower bound on regret, tightly complementing the classical upper bound of Srinivas et al. (2009): for the squared-exponential (SE) kernel, the regret is shown to be Ω(√T) and O(√(T log T)), achieving near-tight characterization (up to a √log T factor); for the Matérn-ν kernel with ν > 2, the existing upper bound is proven strictly suboptimal and improvable. These results provide the tightest known characterization of regret growth for one-dimensional noisy BO and reveal the fundamental role of kernel smoothness in determining regret bounds.

This paper investigates the regret (redundancy) of sequential prediction, data compression, and gambling relative to smooth parametric families—specifically exponential families, general smooth parametric models, and Markov sources. It analyzes the asymptotic regret performance of Bayesian mixture distributions, particularly Jeffreys-prior-based variants, under maximum-likelihood estimation. The key contribution is the first rigorous proof that such Jeffreys-type mixture priors achieve asymptotically minimax regret over all these model classes, with redundancy converging at the optimal rate of $O(1/n)$. This rate matches the information-theoretic lower bound dictated by the Shtarkov normalized constant. By unifying tools from information geometry, asymptotic statistics, and normalized maximum-likelihood theory, the work establishes a fundamental connection between Bayesian mixtures and information-theoretic optimality. It thus provides a unified, theoretically grounded guarantee of optimality for universal coding and prediction.

This paper investigates statistical decision-making for treatment choice under partial identification, focusing on theoretical properties and practical challenges of welfare maximization and regret minimization within the Gaussian likelihood framework. We propose the “profile regret” criterion to systematically distinguish decision rules and uniquely characterize the minimax-regret-optimal nonrandomized decision rule—its first such characterization. Key theoretical findings include: (i) discarding certain data may improve minimax welfare performance; and (ii) under strong partial identification, infinitely many randomized rules achieve minimax regret optimality. The method is validated across canonical settings—including experimental estimate aggregation, LATE extrapolation, and omitted-variable bias correction—demonstrating robustness and practical applicability. Our work establishes a novel theoretical foundation and provides implementable tools for robust policy decision-making under partial identification.

This paper addresses the selection problem of identifying the optimal $m$ units out of $n$ to maximize the true total value, under highly noisy and heteroscedastic observations. We propose an empirical Bayes estimation-and-selection framework. Our key theoretical contribution is the first rigorous proof that, when the prior estimation error is $O_p(r_n)$, the selection regret is bounded by $O_p(r_n^2)$—a rate shown to be tight under the given parametric assumptions. The method integrates parametric prior modeling, asymptotic regret analysis, and calibration using real-world online experimentation data. Extensive evaluation across over 4,000 online A/B tests demonstrates that our approach achieves high-precision identification of optimal interventions with minimal experimental overhead, substantially outperforming naive thresholding methods.

Existing min-max robust learning methods for heterogeneous data with known subgroup structures suffer from poor cross-subgroup generalization and susceptibility to dominance by noisy subgroups. Method: We propose a Minimum Maximum Regret (MMR) supervised learning framework that optimizes for the lower bound on subgroup performance—rather than average performance—thereby introducing regret minimization to subgroup-robust learning for the first time. Our approach integrates decision-theoretic MMR optimization, subgroup-aware loss design, and super-population generalization error analysis. Contribution/Results: The framework ensures robustness, invariance, and strong generalization guarantees. Evaluated on synthetic data and a real-world kidney transplantation cohort spanning hundreds of transplant centers, it significantly improves worst-subgroup predictive performance. Empirical results validate both its theoretical rigor and clinical applicability.

This study investigates the asymptotic behavior of minimal regret in Bayesian statistical decision problems with finite state and action spaces. By introducing multivariate Chernoff information and analyzing incompatible subsets of states, the work establishes—for the first time—an exact expression for the exponential decay rate of Bayesian regret under arbitrary loss functions. This result not only unifies and generalizes the classical Chernoff exponent theory from multi-hypothesis testing but also yields precise regret exponents for complex settings such as list hypothesis testing. The analysis reveals the fundamental mechanism underlying the exponential decay of regret under optimal strategies, providing a comprehensive characterization of the asymptotic optimality in Bayesian sequential decision-making.

This work investigates the direct relationship between the regret (excess risk) of unregularized Bayes rules and the Hellinger distance of marginal densities within a Gaussian empirical Bayes framework. By developing an analytical framework that avoids recursive regularization and leveraging polynomial approximation together with Bernstein-type inequalities in weighted L² spaces, the authors establish sharp, non-asymptotic bounds linking these two quantities. This approach yields sharper—sometimes minimax optimal—regret bounds and clarifies the necessity of regularization under heavy-tailed priors. Specifically, for compactly supported priors, a regret upper bound of order \(O(\varepsilon^2 \log(1/\varepsilon)/\log\log(1/\varepsilon))\) is established; improved bounds are also derived for exponential-tail priors, leading to enhanced regret performance for nonparametric maximum likelihood estimation.

This work addresses the computational burden of sample average approximation in quantifying regret due to uncertainty in stochastic optimization. It establishes, for the first time, an exact equality linking regret to the covariance between uncertain parameters and optimal decisions, proving that the residual term vanishes under specific conditions. This yields closed-form, approximation-free expressions for linear programs and unconstrained quadratic programs. By integrating covariance estimation, smoothness analysis, and concentration inequalities, the proposed method requires only a single pass over the data, reducing computational complexity from 𝒪(Bn²d³) to 𝒪(nd²). Empirical validation on synthetic LP/QP instances, integer programs, and a decade-long rolling portfolio optimization task using CRSP data demonstrates both high efficiency and accuracy.

This work investigates the simple regret and minimax optimality of a fixed-prior expected improvement (EI) strategy for optimizing deterministic black-box functions in a reproducing kernel Hilbert space (RKHS). Assuming a zero-mean Gaussian process prior, the authors analyze a sequential sampling rule that selects points achieving at least a fixed fraction of the maximal EI. They establish, for the first time, finite-budget upper bounds on simple regret: $O(N^{-\nu/d})$ under the Matérn kernel and $O(\exp[-c N^{1/d} \log(eN)])$ under the squared exponential kernel. These rates are minimax optimal for the Matérn kernel and exponentially close to optimal for the squared exponential kernel, uniformly over any RKHS ball. The analysis leverages Gram determinants, Kolmogorov widths, a one-step regret inequality, and the RKHS–Gaussian process duality to characterize algorithmic performance.