minimax pac bounds

This paper addresses the non-monotonic phenomenon in machine learning where increased training data paradoxically degrades test performance. Grounded in PAC learning theory, it provides the first rigorous proof that, under the i.i.d. assumption, the test error of Empirical Risk Minimization (ERM) is monotonically non-increasing with sample size for any PAC-learnable problem with finite VC dimension or finite hypothesis space—establishing monotonicity as an intrinsic property of PAC learnability, not a special condition. The analysis integrates VC-dimension theory, sample complexity bounds, and statistical modeling, and is empirically validated to confirm consistency between theoretical lower bounds and observed risk decay. This result rectifies prior misconceptions treating monotonicity as contingent on additional constraints and furnishes foundational theoretical guarantees for designing robust learning algorithms. (149 words)

This work investigates the fundamental distinction between *sampleable* PAC learning—where learners must succeed only on efficiently samplable distributions—and standard PAC learning—which requires uniform correctness across all distributions—and its implications for learning efficiency. We introduce the novel complexity-theoretic primitive of an *explicit evasion set*, and in the random oracle model, construct a concept class that is exponentially hard to learn under standard PAC (requiring exponentially many samples) yet polynomially learnable under sampleable PAC. This yields the first computational separation between the two learning paradigms. Our result demonstrates that distributional samplability can provably reduce sample complexity, thereby expanding the frontier of efficient learnability. Furthermore, we extend this separation to the online learning setting, establishing that the computational power of the adversary fundamentally constrains learnability: restricting the adversary to polynomial-time computation enables efficient learning where computationally unbounded adversaries preclude it.

This paper investigates the trade-off between sample complexity and round complexity in multi-distribution learning under on-demand sampling. We propose a novel framework—On-Demand Distribution Sampling (OODS)—that unifies modeling for both realizable and universally adversarial settings. Leveraging adaptive sampling, statistical learning theory, and complexity analysis, we design near-optimal algorithms and establish tight upper and lower bounds. In the realizable setting, our algorithm achieves optimal sample complexity $ ilde{O}(d k^{Theta(1/r)} / varepsilon)$; in the adversarial setting, it attains sample complexity $ ilde{O}((d + k)/varepsilon^2)$ within $ ilde{O}(sqrt{k})$ rounds. Crucially, we identify that achieving subpolynomial round complexity necessitates fundamentally new techniques—thereby exposing and overcoming inherent limitations of existing approaches. Our work provides the first rigorous characterization of this trade-off and advances the theoretical foundations of interactive multi-distribution learning.

This work resolves the problem of optimal sample complexity in agnostic PAC learning for hypothesis classes with finite VC dimension. By integrating VC theory, refined concentration inequalities, and the empirical risk minimization framework, it introduces the first algorithm that achieves the information-theoretic lower bound—up to a universal constant factor—for any optimal risk \(L^*\). The proposed method guarantees, with high probability, a risk upper bound of \(L(\hat{h}) \leq L^* + 7\cdot10^8\left(\sqrt{\frac{L^*(d+\log(1/\delta))}{n}} + \frac{d+\log(1/\delta)}{n}\right)\), which exactly matches the known minimax lower bound. This result establishes the first tight characterization of sample complexity across the full range of parameters in agnostic learning.

This work addresses the lack of provable joint guarantees on generalization and convergence in learned optimization algorithms. Methodologically: (1) it establishes the first PAC-Bayesian generalization bound for unbounded losses, leveraging exponential-family posterior distributions; (2) it formulates optimizer learning as a tractable one-dimensional global optimization problem—convex or non-convex—whose solution is analytically characterizable; and (3) it integrates stochastic optimization design with rigorous theoretical analysis to explicitly trade off convergence rate against generalization error. Empirically, the learned optimizers achieve order-of-magnitude improvements over state-of-the-art methods across four diverse real-world tasks—including neural architecture search, meta-learning, adversarial training, and federated learning—while all gains are underpinned by formal theoretical guarantees. This constitutes the first learning-to-optimize framework endowed with a provably tight PAC-Bayesian generalization bound and jointly certified convergence–generalization performance.

This study addresses the long-standing open problem in the theory of machine learning safety certificates regarding whether preference compression bounds are tight. Drawing upon sample compression and learning stability theories, this work constructs extremal instances that achieve the limit by leveraging uniform distributions and order statistics. Furthermore, it presents a concise novel proof based on combinatorial counting, thereby circumventing the intricate derivations associated with traditional infinite-dimensional duality. The primary contribution is the first rigorous confirmation of the tightness of these bounds and the establishment of optimal limits. This achievement substantially reduces the complexity of theoretical analysis and provides a solid theoretical foundation for the safe application of algorithms such as scenario optimization and support vector machines.

研究使用随机梯度oracle在固定维度下采样平滑强对数凹分布的复杂度，给出同时适应于条件数和精度的紧致复杂度边界。

This study investigates the randomized algorithmic complexity lower bounds for linear optimization, uniform sampling, and volume estimation of convex bodies within the membership query model. By constructing specific hard instances, it establishes near-quadratic time lower bounds for these problems. The core contribution lies in elevating the lower bound for uniform sampling from linear to near-quadratic for the first time, while simultaneously deriving an identical bound for volume estimation, thereby filling a notable theoretical gap. Furthermore, the obtained lower bound for linear optimization matches the best known upper bound up to logarithmic factors. These results significantly advance existing theoretical bounds and precisely delineate the fundamental complexity limits of the aforementioned computational tasks.

研究在线算法利用无偏样本作为离线建议以实现超越最坏情况性能的问题，提出紧致竞争比算法，并探讨了对抗鲁棒性。

This work investigates the fundamental trade-off between memory and first-order oracle query complexity for convex optimization under limited memory. By leveraging information-theoretic analysis, adversarially constructed functions, and refined entropy arguments, it establishes the first sharp phase transition in deterministic algorithms: when memory is on the order of $d^2$, a dramatic change in achievable query complexity occurs. The study proves that $\widetilde{\Omega}(d^2)$ memory is necessary to attain near-optimal query complexity for polynomially small suboptimality gaps. These results significantly strengthen existing lower bounds on query complexity across the entire memory spectrum, both for randomized and deterministic algorithms.