A Rate Separation for Agnostic Direct Sums

2026-08-07

AI Summary

This study investigates the dependence of direct and aggregated learning rates in agnostic PAC learning on the single-instance learning curve and the number of repetitions \( r \). By constructing two function classes that share the same single-instance learning rate of \( n^{-1/2} \) yet exhibit distinct direct aggregation rates, the work demonstrates for the first time that the single-instance rate alone does not uniquely determine the aggregation rate, thereby refuting a strong generalization hypothesis in this direction. The findings reveal a “separation phenomenon” in learning rates and underscore the essential role of aggregation mechanisms in agnostic learning, offering a novel theoretical perspective on the interplay between individual and collective learning behavior.

Abstract

Hanneke, Moran, and Waknine \cite{HannekeMoranWaknine2024} asked how the agnostic PAC learning curve of the direct sum $C^r$ depends on the single-instance learning curve $\epsagn(n\mid C)$ and on $r$. We show that the single-instance learning rate does not determine the direct-sum rate. Let $\F$ be the class of the two constant binary functions and let $\G$ consist of the zero function and the identity function. Both classes have agnostic learning curve of order $n^{-1/2}$.

Problem

• agnostic PAC learning
• direct sum
• learning curve
• learning rate
• rate separation

Innovation

• agnostic PAC learning
• direct sum
• learning rate separation
• sample complexity
• function classes