Boosting and the Expressive Power of Simple Weak Learners via the $γ$-VC Dimension

2026-10-07

AI Summary

This study addresses the limitation that the expressive capacity of weak learners in boosting algorithms is constrained by their base class structure, and that the associated sample complexity lacks precise characterization. By integrating statistical learning theory, combinatorial geometry, and analytical techniques from boosting theory, this work establishes for the first time that the γ-VC dimension precisely characterizes the sample complexity of weak-to-strong learning, while systematically elucidating its theoretical relationship with the classical VC dimension. Consequently, it introduces a rigorous metric for quantifying the expressive power of weak learners and significantly refines both the upper and lower bounds of the γ-VC dimension for fundamental concept classes, such as decision stumps.

Abstract

Boosting converts weak hypotheses with a small edge over random guessing into highly accurate predictors, but the expressive power of the resulting classifier can depend strongly on the structure of the base class. We study this phenomenon through the $γ$-VC dimension introduced by Alon et al. (STOC 2021). Our first result shows that this parameter characterizes the sample complexity for weak-to-strong learning up to a constant factor scaling in $γ$. We then sharpen the general relationship between the classic VC dimension and the $γ$-VC dimension. Finally, we also give improved upper and lower bounds on the $γ$-VC dimension for the fundamental concept classes of decision stumps and axis-parallel rectangles in $\mathbb{R}^d$.

Problem

• Boosting
• weak learners
• expressive power
• gamma-VC dimension
• sample complexity

Innovation

• Boosting
• gamma-VC dimension
• sample complexity
• weak-to-strong learning
• expressive power