Majority-of-Three is Optimal

2026-06-11

AI Summary

This work investigates the construction of structurally minimal yet theoretically optimal ensemble learners within the realizable PAC learning setting. By integrating PAC learning theory, majority voting, and probabilistic analysis, the study establishes—for the first time—that aggregating just three independent consistent classifiers via majority vote suffices to achieve the optimal risk bound. This result not only demonstrates the theoretical optimality of triple-classifier majority voting but also substantially simplifies both algorithmic design and accompanying theoretical analysis. Furthermore, it provides a unified improvement over existing approaches, including Hanneke’s algorithm and Bagging, by offering a more streamlined framework with stronger theoretical guarantees.

Abstract

We give a short proof that the majority vote of three independent consistent classifiers is an optimal learner in the realizable PAC setting. This proves optimality for the simplest voting scheme, while simplifying both the algorithmic structure and the probabilistic analysis of previous voting learners, including the algorithm of S. Hanneke and the analysis of bagging by K. Green Larsen.

Problem

• Majority Vote
• Optimal Learner
• PAC Learning
• Consistent Classifiers
• Realizable Setting

Innovation

• Majority-of-Three
• PAC learning
• voting classifier
• optimal learner
• bagging