On the Computability of Multiclass PAC Learning

2025-02-10

AI Summary

This work systematically investigates the computability of multiclass learning within the computable PAC (CPAC) framework, focusing on finite label spaces. Method: We introduce the first computable Natarajan dimension and establish it as a necessary and sufficient characterization of CPAC learnability; we further construct a computable family of discriminators to yield a meta-characterization of CPAC learnability. To clarify foundational distinctions, we prove that the DS dimension—even over finite label spaces—cannot be captured by any discriminator family. Results: Our work fully computabilizes Ben-David et al.’s non-computational discriminator-dimension theory, providing a complete theoretical characterization of CPAC learnability in finite-label settings. It reveals that computability imposes fundamental, inherent limitations on learning capacity, thereby delineating the precise boundary of what is algorithmically learnable in this regime.

Technology Category

• Machine Learning: Learning Theory
• Natural Language Processing: Learning & Optimization for NLP
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We study the problem of computable multiclass learnability within the Probably Approximately Correct (PAC) learning framework of Valiant (1984). In the recently introduced computable PAC (CPAC) learning framework of Agarwal et al. (2020), both learners and the functions they output are required to be computable. We focus on the case of finite label space and start by proposing a computable version of the Natarajan dimension and showing that it characterizes CPAC learnability in this setting. We further generalize this result by establishing a meta-characterization of CPAC learnability for a certain family of dimensions: computable distinguishers. Distinguishers were defined by Ben-David et al. (1992) as a certain family of embeddings of the label space, with each embedding giving rise to a dimension. It was shown that the finiteness of each such dimension characterizes multiclass PAC learnability for finite label space in the non-computable setting. We show that the corresponding computable dimensions for distinguishers characterize CPAC learning. We conclude our analysis by proving that the DS dimension, which characterizes PAC learnability for infinite label space, cannot be expressed as a distinguisher (even in the case of finite label space).

Problem

• Computable multiclass PAC learnability
• Natarajan dimension in CPAC
• Characterizing CPAC with distinguishers

Innovation

• Computable PAC learning framework
• Computable Natarajan dimension
• DS dimension characterization