Structuring Relations Among Learning Paradigms via Protocol--Objective--Resource Reductions

2026-09-26

AI Summary

This study addresses the ambiguous relationships and lack of a unified hierarchy among machine learning paradigms by proposing the Protocol-Objective-Resource (POR) framework. Through reduction techniques such as environment embedding and learner compilation, it disentangles representational capacity from design choices, establishing a structured language for analyzing complexity dominance relations across paradigms. Key contributions include achieving the first rigorous inter-paradigm reductions, demonstrating the necessity of non-trivial accuracy regimes to avoid vacuous comparisons, and revealing that strengthening objectives may increase minimax sample complexity. Furthermore, this work formalizes canonical specialization relationships—establishing that continual learning subsumes transfer learning, which in turn subsumes supervised learning—and enables the transfer of complexity guarantees across paradigms.

Abstract

Modern machine learning spans supervised, transfer, continual, meta-learning, and related regimes that often reuse the same hypothesis classes, architectures, and optimizers but differ in information access, objectives, memory, adaptation, and sample accounting. This makes it difficult to determine whether one paradigm is genuinely distinct, a special case of another, or part of a broader structural hierarchy. We introduce a protocol-objective-resource (POR) framework that separates representational capacity from these design choices. A paradigm is specified by an environment class, observation protocol, admissible learners, performance functional, and resource accounting rule. POR reductions combine environment embeddings, learner compilers, threshold maps, and calibrated resource overheads. Our main theorem shows that such reductions imply worst-case complexity domination on embedded comparison classes, transferring upper bounds forward and lower bounds backward; under labeled-example accounting, this yields sample complexity domination. We also show that calibrated nontrivial accuracy regimes are necessary to avoid vacuous comparisons, and that strengthening the objective can strictly increase minimax sample complexity even with unchanged protocols and learner classes. Instantiating the framework for supervised, transfer, continual, and meta-learning yields canonical special-case relations: continual contains transfer, transfer contains supervised, and meta-learning contains supervised under aligned raw-example accounting. We further derive a non-exact episode-to-example reduction for episodic meta-learning and capture within-paradigm refinements such as replay memory and task identifiers. The framework thus provides a unified language for structuring learning paradigms and transferring complexity guarantees across them.

Problem

• learning paradigms
• machine learning
• structural hierarchy
• sample complexity
• paradigm relations

Innovation

• Protocol-Objective-Resource framework
• POR reductions
• sample complexity domination
• learning paradigms structuring
• minimax complexity