Margin in Abstract Spaces

2026-03-07

AI Summary

This work investigates the minimal mathematical structure required to guarantee parameter-free generalization in margin-based learning. Focusing on margin learnability in arbitrary metric spaces, it defines concept classes via centers and distance thresholds and establishes sufficient conditions for learnability relying solely on the triangle inequality. The core contributions include introducing a universal margin threshold γ, proving that any concept class in a metric space is learnable when R > 3r, and demonstrating that in Banach spaces the sample complexity necessarily scales as (1/γ)^p for some p ≥ 2—and that this rate is achievable. Furthermore, the study refutes the conjecture that all margin-based learnability can be reduced to linear embeddings.

Technology Category

• Machine Learning: Learning Theory
• Game Theory and Economic Paradigms: Adversarial Learning
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

Margin-based learning, exemplified by linear and kernel methods, is one of the few classical settings where generalization guarantees are independent of the number of parameters. This makes it a central case study in modern highly over-parameterized learning. We ask what minimal mathematical structure underlies this phenomenon. We begin with a simple margin-based problem in arbitrary metric spaces: concepts are defined by a center point and classify points according to whether their distance lies below $r$ or above $R$. We show that whenever $R>3r$, this class is learnable in \emph{any} metric space. Thus, sufficiently large margins make learnability depend only on the triangle inequality, without any linear or analytic structure. Our first main result extends this phenomenon to concepts defined by bounded linear combinations of distance functions, and reveals a sharp threshold: there exists a universal constant $\gamma>0$ such that above this margin the class is learnable in every metric space, while below it there exist metric spaces where it is not learnable at all. We then ask whether margin-based learnability can always be explained via an embedding into a linear space -- that is, reduced to linear classification in some Banach space through a kernel-type construction. We answer this negatively by developing a structural taxonomy of Banach spaces: if a Banach space is learnable for some margin parameter $\gamma\geq 0$, then it is learnable for all such $\gamma$, and in infinite-dimensional spaces the sample complexity must scale polynomially in $1/\gamma$. Specifically, it must grow as $(1/\gamma)^p$ for some $p\ge 2$, and every such rate is attainable.

Problem

• margin-based learning
• metric spaces
• learnability
• Banach spaces
• generalization

Innovation

• margin-based learning
• metric spaces
• generalization
• Banach space embedding
• sample complexity