🤖 AI Summary
To address the lack of local complexity tools for generalization analysis in transductive learning, this paper introduces the **Transductive Local Complexity (TLC)** framework—the first extension of Local Rademacher Complexity (LRC) to the transductive setting. Methodologically, we derive sharp concentration inequalities for the test–training empirical process, design a novel peeling strategy, and propose a surrogate variance operator, thereby establishing an excess risk bound analysis paradigm consistent with inductive LRC. The resulting generalization upper bound is the tightest known for transductive kernel learning (TKL). Additionally, we develop a general sharp concentration inequality for sampling without replacement from a uniform distribution, which holds independent theoretical significance. This work fills a fundamental theoretical gap in local complexity theory for transductive learning and substantially enhances both the precision and applicability of generalization analysis.
📝 Abstract
We introduce a new tool, Transductive Local Complexity (TLC), designed to analyze the generalization performance of transductive learning methods and inspire the development of new algorithms in this domain. Our work extends the concept of the popular Local Rademacher Complexity (LRC) to the transductive setting, incorporating significant and novel modifications compared to the typical analysis of LRC methods in the inductive setting. While LRC has been widely used as a powerful tool for analyzing inductive models, providing sharp generalization bounds for classification and minimax rates for nonparametric regression, it remains an open question whether a localized Rademacher complexity-based tool can be developed for transductive learning. Our goal is to achieve sharp bounds for transductive learning that align with the inductive excess risk bounds established by LRC. We provide a definitive answer to this open problem with the introduction of TLC. We construct TLC by first establishing a novel and sharp concentration inequality for the supremum of a test-train empirical processes. Using a peeling strategy and a new surrogate variance operator, we derive the a novel excess risk bound in the transductive setting which is consistent with the classical LRC-based excess risk bound in the inductive setting. As an application of TLC, we employ this new tool to analyze the Transductive Kernel Learning (TKL) model, deriving sharper excess risk bounds than those provided by the current state-of-the-art under the same assumptions. Additionally, the concentration inequality for the test-train process is employed to derive a sharp concentration inequality for the general supremum of empirical processes involving random variables in the setting of uniform sampling without replacement. The sharpness of our derived bound is compared to existing concentration inequalities under the same conditions.
