Universal Multiclass Transductive Online Learning

2026-05-28

AI Summary

This study addresses transductive online multiclass classification in settings where the label space may be unbounded and the sequence of instances is known in advance while labels remain unknown. The work introduces a novel combinatorial structure—the Hierarchically Constrained Littlestone Tree (HCLT)—which, combined with indifference, fully characterizes learnability in this framework. It establishes, for the first time, that any learnable class admits only two possible optimal mistake rates: either bounded or growing logarithmically with the sequence length, thereby providing necessary and sufficient conditions for learnability in general multiclass transductive online learning. These results are further extended to the agnostic setting and to instance sequences generated by stochastic processes, revealing a dichotomous structure in achievable mistake rates and establishing corresponding theoretical bounds.

Abstract

We consider the problem of universal transductive online classification with a possibly unbounded label space. This setting considers online learning, with the sequence of instances (without labels) known to the learner in advance. We say a concept class $\mathcal{H}$ is learnable if there is a learning algorithm $\mathcal{A}$, such that for every realizable sequence, the number of mistakes made by $\mathcal{A}$ grows at most sublinearly with the number of predictions. We characterize the learnability of this setting and show that there are only two possible optimal rates for the learnable classes: either bounded or increasing logarithmically. We introduce a new combinatorial structure, called ``Level-Constrained-Littlestone-Littlestone (LCLL) tree'', which, along with the indifference property, characterizes the learnability. We also extend the learnability result to the agnostic case and the case where only the stochastic process that generates the instance sequence is known.

Problem

• transductive online learning
• multiclass classification
• unbounded label space
• learnability
• universal learning

Innovation

• transductive online learning
• unbounded label space
• LCLL tree
• learnability characterization
• sublinear mistake bound