🤖 AI Summary
This work addresses the computational complexity and high memory overhead associated with optimizing classifier weights in high- or infinite-dimensional kernel spaces. We propose a simple averaging classifier based on kernel mean embeddings and, for the first time, apply the herding algorithm to sparsify it. Unlike conventional weighted kernel classifiers, our approach constructs an unbiased, high-fidelity, and inherently parallelizable sparse approximation without altering the original learning objective. The resulting classifier preserves theoretical consistency and robustness while significantly reducing prediction latency and memory footprint. Empirically, it achieves competitive accuracy alongside superior efficiency and scalability. Moreover, its design naturally supports distributed implementation, offering a lightweight and reliable paradigm for large-scale kernel methods. (126 words)
📝 Abstract
Many classification algorithms produce a classifier that is a weighted average of kernel evaluations. When working with a high or infinite dimensional kernel, it is imperative for speed of evaluation and storage issues that as few training samples as possible are used in the kernel expansion. Popular existing approaches focus on altering standard learning algorithms, such as the Support Vector Machine, to induce sparsity, as well as post-hoc procedures for sparse approximations. Here we adopt the latter approach. We begin with a very simple classifier, given by the kernel mean $$ f(x) = frac{1}{n} sumlimits_{i=i}^{n} y_i K(x_i,x) $$ We then find a sparse approximation to this kernel mean via herding. The result is an accurate, easily parallelized algorithm for learning classifiers.