Learning Kernels by Alignment for Multiclass Bayes Classification

2026-09-06

AI Summary

本文通过学习核对齐来解决多类贝叶斯分类中预先选择核的问题，使用协作学习和推理框架，并引入马氏距离改进了分类性能。

Abstract

Kernel methods separate data representation from decision-making, but typically require the kernel to be chosen in advance. We show that this kernel can instead be learned by alignment, and develop the resulting framework through the recently introduced Collaborative Learning and Inference (CLaI). We show that Collaborative Learning can be viewed as a kernel alignment process, in which an embedding is trained so that its induced similarity matches a label-derived target kernel. We also prove that Collaborative Inference is equivalent to kernel Bayes classification with Parzen-window density estimation. Motivated by these perspectives, we generalise CLaI by replacing cosine similarity with a learned Mahalanobis distance and extend it to multiclass classification. On CIFAR-10, PathMNIST, and SleepEDF, the Mahalanobis formulation improves accuracy, converges faster, and yields lower calibration error than the cosine-based variant. Auxiliary experiments further support these connections, showing that CLaI produces latent signals of the same form as a Gaussian process, while achieving competitive calibration on sepsis prediction. Together, these results establish a principled learned-kernel framework that unifies representation learning, kernel alignment, and Bayesian classification, and extends naturally to the multiclass setting.

Problem

• Kernel Methods
• Multiclass Bayes Classification
• Kernel Alignment
• Collaborative Learning and Inference (CLaI)
• Representation Learning

Innovation

• Kernel Alignment
• Collaborative Learning and Inference (CLaI)
• Mahalanobis Distance
• Multiclass Classification
• Learned Kernel