Using Enriched Category Theory to Construct the Nearest Neighbour Classification Algorithm

2023-12-27

arXiv.org

AI Summary

This work addresses the poor interpretability and rigid decision boundaries of nearest-neighbor (NN) classification. We propose the first fully category-theoretic reconstruction framework for NN methods. Methodologically, we model data and label spaces as Lawvere metric spaces, extend the original dataset via profunctors, and *purely deductively derive* both 1-NN and k-NN classifiers within this categorical structure. This derivation naturally yields weighted Voronoi tessellations and introduces a label metric to define soft, geometry-aware decision boundaries—enabling continuous, label-structure-dependent predictions. Key contributions include: (i) establishing the first category-theory-driven, interpretable theoretical foundation for NN classification; (ii) unifying 1-NN and k-NN under a single formal categorical framework; and (iii) enabling flexible, label-metric-guided generalization, thereby substantially enhancing model transparency and adaptability. The framework bridges abstract category theory with practical machine learning, offering principled insights into NN behavior beyond heuristic justification.

Abstract

This paper is the first to construct and motivate a Machine Learning algorithm solely with Enriched Category Theory, supplementing evidence that Category Theory can provide valuable insights into the construction and explainability of Machine Learning algorithms. It is shown that a series of reasonable assumptions about a dataset lead to the construction of the Nearest Neighbours Algorithm. This construction is produced as an extension of the original dataset using profunctors in the category of Lawvere metric spaces, leading to a definition of an Enriched Nearest Neighbours Algorithm, which, consequently, also produces an enriched form of the Voronoi diagram. Further investigation of the generalisations this construction induces demonstrates how the $k$ Nearest Neighbours Algorithm may also be produced. Moreover, how the new construction allows metrics on the classification labels to inform the outputs of the Enriched Nearest Neighbour Algorithm: Enabling soft classification boundaries and dependent classifications. This paper is intended to be accessible without any knowledge of Category Theory.

Problem

• Machine Learning
• Nearest Neighbor Classification
• Interpretability

Innovation

• Enriched Category Theory
• Co-functor based KNN
• Flexible Classification