Bridging the Gap Between Hyperdimensional Computing and Kernel Methods via the Nyström Method

2026-08-07

AI Summary

This work addresses the limited flexibility of existing Hyperdimensional Computing (HDC) approaches, which rely on fixed mappings from raw data to high-dimensional space and struggle to adapt to diverse machine learning tasks. To overcome this limitation, the authors propose NysHD, a novel method that, for the first time, integrates the Nyström approximation from kernel methods into HDC. NysHD leverages any user-defined positive semi-definite similarity function to automatically construct effective high-dimensional embeddings, thereby establishing a theoretical bridge between kernel methods and HDC. By synergistically combining Nyström approximation, positive definite kernel design, and high-dimensional random encoding, the approach substantially enhances model expressivity. Experimental results on graph and string datasets demonstrate that NysHD improves classification accuracy by 11% and 17% on average over state-of-the-art HDC encoding schemes.

Abstract

Hyperdimensional computing (HDC) is an approach from the cognitive science literature for solving information processing tasks using data represented as high-dimensional random vectors. The technique has a rigorous mathematical backing, and is easy to implement in energy-efficient and highly parallel hardware like FPGAs and "processing-in-memory" architectures. The effectiveness of HDC in machine learning largely depends on how raw data is mapped to high-dimensional space. In this work, we propose NysHD, a new method for constructing this mapping that is based on the Nyström method from the literature on kernel approximation. Our approach provides a simple recipe to turn any user-defined positive-semidefinite similarity function into an equivalent mapping in HDC. There is a vast literature on the design of such functions for learning problems. Our approach provides a mechanism to import them into the HDC setting, expanding the types of problems that can be tackled using HDC. Empirical evaluation against existing HDC encoding methods shows that NysHD can achieve, on average, 11% and 17% better classification accuracy on graph and string datasets respectively.

Problem

• Hyperdimensional Computing
• Kernel Methods
• Nyström Method
• Similarity Function
• High-dimensional Mapping

Innovation

• Hyperdimensional Computing
• Nyström Method
• Kernel Approximation
• Similarity Function
• High-dimensional Mapping