Kernel Logistic Regression Learning for High-Capacity Hopfield Networks

2025-04-10

AI Summary

Conventional Hopfield networks suffer from limited storage capacity (~0.14 patterns per neuron) and poor noise robustness due to reliance on Hebbian learning, which assumes linear separability. To address these limitations, this work proposes a novel training framework based on kernelized logistic regression—the first integration of kernel methods and logistic regression into Hopfield networks. By leveraging implicit high-dimensional nonlinear feature mappings and dual optimization, the approach relaxes the linear separability constraint and achieves superlinear storage capacity. Experiments demonstrate perfect recall (100%) even at a pattern-to-neuron ratio of 1.5, substantially outperforming both Hebbian learning and linear logistic regression baselines. Moreover, the method exhibits enhanced robustness against input noise. This work establishes a new paradigm for modeling associative memory with simultaneously high capacity and strong resilience.

Technology Category

• Machine Learning: Kernel Methods
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Hebbian learning limits Hopfield network storage capacity (pattern-to-neuron ratio around 0.14). We propose Kernel Logistic Regression (KLR) learning. Unlike linear methods, KLR uses kernels to implicitly map patterns to high-dimensional feature space, enhancing separability. By learning dual variables, KLR dramatically improves storage capacity, achieving perfect recall even when pattern numbers exceed neuron numbers (up to ratio 1.5 shown), and enhances noise robustness. KLR demonstrably outperforms Hebbian and linear logistic regression approaches.

Problem

• Increase Hopfield network storage capacity
• Enhance pattern separability using kernels
• Improve noise robustness in recall

Innovation

• Uses Kernel Logistic Regression learning
• Maps patterns to high-dimensional feature space
• Achieves high storage capacity and robustness