Kernel Ridge Regression for Efficient Learning of High-Capacity Hopfield Networks

2025-04-17

AI Summary

Hebbian learning in Hopfield networks suffers from limited storage capacity and poor noise robustness. To address this, we propose a closed-form, non-iterative training method based on kernel ridge regression (KRR)—the first integration of KRR into Hopfield networks—leveraging high-dimensional feature mapping and dual-variable optimization for associative memory modeling. Our approach overcomes the Hebbian capacity bottleneck, achieving a theoretical storage capacity of 1.5 times the number of stored patterns and enabling accurate pattern recovery even under 80% pattern corruption. Crucially, training is analytical and iteration-free, yielding speedups of up to an order of magnitude. The core innovation lies in replacing conventional iterative kernel logistic regression with an exact analytical solution, thereby simultaneously enhancing computational efficiency, noise robustness, and scalability. This establishes a novel, scalable paradigm for large-scale associative memory systems.

Technology Category

• Machine Learning: Kernel Methods
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Hebbian learning limits Hopfield network capacity. While kernel methods like Kernel Logistic Regression (KLR) improve performance via iterative learning, we propose Kernel Ridge Regression (KRR) as an alternative. KRR learns dual variables non-iteratively via a closed-form solution, offering significant learning speed advantages. We show KRR achieves comparably high storage capacity (reaching ratio 1.5 shown) and noise robustness (recalling from around 80% corrupted patterns) as KLR, while drastically reducing training time, establishing KRR as an efficient method for building high-performance associative memories.

Problem

• Enhancing Hopfield network capacity via Kernel Ridge Regression
• Non-iterative learning for faster training speed
• Achieving high storage capacity and noise robustness

Innovation

• Kernel Ridge Regression for Hopfield Networks
• Non-iterative closed-form solution learning
• High storage capacity and noise robustness