Geometric analysis of attractor boundaries and storage capacity limits in kernel Hopfield networks

2026-04-30

AI Summary

This work investigates the dynamical and geometric mechanisms underlying attractor stability in Hopfield networks trained with kernel logistic regression, revealing fundamental limits to their storage capacity. Through experiments involving random sequences and CIFAR-10 embeddings, manifold interpolation, effective barrier analysis, critical slowing-down observations, and signal-to-noise ratio evaluation grounded in Cover’s theorem, the study demonstrates that attractors reside on an “optimization ridge” separated by phase-transition-like boundaries. The findings indicate that the storage limit arises from dynamical instability rather than inseparability in feature space, establishing the network’s operation as a localized exemplar memory system. The model achieves a capacity of P/N ≈ 16 on random data and up to P/N ≈ 20 on structured data, with optimal retrieval performance occurring just before dynamical collapse.

Abstract

High-capacity associative memories based on Kernel Logistic Regression (KLR) exhibit strong storage capabilities, but the dynamical and geometric mechanisms underlying their stability remain poorly understood. This paper investigates the global geometry of attractor basins and the physical determinants of the storage limit in KLR-trained Hopfield networks. We combine empirical evaluations using random sequences and real-world image embeddings (CIFAR-10) with phenomenological morphing experiments and statistical Signal-to-Noise Ratio (SNR) analysis. Our experiments reveal that the network achieves a storage capacity for random sequences up to $P/N \approx 16$ , and maintains stable retrieval for structured data at effective loads near $P/N \approx 20$ . Through morphing analysis, we reveal that attractors on the "Ridge of Optimization" are separated by sharp, phase-transition-like boundaries, characterized by steep effective potential barriers and critical slowing down. Furthermore, by contrasting an SNR analysis with a geometric reference point inspired by Cover's theorem, we show that the ultimate storage limit is constrained primarily not by a lack of geometric separability in the feature space, but by the loss of dynamical stability against crosstalk noise. These findings suggest that KLR networks function as highly localized, exemplar-based memories that operate optimally just before the onset of dynamical collapse, providing new insights into the design of robust, large-scale retrieval systems.

Problem

• attractor boundaries
• storage capacity
• Hopfield networks
• dynamical stability
• geometric analysis

Innovation

• Kernel Hopfield networks
• attractor geometry
• storage capacity limit
• dynamical stability
• Signal-to-Noise Ratio (SNR)