A Geometry-Based Capacity Theory for Finite-Feature Associative Memory

2026-10-06

AI Summary

This study addresses the challenge of predicting exact key-retrieval capacity for finite-feature associative memories under compressed storage. We propose a geometry-based, fitting-free capacity theory that employs Hebbian learning, kernel overlap analysis, and Gram matrix interaction modeling to effectively disentangle noise from structural interference. The core contribution reveals that retrieval quality is governed by representational geometry rather than feature dimensionality alone. Accordingly, we establish geometry-dependent capacity upper bounds, providing explicit decision criteria for budget allocation or representation modification. Experiments on both synthetic and real-world datasets demonstrate that theoretical predictions precisely match empirical behavior, offering effective guidance for optimal system design.

Abstract

We develop a geometry-based capacity theory for exact-key retrieval in compressed finite-feature Hebbian associative memory. For random or approximately isotropic values, retrieval interference separates into finite-feature noise, which decreases with feature dimension, and structural interference, which is determined by squared kernel overlap among stored keys and persists in the infinite-feature limit. This yields a fit-free prediction of retrieval quality, reveals a geometry-dependent capacity ceiling, and predicts the feature budget required for a target retrieval quality. When stored values are correlated, we show that retrieval depends jointly on the key kernel and value Gram matrix, and derive finite-feature approximations that account for this interaction. We validate the theory on synthetic, visual, and medical-image representations. Overall, the framework links representation geometry directly to memory capacity and distinguishes when performance can be improved by increasing the feature budget and when the representation itself must be changed. Across these settings, the predicted retrieval curves closely match empirical behavior and correctly identify changes in the preferred memory design.

Problem

• associative memory
• retrieval capacity
• representation geometry
• finite-feature noise
• exact-key retrieval

Innovation

• associative memory
• capacity theory
• representation geometry
• Hebbian learning
• finite-feature approximation