Constrained Hebbian Learning Supports Efficient Representational Allocation under Structural Constraints

2026-07-17

AI Summary

This study addresses how synaptic resources can be efficiently allocated under neuroanatomical and metabolic constraints to achieve low-redundancy, high-functionality information representations. Building upon the information bottleneck framework and employing a variational mutual information metric, the authors systematically evaluate the capacity of excitatory competitive Hebbian learning to allocate representational resources under fixed audio-visual embeddings and sparse architectural constraints. The results demonstrate that Hebbian learning does not merely enhance classification accuracy; rather, it significantly reduces representational cost while maintaining comparable task performance. It outperforms sparse backpropagation and decoupled direct target propagation (DDTP), approaching the efficiency of shallow non-negative weight backpropagation, thereby revealing its potential to achieve superior cost–performance trade-offs under biologically plausible constraints.

Abstract

Introduction: Biological systems face anatomical and metabolic constraints, including costly synaptic maintenance and limited connectivity. These constraints favor neural codes that compress behaviorally relevant information into low-redundancy patterns. We test whether an excitatory competitive Hebbian rule can support synaptic resource allocation under such constraints and whether the resulting representations occupy a more favorable cost-performance regime than reference learning rules. Methods: Representational cost is quantified using mutual-information-based measures derived from the Variational Information Bottleneck. Experiments use fixed audiovisual embeddings from three audiovisual benchmarks (AVE, Kinetics-Sounds, VGGSound100) to isolate downstream associative plasticity. Hebbian learning is compared with Dense Difference Target Propagation (DDTP) and backpropagation (BP) under matched sparsity and architectural constraints. Results: Hebbian learning achieves lower task-information cost (CTI) than sparse BP and DDTP in the main compressed comparisons, while reaching CTI values comparable to shallow BP with nonnegative weights. Rather than uniformly improving classification performance, Hebbian learning shifts the trade-off between task-relevant information and representational cost, yielding lower CTI at comparable functional performance in several settings. Discussion: The results indicate a cost-performance trade-off rather than uniform accuracy gains. For a given level of task-relevant information, Hebbian representations retain less input information while preserving functional performance, although accuracy is slightly reduced on some datasets. These findings support interpreting Hebbian learning as a mechanism for synaptic resource allocation rather than as a general strategy for maximizing audiovisual classification accuracy.

Problem

• representational allocation
• structural constraints
• synaptic resource allocation
• cost-performance trade-off
• neural coding

Innovation

• Constrained Hebbian Learning
• Representational Efficiency
• Information Bottleneck
• Synaptic Resource Allocation
• Task-Information Cost