Training a Predictive Coding Network on ImageNet using Equilibrium Propagation

2026-06-02

AI Summary

This work addresses the limited scalability of Predictive Coding Networks (PCNs) and Equilibrium Propagation (EP) in large-scale vision tasks by introducing a novel hybrid approach that integrates key innovations from both frameworks. Specifically, it proposes an energy-based PCN architecture, a new equilibrium mechanism tailored for PCNs, and a centralized variant of EP. This combination enables, for the first time, the successful training of a 10-layer convolutional PCN (VGG10) on ImageNet. The method substantially improves scalability, achieving a Top-5 error rate of 13.23%—closely approaching the performance of standard backpropagation baselines (12.2%)—thereby demonstrating its effectiveness and practical viability for large-scale visual recognition.

Abstract

Equilibrium Propagation (EP) is a physics-based training framework that has primarily been employed in energy-based models, including continuous Hopfield networks, nonlinear resistive networks and coupled phase oscillators. However, EP's practical applications have so far remained limited to relatively small-scale problems. Predictive coding networks (PCNs), another class of energy-based models rooted in computational neuroscience, are typically trained with a specialized algorithm and have likewise not yet been demonstrated at large scale. In this work, we develop an EP-based training method for PCNs which combines the centered variant of EP with a novel equilibration scheme for PCNs. Using this approach, we train a 10-layer convolutional PCN (VGG10) on full-size ImageNet, achieving 13.23\% test error rate on the top-5 classification task, close to the 12.2\% backpropagation baseline. To our knowledge, this is the first demonstration of both PCNs and EP-based training at ImageNet scale. These results significantly extend the scalability of both approaches and suggest that the primary challenges in scaling EP in other physical systems may come more from the computational properties of these systems than from inherent limitations of the EP framework.

Problem

• Predictive Coding Networks
• Equilibrium Propagation
• ImageNet
• Scalability
• Energy-based Models

Innovation

• Equilibrium Propagation
• Predictive Coding Networks
• ImageNet-scale training
• energy-based models
• biologically plausible learning