Analog-Friendly Predictive Coding without Activation Derivatives

2026-09-26

AI Summary

Standard nonlinear predictive coding relies on activation function derivatives, hindering efficient deployment on analog neuromorphic hardware. This work proposes a predictive coding framework based on matching Bregman divergences, which reformulates inference and learning via mirror descent into operations requiring only local weighted summations and state integrations, thereby fundamentally eliminating the need for activation derivatives. As the first derivative-free nonlinear predictive coding scheme, it achieves performance comparable to standard predictive coding and backpropagation in digital experiments while fully preserving its intrinsic dynamical properties. This framework establishes a novel paradigm for implementing efficient online learning on brain-inspired analog hardware.

Abstract

Predictive coding (PC) is a local, energy-based alternative to backpropagation (BP) whose iterative inference dynamics make it attractive for implementation on analog hardware. However, standard nonlinear PC requires evaluating the derivative of the activation function during both inference and learning, which can be difficult to realise physically. Here, we introduce \textit{activation-matched Bregman PC}, replacing standard squared-error energies with Bregman divergences matched to the activation function. This formulation eliminates activation derivatives and, when combined with inference via mirror descent, yields local inference and learning rules requiring only weighted sums, local prediction errors, state integration, and the activation function. In digital experiments, Bregman PC performs comparably to standard PC and BP on classification and generative tasks, while preserving characteristic learning dynamics of PC and its convergence to BP under stable large-model parameterisations. These results provide a more analog-friendly formulation of nonlinear PC while retaining its key computational properties.

Problem

• Predictive Coding
• Analog Hardware
• Activation Derivatives
• Backpropagation

Innovation

• Predictive Coding
• Bregman Divergence
• Analog Hardware
• Mirror Descent
• Activation Derivative-Free