Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment

2026-05-28

AI Summary

This work addresses the lack of a theoretical foundation for existing broadcast-based credit assignment methods under general differentiable loss functions, which hinders their ability to serve as biologically plausible alternatives to backpropagation. The authors propose a novel broadcast credit assignment framework grounded in the orthogonality principle between output logits and hidden-layer activations. For the first time, they explicitly identify the broadcast signal as the loss gradient (score) and establish a unified theory applicable to a broad family of losses, including cross-entropy and Bregman divergences. By introducing a score vector expansion technique to enhance signal expressivity, the framework naturally yields a three-factor learning rule. Empirical evaluations on CIFAR-10 and Tiny ImageNet demonstrate substantial improvements over current broadcast-based methods, confirming both its effectiveness and generality.

Abstract

We introduce Score Broadcast and Decorrelation (SBD), a principled framework for broadcast-based credit assignment for general families of differentiable losses. Error broadcast is a biologically plausible alternative to backpropagation that sends output information to hidden layers without weight transport. The Error Broadcast and Decorrelation (EBD) framework, recently introduced for the mean-squared-error (MSE) setting, grounded this mechanism in the stochastic orthogonality of optimal estimators, under which the optimal residual is orthogonal to functions of the input. We generalize that foundation by introducing an orthogonality principle between the output score (the gradient of loss with respect to the final-layer output) and hidden-layer activations, which holds whenever the optimal score has conditional mean zero. This single principle unifies broadcast-based credit assignment across the standard differentiable-loss families, including cross-entropy, Bregman divergences, proper scoring rules, and exponential-family negative log-likelihoods. The framework supplies a theoretical grounding for the three-factor learning rule under general losses, with the neuromodulatory factor derived as the broadcast loss score. We derive the cross-entropy case explicitly, characterize the admissible loss class, and introduce a score vector expansion technique that enriches the broadcast signal while preserving the orthogonality framework. Experiments on CIFAR-10 and Tiny ImageNet show that SBD substantially improves over existing broadcast approaches, with score vector expansion delivering further gains. Overall, this work identifies the loss score as the signal to broadcast, supplies the orthogonality theory and theoretical grounding for the three-factor learning rule from neuroscience, and shows how score vector expansion enriches the decorrelation directions of the resulting objective.

Problem

• credit assignment
• error broadcast
• orthogonality principle
• differentiable losses
• biologically plausible learning

Innovation

• Score Broadcast
• Decorrelation
• Credit Assignment
• Orthogonality Principle
• Three-factor Learning Rule