Classifications in modular restricted Boltzmann machines

2026-10-06

AI Summary

This study addresses the learning stability and efficiency bottlenecks of modular Restricted Boltzmann Machines (RBMs) in orthogonal pattern classification and mixed-pattern decoupling. We construct a modular RBM comprising L Hopfield models coupled via Hebbian and anti-Hebbian mechanisms, leveraging competitive connections for pattern decoupling and employing one-step contrastive divergence for multi-label classification. Theoretically, we extend the Hopfield–RBM duality to the modular setting, proving that empirical mean weights constitute fixed points of the learning dynamics and deriving non-asymptotic residual drift bounds. Experimental results demonstrate that the proposed method achieves efficient classification of orthogonal patterns while simultaneously performing joint classification and decoupling on mixed patterns.

Abstract

We consider a modular associative neural network made of $L$ Hopfield models (HMs), coupled so that intra-module interactions are Hebbian and inter-module interactions are anti-Hebbian; this competitive coupling is known to endow the network with pattern-disentanglement capabilities. The integral representation of this system coincides with an assembly of $L$ restricted Boltzmann machines (RBMs) whose hidden layers are coupled, thereby extending the HM-RBM duality to the modular setting. We then train this modular RBM, via one-step contrastive divergence, to perform a classification task in which a query encoded on the visible layers is mapped onto an $L$-tuple of labels read off the hidden layers. When the query is composed of $L$ patterns that are mutually orthogonal on average, we prove that the RBM weights obtained as empirical means over the training dataset, as suggested by the HM-RBM equivalence, constitute a fixed point of the learning dynamics, and we derive an explicit, non-asymptotic bound on the residual drift. We then turn to a harder classification task in which each module is queried with a mixture of $L$ patterns and we show numerically that the same setting for the RBM weights still provides an effective set-up, letting the network jointly classify and disentangle the mixture.

Problem

• modular restricted Boltzmann machines
• classification
• pattern disentanglement
• Hopfield models
• contrastive divergence

Innovation

• Modular Restricted Boltzmann Machine
• Hopfield-RBM Duality
• Pattern Disentanglement
• Contrastive Divergence
• Non-asymptotic Bound