Topological Simplification in Predictive Coding Networks

2026-08-03

AI Summary

This study investigates the evolution of topological structures in learned representations within Predictive Coding Networks (PCNs) and their relationship to model capacity and reconstruction capability. For the first time, persistent homology is quantitatively applied to PCNs, enabling a layer-wise topological analysis that reveals the compression–reconstruction trade-off mechanism. The findings demonstrate that low-capacity models collapse connected components earlier in the network hierarchy, and that delayed topological simplification correlates with lower reconstruction error. Notably, PCNs exhibit topological simplification on average 3.6 layers later than comparably sized multilayer perceptrons (MLPs), highlighting their superior representational preservation. The methodology integrates persistent homology, Spearman correlation tests, and seed-level bootstrap comparisons to systematically characterize how architecture and activation functions influence inter-layer topological dynamics.

Abstract

We study the topology of learned representations in predictive coding networks (PCNs), a neuro-inspired bidirectional architecture, using a quantitative layer-wise persistent homology analysis. We train well-performing PCNs on a synthetic classification dataset ($\geq 99.9\%$ test accuracy) and on MNIST ($\geq 95\%$ test accuracy), and measure how topological features change across layers for different architectures and activation functions. We find that smaller PCNs collapse connected components across layers earlier than larger models (Spearman $\unicode{x1D70C} \in [0.72, 0.79]$ across activations), with model size measured as the sum of hidden-layer widths. We also observe a strong negative correlation ($\unicode{x1D70C} = -0.58$) between the depth at which simplification occurs and reconstruction error; i.e., architectures that simplify later reconstruct better. Finally, a seed-level bootstrap comparison across architectures and activations shows that PCNs consistently collapse connected components later than matched MLPs, with an average difference of $3.6$ layers. These results suggest that persistent homology offers a useful quantitative lens on the compression--reconstruction tradeoff in PCNs, and that both model capacity and the recurrent, bidirectional dynamics of predictive coding inference shape when this tradeoff is resolved across layers.

Problem

• predictive coding networks
• topological simplification
• persistent homology
• representation topology
• reconstruction error

Innovation

• predictive coding networks
• persistent homology
• topological simplification
• bidirectional dynamics
• representation topology