Analyzing Neural Network Information Flow Using Differential Geometry

2026-01-22

AI Summary

This work proposes a novel approach rooted in graph theory and differential geometry to enable symbolic analysis of neural networks, such as robustness evaluation and model repair. The network architecture is modeled as a graph, and for the first time, Ollivier–Ricci curvature is introduced to define “Neural Curvature” (NC), quantifying the importance of edges in information flow. NC is computed dynamically based on input activations, revealing that edges with negative curvature act as functional bottlenecks, whereas those with positive curvature exhibit high redundancy. Experiments on MNIST, CIFAR-10, and CIFAR-100 demonstrate that removing negatively curved edges severely degrades performance, while pruning positively curved edges has minimal impact. Compared to existing methods, this framework more accurately identifies redundant connections, enabling highly effective structured pruning.

Technology Category

Machine Learning: Learning with ManifoldsComputer Vision: Adversarial Attacks & RobustnessNatural Language Processing: Safety and Robustness

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsWeb Mining and Content Analysis: Robustness and generalizability of Web mining methodsSocial Networks and Social Media: Social media analysis through the lenses of networks

Abstract

This paper provides a fresh view of the neural network (NN) data flow problem, i.e., identifying the NN connections that are most important for the performance of the full model, through the lens of graph theory. Understanding the NN data flow provides a tool for symbolic NN analysis, e.g.,~robustness analysis or model repair. Unlike the standard approach to NN data flow analysis, which is based on information theory, we employ the notion of graph curvature, specifically Ollivier-Ricci curvature (ORC). The ORC has been successfully used to identify important graph edges in various domains such as road traffic analysis, biological and social networks. In particular, edges with negative ORC are considered bottlenecks and as such are critical to the graph's overall connectivity, whereas positive-ORC edges are not essential. We use this intuition for the case of NNs as well: we 1)~construct a graph induced by the NN structure and introduce the notion of neural curvature (NC) based on the ORC; 2)~calculate curvatures based on activation patterns for a set of input examples; 3)~aim to demonstrate that NC can indeed be used to rank edges according to their importance for the overall NN functionality. We evaluate our method through pruning experiments and show that removing negative-ORC edges quickly degrades the overall NN performance, whereas positive-ORC edges have little impact. The proposed method is evaluated on a variety of models trained on three image datasets, namely MNIST, CIFAR-10 and CIFAR-100. The results indicate that our method can identify a larger number of unimportant edges as compared to state-of-the-art pruning methods.

Problem

• neural network
• information flow
• edge importance
• graph curvature
• Ollivier-Ricci curvature

Innovation

• Ollivier-Ricci curvature
• neural network pruning
• information flow analysis
• graph curvature
• neural curvature