Understanding the geometry of deep learning with decision boundary volume

2026-03-15

AI Summary

This work proposes a computable geometric metric—local surface volume of decision boundaries—derived from differential geometry to quantify the geometric structure of decision boundaries in deep neural networks, thereby explaining their accuracy and generalization capabilities. For the first time, the Weyl tube formula is adapted to high-dimensional deep learning settings and validated on both convolutional and fully connected networks in image classification tasks. Experimental results demonstrate that, in convolutional networks, smaller boundary volumes—indicating smoother decision boundaries—are significantly correlated with higher classification accuracy, whereas fully connected networks exhibit stronger task-dependent behavior. This study establishes a geometric link between model complexity and performance, offering a novel perspective for understanding generalization in deep learning.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Learning & Optimization for CV
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

For classification tasks, the performance of a deep neural network is determined by the structure of its decision boundary, whose geometry directly affects essential properties of the model, including accuracy and robustness. Motivated by a classical tube formula due to Weyl, we introduce a method to measure the decision boundary of a neural network through local surface volumes, providing a theoretically justifiable and efficient measure enabling a geometric interpretation of the effectiveness of the model applicable to the high dimensional feature spaces considered in deep learning. A smaller surface volume is expected to correspond to lower model complexity and better generalisation. We verify, on a number of image processing tasks with convolutional architectures that decision boundary volume is inversely proportional to classification accuracy. Meanwhile, the relationship between local surface volume and generalisation for fully connected architecture is observed to be less stable between tasks. Therefore, for network architectures suited to a particular data structure, we demonstrate that smoother decision boundaries lead to better performance, as our intuition would suggest.

Problem

• decision boundary
• geometry
• deep learning
• generalization
• model complexity

Innovation

• decision boundary volume
• geometric interpretation
• Weyl tube formula
• model generalization
• neural network geometry