Universality and Approximation Rates of Graph Neural Networks with Random Features

2026-07-29

AI Summary

This work investigates the expressive power of graph neural networks, specifically their ability to approximate arbitrary permutation-invariant or equivariant functions. To this end, the authors propose Permutation-Equivariant Neural Networks with partial random node features (PENN) and establish, for the first time, a universal approximation theorem for such architectures: on fixed-size directed graphs, PENN can probabilistically approximate any measurable permutation-invariant or equivariant function to arbitrary precision. Moreover, for functions that are $k$-times continuously differentiable with $k \geq 2$, the study derives a quantitative upper bound on the approximation error in terms of network depth and the number of non-zero weights, thereby revealing an explicit relationship between model complexity and approximation rate.

Abstract

We investigate message-passing graph neural networks with random node features. Random node features are known to enhance the expressiveness of graph neural networks (GNNs) both theoretically and empirically. Here, we establish a novel universality result focusing on permutation-equivariant neural networks (PENNs), a class of GNNs built from feedforward neural network components that subsumes many prominent GNN architectures. We show that PENNs, combined with partially random node features, can approximate arbitrarily well in probability any measurable permutation-invariant or permutation-equivariant function on directed graphs of fixed size with multidimensional node and edge features. For $k$-times continuously differentiable functions, $k\geq 2$, we also derive upper bounds on the approximation rates, relating the complexity of the feedforward components of a PENN in terms of layer depth and number of nonzero weights to the desired approximation accuracy.

Problem

• Graph Neural Networks
• Random Features
• Universality
• Approximation Rates
• Permutation Equivariance

Innovation

• Graph Neural Networks
• Random Features
• Permutation Equivariance
• Universality
• Approximation Rates