PAC-Bayesian Adversarially Robust Generalization for Message Passing Graph Neural Networks: A Sensitivity Analysis

2026-06-04

AI Summary

This work addresses the lack of tight theoretical guarantees for the robust generalization of graph neural networks (GNNs) under adversarial attacks. By extending the sensitivity-aware PAC-Bayesian framework to message-passing GNNs, the authors introduce a sensitivity matrix based on the output Jacobian to characterize the impact of parameter perturbations on model outputs and construct an anisotropic Gaussian posterior distribution. This approach reduces the dominant dimension-dependent factor in the generalization bound from the hidden layer width to the number of classes \(K\), substantially tightening the bound. The resulting adversarially robust generalization bound provides a principled theoretical foundation for designing more robust graph neural networks.

Abstract

Whilst the vulnerability of graph neural networks (GNNs) to adversarial attacks poses a critical threat to graph representation learning, the understanding of the robust generalization behavior remains a fundamental challenge in the adversarial setting. Recently, PAC-Bayesian margin-based generalization analysis substantially advances this line of research by providing a flexible and data-dependent analytical framework. However, existing robust analyses often rely on isotropic Gaussian posteriors and control weight perturbations in the full parameter space, which limits the ability to capture heterogeneous parameter sensitivity yet hinges on hidden-width-dependent complexity terms, resulting in not-tight-enough generalization bounds. In this paper, we extend a recently proposed sensitivity-aware PAC-Bayesian framework from deep neural networks to message passing GNNs (MPGNNs) and derive a tighter robust generalization bound in the adversarial setting. Specifically, we first quantify how sensitive the perturbations across different parameter blocks are to the network outputs by deriving the output Jacobians with respect to the weight parameters. Exploiting the fact that these Jacobian matrices have rank at most $K$ in $K$-class graph classification, we then construct Jacobian-aligned sensitivity matrices and use anisotropic Gaussian posteriors with optimized covariances to upper bound the KL divergence in a tight way. Notably, by refining the spectral-norm dependence on the learned weights and reducing the leading dimension factor from hidden-width-dependent terms to the number of classes $K$, our analysis yields much tighter robust generalization guarantees for MPGNNs, thereby guiding their designs to enhance adversarial robustness.

Problem

• adversarially robust generalization
• graph neural networks
• PAC-Bayesian analysis
• parameter sensitivity
• generalization bounds

Innovation

• PAC-Bayesian
• adversarially robust generalization
• message passing GNNs
• sensitivity-aware analysis
• anisotropic Gaussian posterior