When Design Rules Break: Benchmark Composition Determines Whether Label Informativeness Predicts GNN Aggregator Choice

2026-06-08

AI Summary

It remains unclear whether design principles for aggregator selection in graph neural networks (GNNs) are universally applicable, particularly whether label informativeness reliably guides aggregation strategy choice. This study systematically evaluates the performance of common aggregators—such as sum, mean, and max—across 24 node classification datasets, leveraging GIN and PNA architectures, stochastic block model ablations, and graph statistics including spectral gap and degree distribution. The findings reveal that prevailing design rules hold only on traditional sparse benchmarks and break down on dense social graphs like Facebook-100. Notably, the spectral gap effectively identifies graph structures with atypical aggregator preferences, and sum aggregation significantly outperforms mean by 7–13% on dense graphs. Moreover, PNA does not consistently surpass simpler aggregators on standard citation networks.

Abstract

We examine whether graph neural network (GNN) design rules generalize across benchmark families by studying aggregator selection (sum, mean, max) on 24 node-classification datasets spanning citation, heterophilic, LINKX Facebook-100, co-purchase, and co-authorship graphs. Edge homophily is only weakly predictive of the GIN-Sum versus GIN-Mean performance gap. Label informativeness predicts this gap well on legacy benchmarks but degrades substantially when Facebook-100 graphs are included. In these dense friendship networks, near-zero label informativeness coexists with a strong preference for sum aggregation, producing gains of 7-10% and up to 13% under extended training. Stochastic block model ablations, including degree-corrected variants matching Facebook-100 degree scales, fail to reproduce this behavior, indicating that mean degree alone does not explain the effect. Among several label-independent graph statistics, the spectral gap uniquely distinguishes these graphs from other low-informativeness datasets, with the effect localized to one-hop neighborhoods and replicated across architectures. We further identify training regimes that interact with aggregator choice and show that PNA can underperform the best single-aggregator GIN on standard citation benchmarks. Our results suggest that benchmark composition, rather than numerical insufficiency, determines whether design rules appear to generalize, and that the Facebook-100 regime provides a concrete target for future adaptive aggregation methods.

Problem

• graph neural networks
• aggregator selection
• benchmark generalization
• label informativeness
• Facebook-100

Innovation

• graph neural networks
• aggregator selection
• label informativeness
• spectral gap
• benchmark composition