Bilevel Graph Structure Learning, Revisited: Inner-Channel Origins of the Reported Gain

2026-05-08

AI Summary

This work investigates whether performance gains in bilevel graph structure learning stem from graph rewiring or inner-loop training dynamics. To disentangle these contributions, the authors propose a frozen-φ control method that freezes the graph topology while preserving the original inner-loop training schedule, thereby isolating the effects of rewiring and inner-loop dynamics. Combining graph distillation with spectral analysis, they demonstrate that inner-loop dynamics predominantly drive performance improvements—accounting for 78–101% of the gain in spatiotemporal flow prediction and 37–44% in node classification. Furthermore, they establish a three-premise framework capable of predicting the sign of bilevel gains and prove that classical spectral metrics can be decoupled from task-specific performance improvements.

Abstract

Bilevel graph structure learning is widely understood to improve graph neural networks by jointly optimizing model parameters and a learned graph structure, with the resulting performance gain attributed to the rewired adjacency. We find that this attribution may be overstated: training-dynamics effects in the inner loop, rather than the rewiring itself, capture a substantial share of the gain. To establish this, we introduce frozen-$φ$, a control that freezes the graph while retaining the inner-loop training schedule. This decomposes the bilevel gain into an inner channel of $T$-step training dynamics with implicit gradient regularization and a graph channel of the graph rewiring itself. On spatio-temporal flow forecasting the inner channel matches or exceeds the full bilevel pipeline, accounting for 78-101% of the gain; on node classification it accounts for 37-44% under a Bernoulli edge-level parameterization. We also verify that classical spectral diagnostics can dissociate from task gain. We propose frozen-$φ$ as a standardized diagnostic for bilevel graph structure learning, with graph distillation as a method-agnostic complement. A three-precondition framework further predicts the sign of the bilevel gain on all six benchmarks.

Problem

• bilevel graph structure learning
• graph neural networks
• training dynamics
• graph rewiring
• performance gain attribution

Innovation

• bilevel graph structure learning
• training dynamics
• frozen-phi
• implicit gradient regularization
• graph rewiring