permutation-invariant representations

Prior theoretical analyses of graph neural networks (GNNs) have predominantly focused on graph-level expressivity, leaving link-level representation power largely unexplored. This work establishes the first systematic theoretical framework for link representation expressivity, introducing a unified $k_phi$-$k_ ho$-$m$ formal model family that characterizes hierarchical expressivity of mainstream GNNs in link prediction. We design the first synthetic evaluation protocol specifically tailored to link-level expressivity, uncovering the critical role of graph symmetry in predictive performance. Based on these insights, we propose a data-aware model selection principle. Theoretical analysis demonstrates that higher-order expressive models substantially outperform basic message-passing GNNs on highly symmetric graphs. By bridging a fundamental gap in the theoretical understanding of link representations, this work provides a novel paradigm for both GNN architecture design and rigorous empirical evaluation.

Existing neural network weight representation methods are constrained by architecture and scale, limiting generalization across heterogeneous architectures and datasets. This paper proposes the SNE encoder—the first approach to produce unified, set-level representations of neural networks regardless of architecture or parameter count, enabling cross-architecture and cross-dataset network property prediction. Our method introduces three key innovations: (1) a Logit Invariance constraint that jointly models computational hierarchy and weight-space symmetry; (2) a tunable, hierarchical encoding pipeline comprising padding, chunking, and encoding stages; and (3) formal definition and solution of the novel task of cross-dataset/cross-architecture prediction. Evaluated on standard benchmarks, SNE significantly outperforms existing baselines, demonstrating strong generalization capability and explicit architecture independence.

Existing node embedding methods suffer from two key limitations: vector addition lacks network semantic interpretability, and relationships among multi-scale (coarse-grained) embeddings remain ill-defined. This paper proposes a multi-scale node embedding framework that unifies the resolution of both issues for the first time. Leveraging a hierarchical coarse-graining mechanism grounded in renormalization theory, and imposing vector-sum constraints alongside low-dimensional reconstruction optimization in the embedding space, our method ensures that the embedding of any coarse-grained block node is strictly equal to the statistical mean of its constituent node embeddings. This guarantees statistical consistency across resolutions. Evaluated on international trade and input-output networks, the framework achieves high-fidelity structural reconstruction—e.g., accurate triangle counting—and supports arbitrary-scale graph generation. It significantly enhances interpretability and practicality in multi-scale graph modeling and synthesis.

Graph Neural Networks (GNNs) suffer from expressive limitations in modeling structural interactions within graphs. To address this, we introduce a novel perspective—permutation-invariant graph partitioning—and establish its first theoretical connection to graph isomorphism, revealing a fundamental trade-off between partitioning strategies and GNN expressivity. Building on this insight, we propose Graph Partitioning Neural Networks (GPNNs), which depart from conventional message-passing paradigms by explicitly capturing inter-subgraph structural dependencies. GPNNs incorporate a differentiable graph clustering module that ensures both permutation invariance and computational efficiency. Extensive experiments across multiple graph benchmark tasks demonstrate that GPNNs consistently outperform state-of-the-art GNNs. Notably, on structural interaction identification tasks, GPNNs achieve average accuracy gains of 5.2%–9.7%, validating their ability to jointly enhance expressive power and computational efficiency.

This work addresses fully inductive knowledge graph link prediction—requiring zero-shot generalization to both entirely unseen entities and novel relation types at inference time—a setting where existing methods exhibit insufficient cross-dataset generalization. Method: We introduce and formally define the “double permutation-equivariant representation” framework, proving it a necessary condition for this task; we unify the modeling of double equivariance in GNN architectures via group actions and representation theory, integrating theoretical derivation with empirical validation. Contribution/Results: Our analysis reveals a fundamental limitation of double equivariance in cross-domain meta-learning, demonstrating its inadequacy for universal knowledge graph foundation models; we explicitly identify the critical theoretical gap preventing the realization of such cross-domain foundation models—namely, the absence of a principled mechanism for transferring relational abstractions across heterogeneous schema and entity distributions.

This work addresses the limited expressivity of graph neural networks (GNNs), which are constrained by the 1-Weisfeiler-Lehman (1-WL) test and thus unable to distinguish graph structures beyond degree sequences or capture nodes’ structural roles in higher-order interactions. To overcome this, we propose the ISP-WL test and its neural instantiation, ISPGNN, which for the first time integrates a hierarchy based on graph invariants into message passing. By hierarchically encoding structural heterogeneity, ISPGNN explicitly models nodes’ higher-order roles across multiple levels, thereby surpassing the expressiveness of 1-WL. The method exhibits strong resistance to oversmoothing and flexible structure-aware capabilities, achieving significant performance gains over existing GNNs and high-expressivity models on graph classification, node classification, and influence estimation tasks, demonstrating its effectiveness and generalization ability.

Existing graph benchmarks struggle to disentangle whether model performance stems from node features or graph structure, thereby obscuring the true utilization of relational information. This work introduces, for the first time, graph invariants as systematic diagnostic tools, constructing task-agnostic, permutation-invariant structural descriptors to build non-trainable structural proxy models. Experiments across 26 datasets demonstrate that this simple proxy matches or even surpasses state-of-the-art GNN and Transformer baselines on structure-sensitive tasks, revealing the substantial yet previously underappreciated role of structural information in current benchmarks. These findings challenge the prevailing expressivity-centric evaluation paradigm and offer new perspectives for multitask performance prediction and structural heterogeneity analysis.

This study investigates the evolution of topological structures in learned representations within Predictive Coding Networks (PCNs) and their relationship to model capacity and reconstruction capability. For the first time, persistent homology is quantitatively applied to PCNs, enabling a layer-wise topological analysis that reveals the compression–reconstruction trade-off mechanism. The findings demonstrate that low-capacity models collapse connected components earlier in the network hierarchy, and that delayed topological simplification correlates with lower reconstruction error. Notably, PCNs exhibit topological simplification on average 3.6 layers later than comparably sized multilayer perceptrons (MLPs), highlighting their superior representational preservation. The methodology integrates persistent homology, Spearman correlation tests, and seed-level bootstrap comparisons to systematically characterize how architecture and activation functions influence inter-layer topological dynamics.