message passing

This work investigates whether heterogeneous pre-trained vision models can spontaneously establish a shared referential protocol for cross-architecture, cross-paradigm semantic communication in an unsupervised setting. Method: We propose a multi-agent self-supervised referential game framework that integrates contrastive learning, feature alignment, and protocol distillation to enable collaborative construction of transferable high-level semantic representations. Contribution/Results: Experiments show that models achieve 89% inter-model referential accuracy; new agents rapidly adapt to the established protocol with minimal interaction; the learned protocol explicitly encodes object-level semantics and generalizes to unseen categories. To our knowledge, this is the first work demonstrating self-organized semantic communication across vision models trained under disparate paradigms—e.g., masked autoencoding, contrastive learning, and supervised pretraining—thereby establishing a novel paradigm for interpretable, cooperative neural network systems.

Graph neural networks (GNNs) suffer from catastrophic prediction failure under asynchronous inference in distributed multi-agent systems—severely limiting their deployment in resource-constrained, inherently desynchronized settings such as robot swarms and sensor networks. To address this, we first establish the first theoretical robustness guarantee for message-passing GNNs under partially asynchronous (Hogwild-style) inference. We then propose an energy-driven implicit GNN architecture: it formulates message aggregation via an energy function and solves representations through provably convergent fixed-point iteration. This design ensures both theoretical convergence and strong empirical performance. On multi-agent synthetic tasks, our method significantly outperforms existing implicit GNNs; on real-world graph benchmarks, it achieves competitive accuracy. Our work thus bridges theory and practice by providing formal robustness guarantees for asynchronous GNN inference while delivering state-of-the-art performance across diverse evaluation scenarios.

Deepening graph neural networks (GNNs) is hindered by oversmoothing and over-squashing—where the latter arises from inherent topological constraints on long-range information propagation and is often confounded with training dynamics (e.g., gradient vanishing), complicating rigorous evaluation of graph rewiring methods. This paper introduces the first training-free message-passing evaluation framework, explicitly decoupling over-squashing effects from optimization artifacts to isolate the intrinsic impact of topological rewiring. We systematically assess diverse rewiring strategies—including *k*-hop expansion and random edge addition—on node- and graph-level classification tasks. Leveraging parameter-free message passing and multi-scale graph structural analysis across multiple real-world datasets, our experiments reveal that most rewiring techniques yield no statistically significant performance gain. These findings challenge the widely held assumption of universal efficacy for rewiring and expose fundamental limitations in current rewiring paradigms.

This paper addresses three critical challenges in hypergraph learning: (i) the ambiguity of homophily definitions, (ii) architectural neglect of higher-order structural properties, and (iii) structural biases in prevailing benchmark datasets. To tackle these, we propose the first theoretical framework for higher-order homophily, formally defining and empirically validating it as a key determinant of hypergraph neural network (HNN) performance. We introduce a unified MultiSet message-passing paradigm and a novel architecture—MultiSetMixer—featuring hyperedge-aware node representations and joint node-hyperedge random sampling. Furthermore, we systematically expose fundamental structural distributional biases across mainstream benchmarks. Extensive experiments demonstrate that our approach achieves significant improvements over state-of-the-art methods across multiple benchmarks. The work establishes a theoretically grounded, interpretable, and scalable paradigm for hypergraph learning.

Graph Neural Networks (GNNs) suffer from the over-squashing problem during long-range information aggregation, where excessive neighborhood compression distorts message fidelity. Existing Expander Graph Propagation (EGP) methods mitigate this via truncated Cayley graphs, but such truncation inherently degrades the graph’s expansion properties. This paper proposes the first GNN framework performing *untruncated* message propagation on the *full* Cayley graph constructed from the group SL(2,ℤₙ), thereby strictly preserving optimal expander properties and enabling truly bottleneck-free aggregation. Our approach integrates group-theoretic graph construction, breadth-first message passing, and a GNN-layer-decoupled propagation mechanism that is agnostic to input graph topology. Experiments on multiple real-world graph benchmarks demonstrate that our method matches or surpasses state-of-the-art graph rewiring techniques in accuracy, significantly outperforms EGP, and incurs lower computational overhead.

This study systematically investigates the effectiveness boundaries and practical value of Graph Neural Networks (GNNs) across twelve application domains. Building upon a unified design space, it derives both spectral and spatial formulations of GNNs from first principles, analyzes their expressive power through the lens of the Weisfeiler–Leman test, and evaluates domain-specific graph construction strategies and architectural choices. The work establishes the first cross-domain analytical framework that disentangles genuine performance gains from baseline biases, uncovering common challenges such as heterophily, scaling effects, and deployment gaps. It clarifies the applicability limits of GNNs, highlights the discrepancy between leaderboard-topping models and deployable ones, and offers constraint-aware practical guidelines to address issues including oversmoothing, over-squashing, and distributional shifts.

This work addresses the limitation of existing graph neural networks, which rely on fixed message-passing schemes and struggle to adapt to diverse local graph structures. The authors propose an adaptive information flow control framework wherein each node is modeled as an agent governed by a shared, end-to-end trainable policy. Through a dynamic predict–act–observe–correct interaction loop, these agents modulate information propagation in a context-aware manner. The approach introduces a novel tripartite action mechanism—comprising source reception, channel selection, and gain-aware halting—to enable fine-grained control over message passing. Extensive experiments demonstrate consistent and significant improvements over baseline methods across node classification, graph classification, and cross-domain transfer tasks, highlighting the model’s strong generalization capability to complex and varied graph topologies.

This study investigates whether large language models rely on local pattern matching or infer latent graph structures during in-context learning. By designing random-walk tasks over two competing graph topologies and employing causal probing techniques—including PCA-based representation analysis, residual stream activation patching, and graph-difference linear steering—the work reveals that models concurrently encode both global topology and local transition information. The findings demonstrate that intermediate layers maintain orthogonal representations of the two graph structures, that late-layer interventions can transfer graph preferences across contexts, and that linear steering is both effective and highly specific. These results support a dual-mechanism account of in-context graph learning, wherein belief-like reasoning and inductive circuitry operate in parallel.

This work addresses rank collapse in message-passing neural networks (MPNNs), a generalization of the over-smoothing phenomenon, by identifying two underlying mechanisms: shared component amplification (SCA) and component dominance (CD). For the first time, the paper decomposes over-smoothing into these distinct factors and establishes a theoretical connection between MPNNs and personalized PageRank. To mitigate rank collapse without altering the backbone architecture, the authors propose a multi-relational splitting (MRS) framework and introduce multi-input multi-output graph convolution (MIMO-GC) along with its local variant, LMGC, which inject multi-relational structure into standard MPNNs. Furthermore, they design an infinitely deep MPNN variant grounded in personalized PageRank. The proposed methods effectively alleviate rank collapse, enable infinite-depth propagation, preserve initial features, and significantly enhance model expressiveness and stability.

This work reveals that MLP layers in Transformers implement a binary routing mechanism via binarized neuron activations to determine whether to apply nonlinear transformations to input tokens. Through neuron activation analysis, causal interventions, comparisons between binary and continuous features, and polynomial fitting, the study provides the first evidence of functional binary routing in MLPs and uncovers an architectural evolution from shallow single-gateway to deep consensus-based configurations, thereby extending the piecewise affine theory of deep networks. Experiments on GPT-2 Small demonstrate that when consensus is intact, removing MLPs increases perplexity by only 10.1%, whereas under collapse it surges by 43.3%; moreover, the binary decisions incur negligible information loss, achieving 79.2% accuracy compared to 78.8% with full continuous representations.