Heterophily-Aware Adaptive Knowledge Distillation for Hypergraph Neural Networks

2026-06-07

AI Summary

This work addresses the significant performance degradation of hypergraph neural networks on heterophilous nodes, which leads to unreliable and uneven teacher model outputs in knowledge distillation. To mitigate this issue, the authors propose HADES, a novel approach that, for the first time, incorporates node heterophily into hypergraph knowledge distillation. HADES dynamically quantifies node-level heterophily to assess the reliability of teacher-provided knowledge and adaptively modulates the strength of knowledge transfer accordingly. Extensive experiments demonstrate that HADES consistently outperforms baseline methods across multiple real-world hypergraph datasets, enabling lightweight student models to not only surpass standard distillation approaches but even exceed the performance of their teacher models, while achieving up to a 12.3× speedup in inference time.

Abstract

Hypergraph knowledge distillation aims to retain the predictive performance of a hypergraph neural network (HNN) teacher while reducing inference costs through a lightweight student model. In this work, we observe that HNNs exhibit substantially lower prediction performance on heterophilic nodes connected through semantically diverse hyperedges, indicating that the reliability of teacher knowledge varies across nodes. Motivated by this observation, we propose HADES, a heterophily-aware adaptive distillation method for hypergraph neural networks. HADES quantifies node heterophily and leverages it as an estimate of teacher reliability to modulate the transfer of teacher knowledge during distillation. Experimental results on real-world hypergraphs demonstrate that HADES consistently improves student performance across different HNN teachers and distillation objectives. In many cases, the resulting student models surpass the predictive performance of their teachers while achieving up to 12.3 times faster inference.

Problem

• heterophily
• hypergraph neural networks
• knowledge distillation
• teacher reliability
• node heterophily

Innovation

• heterophily-aware
• adaptive knowledge distillation
• hypergraph neural networks
• teacher reliability
• node heterophily