🤖 AI Summary
This work addresses the limited robustness of existing wireless foundation models in real-world environments, where performance degrades under noise and interference. To overcome this, the study introduces spiking neural networks (SNNs) into wireless foundation modeling for the first time, integrating the temporal sparsity and event-driven dynamics of SNNs with an ANN-based Transformer architecture. The resulting model achieves superior noise resilience, strong generalization, and enhanced energy efficiency. Through self-supervised pretraining on diverse wireless datasets, it learns transferable representations that significantly outperform conventional ANN-based approaches in tasks such as channel prediction, while also accelerating pretraining convergence.
📝 Abstract
This paper proposes SpikeWFM, a novel hybrid architecture that integrates spiking neural networks (SNNs) with conventional artificial neural network (ANN)-based transformers for wireless foundation models (WFMs). Inspired by the noise-robust and energy-efficient information processing in the human brain, SpikeWFM aims to enhance the resilience of WFMs against noise and interference while maintaining strong generalization capabilities across diverse wireless scenarios. Drawing from the success of large language models, WFMs leverage self-supervised pre-training on large-scale datasets spanning various wireless environments to learn a unified embedding that supports a wide range of downstream tasks, including channel prediction, channel estimation, beam predition, positioning and etc. Such models typically outperform task-specific designs and exhibit superior adaptability to unseen conditions. However, existing WFMs remain vulnerable to realistic noise and interference in practical wireless systems. To address this limitation, we incorporate spiking neurons into the transformer-based WFM architecture. We provide a brief theoretical analysis demonstrating how the SNN-ANN hybrid effectively mitigates noise and interference through temporal sparsity and event-driven processing. Experimental results show that SpikeWFM consistently outperforms conventional ANN-based WFMs in both pre-training convergence and channel prediction accuracy. Additional results on communication and sensing tasks will be presented in the full journal version of this work.