SpikingMoE: SDPrompt-Guided Dynamic Expert Fusion in Spiking Neural Networks

2026-05-21

AI Summary

This work addresses the lack of efficient dynamic routing mechanisms in spiking neural networks (SNNs) by introducing Mixture-of-Experts (MoE) into a spiking-driven Transformer architecture for the first time. Inspired by the lateral geniculate nucleus, the proposed SDPrompt mechanism enables input-dependent dynamic expert fusion while employing binary spike-based communication to ensure both biological plausibility and compatibility with neuromorphic hardware. Experimental results demonstrate that the model achieves Top-1 accuracies of 94.09% on CIFAR-10 and 74.54% on CIFAR-100, validating the effectiveness of dynamic expert routing in SNNs. Additionally, the authors release the first open-source SNN framework supporting this novel architecture.

Abstract

Spiking Neural Networks (SNNs) provide an energy-efficient paradigm for visual recognition. We present SpikingMoE, which integrates a spike-driven Transformer with a Mixture-of-Experts (MoE) framework for dynamic computation. Inspired by the lateral geniculate nucleus (LGN), a spike-driven prompt (SDprompt) enables input-dependent expert routing in a biologically plausible manner. By replacing standard MLPs with spike-compatible expert modules and enforcing binary spike communication, SpikingMoE is designed for neuromorphic hardware. Experiments on CIFAR-10 and CIFAR-100 achieve 94.09% and 74.54% top-1 accuracy, showing that modular expert routing can be incorporated while retaining reasonable performance. To our knowledge, SpikingMoE is the first open-source SNN framework that integrates MoE into a spike-driven Transformer with LGN-inspired routing.

Problem

• Spiking Neural Networks
• Mixture-of-Experts
• Dynamic Expert Fusion
• Visual Recognition
• Neuromorphic Hardware

Innovation

• Spiking Neural Networks
• Mixture-of-Experts
• Spike-driven Prompt
• Neuromorphic Computing
• Dynamic Routing