Neuromorphic Diffusion Language Models: Addressing Compute and Memory Bottlenecks via Sparsity and Block Denoising

2026-07-24

AI Summary

This work addresses the computational inefficiency, high memory demand, and excessive energy consumption inherent in autoregressive large language models due to their sequential token-by-token generation. To overcome these limitations, the study introduces N-MDLM, the first diffusion-based language model integrated with neuromorphic computing. N-MDLM leverages a block diffusion mechanism to enable parallel multi-token generation and exploits the sparse activation properties of spiking neural networks to bypass inactive channels, thereby significantly enhancing throughput and energy efficiency. Through token-level roofline performance modeling, N-MDLM demonstrates superior performance over both conventional autoregressive LLMs and standard diffusion language models on translation tasks, maintaining robust effectiveness even under stringent computational constraints.

Abstract

Autoregressive (AR) large language models (LLMs) are inherently inefficient at inference time because each generated token requires accessing the full set of model parameters, leading to low operational intensity and high energy consumption. Masked diffusion language models (MDLMs) partially address this limitation for memory-bound settings by allowing multiple tokens to be generated per parameter access. In order to further enhance inference efficiency on modern platforms with extensive in-chip memory, this work proposes neuromorphic MDLMs (N-MDLMs), which integrate block diffusion with spike-based neuromorphic computation to jointly improve throughput and energy efficiency. While block diffusion increases token throughput by producing multiple tokens per parameter access, spike-induced sparsity reduces effective parameter traffic and computations by skipping inactive channels. To analyze the synergistic effect of sparsity and diffusion, we develop a token-level roofline-inspired model that captures the combined impact of block-parallel generation and spike sparsity on decoding efficiency. Experimental results on translation tasks show that, thanks to spike-induced sparsity, N-MDLMs achieve substantial improvements in energy efficiency and throughput even in compute-bound platforms for which MDLMs would fail to improve over AR-LLMs.

Problem

• compute bottleneck
• memory bottleneck
• inference efficiency
• large language models
• energy efficiency

Innovation

• neuromorphic computing
• diffusion language models
• sparsity
• block denoising
• energy efficiency