MDTransformer: A Hardware-Software Co-Design of Mode-Division Photonic Transformer Accelerator with Inverse-Designed Coherent Crossbar

2026-07-28

AI Summary

This work addresses the limitations of existing photonic Transformer accelerators, which rely on multi-wavelength sources and active phase modulators, resulting in poor energy efficiency and large footprint. The authors propose MDTransformer, a novel architecture that leverages mode-division multiplexed optical data streams and hardware-software co-design to perform complex-valued matrix multiplication entirely in the optical domain. Its key innovation lies in integrating inverse-designed multimode couplers with IQ modulation within a compact photonic tensor core, enabling four parallel complex arithmetic channels using TE₀–TE₃ guided modes in a single waveguide—without requiring spectral filtering—and supporting continuous-wave operation from a single laser. Compared to state-of-the-art approaches, MDTransformer reduces area by 40.4%, power consumption by 63.6%, and energy per operation by 40.6%, while maintaining comparable latency, and demonstrates high efficiency on DeiT and BERT models.

Abstract

Recently, photonic transformer accelerators (PTAs) have successfully achieved significant speedup and energy efficiency improvements over electronic accelerators for expediting Transformer inference. However, state-of-the-art rely on expensive multi-wavelength light generation and large dot-product units due to active phase-shifter components, thus making their approach inefficient and impractical. To address this, we propose MDTransformer, a novel hardware-software co-design of PTA based on mode-division optical dataflow and operations. Specifically, MDTransformer performs complex matrix operations using spatial-mode interference, that leverages the inverse-designed multi-mode couplers, crossings, and Mach-Zehnder IQ modulators into a compact mode-division photonic tensor core (MPTC), capable of executing matrix multiplications in the optical domain. Its each guided mode (i.e., TE0-TE3) acts as an independent computational lane, enabling four-fold parallelism-per-waveguide without spectral filtering or free-spectral-range limitations. Moreover, its coherent detection and IQ modulation jointly encode amplitude and phase, realizing complex-valued arithmetic for full-range operations in transformers. MDTransformer offers analog multiplication with sub-4-bit effective precision and inter-modal crosstalk below -30 dB. Its inverse-designed approach also offers scalable and full compatibility with single-laser continuous-wave operation at 1550 nm. Experimental results show that MDTransformer achieves 40.4% area reduction, 63.6% power saving, 40.6% energy saving, and comparable latency over the state-of-the-art PTA across different workloads (i.e., DeiT-Tiny/Small/Base and BERT-Base/Large). These results show that MDTransformer offers a practical solution for high-performance and energy-efficient transformer-based systems.

Problem

• photonic transformer accelerator
• multi-wavelength light generation
• active phase-shifters
• energy efficiency
• hardware impracticality

Innovation

• mode-division multiplexing
• photonic transformer accelerator
• inverse-designed photonics
• coherent crossbar
• hardware-software co-design