Problem
Research questions and friction points this paper is trying to address.
End-to-end autonomous driving
Spiking neural networks
Trajectory planning
Energy efficiency
Edge deployment
SDPAD: A Fully Spike-Driven Pipeline for End-to-End Autonomous Driving
🤖 AI Summary
This study addresses the significant accuracy gap between spiking neural networks (SNNs) and artificial neural networks (ANNs) in end-to-end autonomous driving planning, alongside the challenge of maintaining edge energy efficiency. We propose the first fully spike-driven end-to-end autonomous driving planner. Methodologically, we construct a single feedforward inference pipeline that eliminates temporal simulation loops by integrating quantized ANN-to-SNN conversion, Spike Maximum Depth Distribution (Spike-3D-Lift), Spike-QFormer, and deformable spike cross-attention mechanisms. Experimental results on the nuScenes dataset demonstrate that our model achieves an average L2 error of 0.40m, a collision rate of 0.12%, and an energy consumption of only 69.9mJ, while attaining a NAVSIM PDMS score of 86.3. These findings comprehensively surpass existing SNN baselines, validating the potential of SNNs to match dense ANN performance in complex driving tasks.
📝 Abstract
End-to-end autonomous driving demands trajectory planners that are both highly accurate and cheap enough for edge deployment. State-of-the-art artificial neural network (ANN) planners meet the accuracy requirement at the cost of heavy dense computation, while spiking neural networks (SNNs)---though promising orders-of-magnitude energy savings through sparse, event-driven arithmetic---still lag far behind in planning accuracy. We present \textbf{SDPAD}, a fully spike-driven end-to-end planning pipeline that closes this gap. SDPAD converts a pre-trained ANN perception stack into integer-spike form via quantized ANN2SNN conversion, lifts multi-view images into the bird's-eye-view (BEV) space with a spike-driven-max (SDM) depth distribution (Spike-3D-Lift), and plans through the Spike-QFormer, a spiking query transformer in which ego, agent, and map queries distilled from the BEV scene are fused by learnable waypoint queries via cross-attention, followed by deformable spike-cross-attention refinement. Every operation is gated by integer spikes and inference is a single feed-forward pass without temporal simulation loops. On the nuScenes open-loop benchmark, SDPAD achieves an average $L_2$ error of 0.40\,m and a collision rate of 0.12\%, on par with strong ANN planners while consuming 69.9\,mJ---less than 2\% of recent ANN baselines. In closed-loop evaluation on the NAVSIM navtest split, SDPAD reaches 86.3 PDMS, surpassing the previous SNN planner SAD by 4.3 points and matching mainstream ANN planners at a fraction of their energy. To our knowledge, SDPAD is the first fully spike-driven planner evaluated in end-to-end autonomous driving, demonstrating that SNNs can rival dense ANNs in complex driving tasks.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Spiking Neural Networks
End-to-End Autonomous Driving
ANN2SNN Conversion
Spike-QFormer
Bird's-Eye-View
🔎 Similar Papers
No similar papers found.
