A Knowledge-Driven Diffusion Policy for End-to-End Autonomous Driving Based on Expert Routing

2025-09-05

AI Summary

End-to-end autonomous driving faces three key challenges: multimodal action generation, insufficient temporal stability, and poor cross-scenario generalization. To address these, we propose a knowledge-driven diffusion policy framework that innovatively integrates conditional diffusion models with a sparse-gated Mixture-of-Experts (MoE) mechanism. The diffusion model captures multimodal, temporally coherent action distributions, while the MoE dynamically routes expert modules based on driving context, enabling interpretable and scalable knowledge composition. Departing from conventional deterministic policies, our generative approach enhances behavioral diversity and long-horizon consistency. Evaluated on multi-scenario benchmarks including CARLA, our method achieves significant improvements: +12.3% task success rate, −38.7% collision rate, and superior control smoothness and environmental adaptability—demonstrating both effectiveness and robustness.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Mixture of Experts (MoE)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

End-to-end autonomous driving remains constrained by the need to generate multi-modal actions, maintain temporal stability, and generalize across diverse scenarios. Existing methods often collapse multi-modality, struggle with long-horizon consistency, or lack modular adaptability. This paper presents KDP, a knowledge-driven diffusion policy that integrates generative diffusion modeling with a sparse mixture-of-experts routing mechanism. The diffusion component generates temporally coherent and multi-modal action sequences, while the expert routing mechanism activates specialized and reusable experts according to context, enabling modular knowledge composition. Extensive experiments across representative driving scenarios demonstrate that KDP achieves consistently higher success rates, reduced collision risk, and smoother control compared to prevailing paradigms. Ablation studies highlight the effectiveness of sparse expert activation and the Transformer backbone, and activation analyses reveal structured specialization and cross-scenario reuse of experts. These results establish diffusion with expert routing as a scalable and interpretable paradigm for knowledge-driven end-to-end autonomous driving.

Problem

• Generating multi-modal actions for autonomous driving
• Maintaining temporal stability in driving policies
• Generalizing across diverse driving scenarios

Innovation

• Diffusion policy for multi-modal action generation
• Expert routing mechanism for contextual adaptability
• Sparse mixture-of-experts enabling knowledge composition