Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving

2026-07-31

AI Summary

This work addresses the limited interpretability and performance of end-to-end autonomous driving models in complex scenarios, as well as the reliance of existing vision-language reasoning approaches on costly and error-prone human annotations. The authors propose a teacher–student framework in which a teacher model generates and reflects upon reasoning rationales conditioned on actual driving actions, while a student model acquires this reasoning capability via knowledge distillation and employs a separate waypoint decoder to translate structured textual rationales into continuous trajectories. The method introduces a novel outcome-guided reflective reasoning mechanism that enhances zero-shot generalization without requiring intermediate human labels and explicitly leverages reasoning to guide driving predictions. Evaluated on the Waymo benchmark, the model significantly outperforms existing approaches in zero-shot reasoning, waypoint accuracy, and efficiency, achieving approximately a 24% improvement over baseline models without reasoning modules.

Abstract

End-to-end (E2E) autonomous driving aims to learn a direct mapping from visual observations to control actions. However, these E2E models often act as black boxes and struggle with complex scenarios. To address this, recent works incorporate Vision-Language Models (VLMs) to provide explicit reasoning, enhancing both interpretability and driving robustness. These approaches typically rely on pre-generated annotations, which suffer from potentially flawed labels and require costly human labor. In this work, we propose a new framework that integrates structured reasoning and geometric precision through a teacher-student architecture. The teacher model introduces reflective reasoning, where the VLM generates logical explanations and then reflectively refines the reasoning under the supervision of ground-truth action. This enhances zero-shot generalization without intermediate labels. The student model distills the teacher's reasoning capabilities via supervised fine-tuning. We also design a separate waypoint decoder that interprets textual reasoning into continuous trajectories. Our proposed solution integrates two goals: providing explicit reasoning for interpretability and delivering robust and accurate driving performance. It leverages the synergy between these two objectives within a staged inference engine to enhance driving performance and explicitly uses the reasoning to guide driving prediction. Evaluated on Waymo benchmarks, our framework outperforms classical reasoning-based baselines in zero-shot reasoning, waypoint accuracy, and inference efficiency. Our experiments validate this design, demonstrating that the reasoning text makes a significant contribution to driving inference, resulting in around a 24% improvement in performance compared to an identical model that lacks reasoning. Our work advances reasoning-driven autonomous driving toward interpretable and deployable systems.

Problem

• autonomous driving
• Vision-Language Models
• interpretability
• reasoning
• end-to-end learning

Innovation

• Outcome-Guided Distillation
• Reflective Reasoning
• Vision-Language Models
• Teacher-Student Framework
• Autonomous Driving