Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs

2026-08-03

AI Summary

This work addresses a critical flaw in existing vision-language models for autonomous driving: the exposure to ground-truth future trajectories during training induces anchoring bias, which compromises causal reasoning and leads to hallucinations. To mitigate this, the authors propose DEFT-RLVR, a novel approach that reframes planning as a multiple-choice decision within the AD-MCQ framework, where the model selects from explicit trajectory candidates of controllable difficulty. Crucially, the true trajectory is revealed only after inference for validation, thereby transforming future trajectories from pre-decision anchors into post-decision verification targets. This mechanism effectively eliminates anchoring bias while preserving or even enhancing general visual capabilities. Experimental results demonstrate that DEFT-RLVR significantly improves the causal fidelity and reliability of driving reasoning without sacrificing performance on standard vision tasks.

Abstract

Recent Vision-Language-Action (VLA) models for autonomous driving (AD) increasingly utilize chain-of-thought (CoT) supervision to enhance the reasoning capabilities of their Vision-Language Model (VLM) components, yet existing annotation pipelines commonly expose the teacher model to the logged ground-truth (GT) future trajectory. We empirically show that this induces trajectory anchoring bias: teacher models rationalize the revealed outcome rather than infer a decision from scene evidence, producing less causally faithful CoTs and substantially more severe hallucinations, especially in causally challenging scenes. Removing the GT trajectory eliminates this shortcut, but open-ended trajectory generation entangles high-level decision-making with precise geometric synthesis and low-level dynamics. To make trajectory-level driving decisions verifiable without requiring open-ended trajectory synthesis, we introduce Autonomous-Driving Multiple-Choice Question (AD-MCQ), which casts planning as selection among explicit trajectory candidates. Taking this a step further, we propose Deferred Exposure of Future Trajectories for RLVR (DEFT-RLVR) to transform future trajectories from pre-decision anchors into post-decision verification targets. Experimental results show that DEFT-RLVR improves AD reasoning while preserving or even enhancing general visual capabilities. With VLM-only inference and controllable difficulty through candidate construction, AD-MCQ provides a flexible, scalable, and extensible foundation for future research on verifiable AD reasoning.

Problem

• trajectory anchoring bias
• verifiable reasoning
• autonomous driving
• chain-of-thought
• hallucination

Innovation

• trajectory anchoring bias
• verifiable reasoning
• multiple-choice planning
• deferred exposure
• Vision-Language-Action models