Track-Guided Hierarchical Reinforcement Learning for Autonomous Vehicle Drifting with Minimum-Lap-Time Planning

2026-07-31

AI Summary

This work addresses the challenge of simultaneously achieving dynamic drift stability and minimum lap time in autonomous rally racing by proposing a track-guided reinforcement learning (TgRL) approach. The method employs a planning–control co-design framework: it first generates a minimum-time drift trajectory via optimal control as a prior, then implements a staged, progressive training strategy to enable hierarchical policy learning—from local drift stabilization to global race-line optimization. The reward function integrates both immediate control performance and ultimate lap-time objectives. Simulation results demonstrate that the proposed method significantly reduces lap time while maintaining high nonlinear drift stability, thereby validating its effectiveness and superiority.

Abstract

In Formula 1, drivers optimize racing lines within tire grip limits to minimize lap times; however, in rally racing, drivers intentionally break traction to drift on loose surfaces. This maneuver rapidly aligns the vehicle for corner exits, ultimately reducing lap time. Autonomously executing such maneuvers formulates a complex dual-objective control problem: stabilizing highly nonlinear drift dynamics while strictly minimizing lap time. Addressing this challenge motivates the development of advanced Minimum-Lap-Time (MLT) drift control architectures. This paper proposes a planning-control framework specifically designed for MLT drifting scenario. First, we formulate an optimal control problem to generate a MLT drift planning trajectory, which is used as prior data to train a deep reinforcement learning drift controller. Given that drifting involves extremely large sideslip angles and is therefore challenging to learn directly, a Track-guided Reinforcement Learning (TgRL) drift control method is proposed to enable progressive training in a step-by-step manner, from drift control policy, to drift corner policy, and finally to a comprehensive drift race policy. The reward function incorporates both an instant reward term and an end reward term derived from the Minimum-Lap-Time objective. Simulation results demonstrate that the proposed framework enables the agent to learn a drift racing policy that not only ensures vehicle motion control performance but also effectively reduces lap time.

Problem

• autonomous vehicle drifting
• minimum-lap-time
• reinforcement learning
• drift control
• optimal racing line

Innovation

• Minimum-Lap-Time (MLT)
• drifting control
• Track-guided Reinforcement Learning (TgRL)
• hierarchical reinforcement learning
• autonomous racing