Physics-Informed Reinforcement Learning of Spatial Density Velocity Potentials for Map-Free Racing

2026-04-10

AI Summary

This work addresses the challenge of autonomous racing without pre-built maps, requiring motion planning solely from instantaneous sensor data while operating at the limits of tire friction and acceleration and generalizing to unseen tracks. The authors propose an end-to-end deep reinforcement learning approach that parameterizes nonlinear vehicle dynamics in spectral spatial density space and introduces a physics-informed reward function employing implicit value truncation instead of explicit collision penalties to enhance sim-to-real transfer. The network architecture features a high-resolution first layer to extract corner apex features and a second layer encoding dynamic driving behaviors. Experiments demonstrate that the method outperforms human drivers by 12% on out-of-distribution tracks, achieves computational overhead less than 1% of that required by behavioral cloning and model-based deep reinforcement learning, and efficiently approaches the theoretical tire friction circle limit.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Learning to Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: The sharing economy

Abstract

Autonomous racing without prebuilt maps is a grand challenge for embedded robotics that requires kinodynamic planning from instantaneous sensor data at the acceleration and tire friction limits. Out-Of-Distribution (OOD) generalization to various racetrack configurations utilizes Machine Learning (ML) to encode the mathematical relation between sensor data and vehicle actuation for end-to-end control, with implicit localization. These comprise Behavioral Cloning (BC) that is capped to human reaction times and Deep Reinforcement Learning (DRL) which requires large-scale collisions for comprehensive training that can be infeasible without simulation but is arduous to transfer to reality, thus exhibiting greater performance than BC in simulation, but actuation instability on hardware. This paper presents a DRL method that parameterizes nonlinear vehicle dynamics from the spectral distribution of depth measurements with a non-geometric, physics-informed reward, to infer vehicle time-optimal and overtaking racing controls with an Artificial Neural Network (ANN) that utilizes less than 1% of the computation of BC and model-based DRL. Slaloming from simulation to reality transfer and variance-induced conservatism are eliminated with the combination of a physics engine exploit-aware reward and the replacement of an explicit collision penalty with an implicit truncation of the value horizon. The policy outperforms human demonstrations by 12% in OOD tracks on proportionally scaled hardware, by maximizing the friction circle with tire dynamics that resemble an empirical Pacejka tire model. System identification illuminates a functional bifurcation where the first layer compresses spatial observations to extract digitized track features with higher resolution in corner apexes, and the second encodes nonlinear dynamics.

Problem

• map-free racing
• out-of-distribution generalization
• end-to-end control
• kinodynamic planning
• autonomous racing

Innovation

• Physics-Informed Reinforcement Learning
• Map-Free Autonomous Racing
• Spectral Density Representation
• Implicit Collision Avoidance
• Friction Circle Optimization