Optimal Reward Shaping: Autonomous Car Parking Case Study

2026-07-26

AI Summary

This work addresses the challenges in model-free reinforcement learning under nonholonomic constraints, where reward function design is difficult and policies often suffer from paralysis or excessive conservatism, leading to suboptimal solutions. To overcome these issues, the paper proposes a parameterized reward shaping framework that integrates coverage-gated alignment feedback, driving-direction-switching regularization, and an aligned episode termination mechanism. A novel joint meta-optimization scheme is introduced to simultaneously optimize environmental reward parameters and algorithmic hyperparameters, revealing their deep coupling and, for the first time, applying this approach to automatic parking tasks. By combining surrogate-based Bayesian optimization with deep Q-networks, the method significantly outperforms uncalibrated baselines in both success rate and trajectory smoothness.

Abstract

Designing effective reward functions for model-free reinforcement learning under non-holonomic constraints remains a persistent challenge, often resulting in severe local minima such as policy paralysis or over-conservative hazard avoidance. In this work, we present a parameterized reward shaping framework featuring coverage-gated alignment feedback, drive-direction switch regularization, and an aligned episode termination mechanism evaluated on an autonomous parallel parking task. Crucially, we show that environmental reward parameters and algorithmic hyperparameters are deeply co-dependent, requiring joint meta-optimization to achieve stable convergence. By employing surrogate-based Bayesian optimization, our co-optimized Deep Q-Network (DQN) agent resolves characteristic control failure modes, significantly outperforming uncalibrated baselines across both success rate and trajectory smoothness.

Problem

• reward shaping
• reinforcement learning
• non-holonomic constraints
• local minima
• autonomous parking

Innovation

• reward shaping
• Bayesian optimization
• non-holonomic constraints
• Deep Q-Network
• meta-optimization