Continual-RL for Generalization in Autonomous Racing on the RoboRacer Platform

2026-07-27

AI Summary

This work addresses the challenge of poor generalization in autonomous racing when encountering novel track and surface conditions due to insufficient simulation coverage. To overcome this limitation, the authors propose a continual reinforcement learning framework based on continual backpropagation that trains a universal driving policy exclusively from real-world data. This approach represents the first implementation of purely real-data-driven continual reinforcement learning on the RoboRacer platform, further enhanced by offline reinforcement learning for policy fine-tuning and plasticity analysis. Experimental results demonstrate that the learned policy rapidly adapts to new scenarios within 15 minutes, significantly outperforming classical controllers and thereby validating its strong generalization capability and rapid adaptability in real-world environments.

Abstract

A key challenge in modern robotics is to adapt to changing environments, a challenge that is exacerbated when simulations cannot encompass every possible real-world configuration, and therefore Reinforcement Learning (RL) in the physical world becomes necessary. Continual Reinforcement Learning provides the tools to address this challenge; however, both the frameworks and the methods remain underexplored. Autonomous Racing and in particular the RoboRacer competition provide a testing ground for such methods, as learning to drive on a new track-floor combination with the least amount of new experience naturally frames a continual learning problem. This work tries to address this gap by proposing a continual RL framework based on Continual Backpropagation that is able, with only real-world data, to train a generalistic policy on a set of tracks and then fine- tune it within 15 minutes to outperform classical controllers. Furthermore, a comparison method based on offline RL is proposed, and a simulation analysis of the plasticity properties of the methods is conducted.

Problem

• Continual Reinforcement Learning
• Autonomous Racing
• Generalization
• Real-world Adaptation
• RoboRacer

Innovation

• Continual Reinforcement Learning
• Continual Backpropagation
• Autonomous Racing
• Real-world Adaptation
• Offline RL