TreeIRL: Safe Urban Driving with Tree Search and Inverse Reinforcement Learning

2025-09-16

AI Summary

Addressing the challenge of simultaneously ensuring safety, efficiency, and human-like driving behavior in urban autonomous driving, this paper proposes a novel trajectory planning framework integrating Monte Carlo Tree Search (MCTS) and deep Inverse Reinforcement Learning (IRL). Methodologically, it is the first to apply MCTS to open-road autonomous driving planning, generating multiple safe candidate trajectories; subsequently, deep IRL infers human driving preferences from expert demonstrations to score and select trajectories based on human-likeness. The framework achieves joint optimization across safety, ride comfort, and behavioral similarity to human drivers. Extensive real-world validation—over 500 miles in Las Vegas urban environments—demonstrates robust performance, while large-scale simulation benchmarks show significant improvements over both conventional and state-of-the-art planners. The proposed approach establishes new performance benchmarks, achieving state-of-the-art (SOTA) results in comprehensive evaluation metrics.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

We present TreeIRL, a novel planner for autonomous driving that combines Monte Carlo tree search (MCTS) and inverse reinforcement learning (IRL) to achieve state-of-the-art performance in simulation and in real-world driving. The core idea is to use MCTS to find a promising set of safe candidate trajectories and a deep IRL scoring function to select the most human-like among them. We evaluate TreeIRL against both classical and state-of-the-art planners in large-scale simulations and on 500+ miles of real-world autonomous driving in the Las Vegas metropolitan area. Test scenarios include dense urban traffic, adaptive cruise control, cut-ins, and traffic lights. TreeIRL achieves the best overall performance, striking a balance between safety, progress, comfort, and human-likeness. To our knowledge, our work is the first demonstration of MCTS-based planning on public roads and underscores the importance of evaluating planners across a diverse set of metrics and in real-world environments. TreeIRL is highly extensible and could be further improved with reinforcement learning and imitation learning, providing a framework for exploring different combinations of classical and learning-based approaches to solve the planning bottleneck in autonomous driving.

Problem

• Safe autonomous driving in urban environments
• Combining tree search and inverse reinforcement learning
• Balancing safety, progress, comfort and human-likeness

Innovation

• Combines Monte Carlo tree search with inverse reinforcement learning
• Uses MCTS to generate safe candidate trajectories
• Deep IRL scoring selects most human-like trajectory