Align2Act: Instruction-Tuned Models for Human-Aligned Autonomous Driving

2025-10-12

AI Summary

To address the challenge of aligning large language model (LLM)-based driving decisions with human behavior in complex traffic scenarios, this paper proposes Align2ActChain—a novel framework that integrates instruction-tuned LLMs (LLaMA-2-7B fine-tuned via LoRA) with structured driving instructions, stepwise reasoning, and explicit traffic rule encoding to yield interpretable, human-aligned motion planning. The method is trained on the million-scale nuPlan dataset and generates multi-step action chains to jointly enforce semantic controllability and safety constraints. In closed-loop evaluation, Align2ActChain achieves non-reactive and reactive scores of 70.31 and 66.96, respectively, and an open-loop score of 85.17—substantially outperforming prior approaches. These results demonstrate its superior human similarity, safety compliance, and generalization capability in realistic, complex driving environments.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Motion planning in complex scenarios is a core challenge in autonomous driving. Conventional methods apply predefined rules or learn from driving data to generate trajectories, while recent approaches leverage large language models (LLMs) for decision-making. However, it remains unclear whether LLMs truly capture human driving logic. We propose Align2Act, a motion planning framework that transforms instruction-tuned LLMs into interpretable planners aligned with human behavior. We derive structured driving instructions based on human reasoning patterns (e.g., anticipate hazards, yield at intersections) and traffic rules (e.g., stop at red lights, maintain lane boundaries). Our Align2ActChain module guides step-by-step reasoning to produce both an interpretable rationale and a safe trajectory. By fine-tuning LLaMA-2-7B with LoRA on one million scenarios from the nuPlan dataset, our method achieves an open-loop score of 85.17 and closed-loop scores of 70.31 (non-reactive) and 66.96 (reactive) on Test14-random. Unlike prior work focused on synthetic or open-loop settings, we demonstrate improved planning quality and human-likeness on the real-world nuPlan closed-loop benchmark. Ablation studies confirm that structured reasoning significantly improves performance over baseline LLM planners.

Problem

• Developing human-aligned motion planning for autonomous vehicles
• Ensuring LLMs capture human driving logic in complex scenarios
• Creating interpretable planners that follow traffic rules safely

Innovation

• Uses instruction-tuned LLMs for human-aligned motion planning
• Implements structured reasoning chains for interpretable trajectory generation
• Fine-tunes LLaMA-2 with LoRA on real-world driving scenarios