AuDeRe: Automated Strategy Decision and Realization in Robot Planning and Control via LLMs

2025-04-03

AI Summary

To address rigid policy selection, heavy reliance on manual parameter tuning, and poor generalization in robotic planning and control, this paper proposes an LLM-driven automated policy decision-making and execution framework. The framework leverages task descriptions, environmental constraints, and system dynamics to achieve semantic–functional alignment from natural language to planning/control API calls via iterative prompt engineering and performance feedback—bypassing conventional end-to-end trajectory generation paradigms. Its core innovation is the first-ever policy-level LLM reasoning mechanism, enabling dynamic selection of optimal algorithms and autonomous policy refinement. Experiments across diverse spatiotemporally constrained robotic tasks demonstrate significantly reduced expert intervention, markedly improved autonomy, and consistent superiority over baseline methods where LLMs directly generate trajectories, control sequences, or executable code.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Behavior Learning & Control
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Recent advancements in large language models (LLMs) have shown significant promise in various domains, especially robotics. However, most prior LLM-based work in robotic applications either directly predicts waypoints or applies LLMs within fixed tool integration frameworks, offering limited flexibility in exploring and configuring solutions best suited to different tasks. In this work, we propose a framework that leverages LLMs to select appropriate planning and control strategies based on task descriptions, environmental constraints, and system dynamics. These strategies are then executed by calling the available comprehensive planning and control APIs. Our approach employs iterative LLM-based reasoning with performance feedback to refine the algorithm selection. We validate our approach through extensive experiments across tasks of varying complexity, from simple tracking to complex planning scenarios involving spatiotemporal constraints. The results demonstrate that using LLMs to determine planning and control strategies from natural language descriptions significantly enhances robotic autonomy while reducing the need for extensive manual tuning and expert knowledge. Furthermore, our framework maintains generalizability across different tasks and notably outperforms baseline methods that rely on LLMs for direct trajectory, control sequence, or code generation.

Problem

• Enhancing robotic autonomy via LLM-based strategy selection
• Reducing manual tuning needs in robot planning and control
• Improving task adaptability across varying complexity scenarios

Innovation

• LLM-based strategy selection for robotics
• Iterative reasoning with performance feedback
• Generalizable planning and control API integration