AD-VF: LLM-Automatic Differentiation Enables Fine-Tuning-Free Robot Planning from Formal Methods Feedback

2025-09-22

AI Summary

To address safety and compliance risks in robotic planning arising from LLM hallucinations or misalignment, this paper proposes LAD-VF: a fine-tuning-free, annotation-free framework that enables automated, auditable prompt optimization via LLM-AutoDiff—integrating formal verification feedback with text-based loss functions. Safety constraints are encoded as verifiable logical specifications; gradient-guided prompt iteration then systematically improves instruction-following fidelity. Evaluated on navigation and manipulation tasks, LAD-VF increases specification compliance from 60% to over 90%, significantly enhancing reliability, interpretability, and modular interoperability. By eliminating dependence on human-labeled data or model parameter updates, LAD-VF establishes a novel paradigm for deploying trustworthy LLMs in resource-constrained physical environments.

Technology Category

• Natural Language Processing: Safety and Robustness
• Intelligent Robots: Learning & Optimization for ROB
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) can translate natural language instructions into executable action plans for robotics, autonomous driving, and other domains. Yet, deploying LLM-driven planning in the physical world demands strict adherence to safety and regulatory constraints, which current models often violate due to hallucination or weak alignment. Traditional data-driven alignment methods, such as Direct Preference Optimization (DPO), require costly human labeling, while recent formal-feedback approaches still depend on resource-intensive fine-tuning. In this paper, we propose LAD-VF, a fine-tuning-free framework that leverages formal verification feedback for automated prompt engineering. By introducing a formal-verification-informed text loss integrated with LLM-AutoDiff, LAD-VF iteratively refines prompts rather than model parameters. This yields three key benefits: (i) scalable adaptation without fine-tuning; (ii) compatibility with modular LLM architectures; and (iii) interpretable refinement via auditable prompts. Experiments in robot navigation and manipulation tasks demonstrate that LAD-VF substantially enhances specification compliance, improving success rates from 60% to over 90%. Our method thus presents a scalable and interpretable pathway toward trustworthy, formally-verified LLM-driven control systems.

Problem

• Ensuring LLM-generated robot plans comply with safety constraints
• Eliminating resource-intensive fine-tuning for formal feedback integration
• Providing interpretable prompt refinement instead of parameter updates

Innovation

• Fine-tuning-free framework using formal verification feedback
• Iterative prompt refinement via LLM-AutoDiff integration
• Formal-verification-informed text loss for automated prompt engineering