🤖 AI Summary
This study investigates whether large language models (LLMs) can effectively replace or collaborate with classical symbolic planners for task planning. To this end, the authors propose a novel paradigm that integrates a PDDL-based planner with an LLM via the Model Context Protocol (MCP): actions from PyPDDLEngine are exposed as callable tools, enabling the LLM to generate plans interactively and incrementally based on environmental feedback, in contrast to conventional one-shot plan generation. Evaluated on 102 Blocksworld tasks, the LLM agent achieved a planning success rate of 66.7%, slightly outperforming direct prompting (63.7%) but at 5.7 times higher token cost. Both approaches produced notably shorter plans, suggesting reliance on memorized patterns from training data rather than robust generalization. This work offers a fine-grained interaction framework for synergizing LLMs with symbolic planning.
📝 Abstract
Task planning, the problem of sequencing actions to reach a goal from an initial state, is a core capability requirement for autonomous robotic systems. Whether large language models (LLMs) can serve as viable planners alongside classical symbolic methods remains an open question. We present PyPDDLEngine, an open-source Planning Domain Definition Language (PDDL) simulation engine that exposes planning operations as LLM tool calls through a Model Context Protocol (MCP) interface. Rather than committing to a complete action sequence upfront, the LLM acts as an interactive search policy that selects one action at a time, observes each resulting state, and can reset and retry. We evaluate four approaches on 102 International Planning Competition (IPC) Blocksworld instances under a uniform 180-second budget: Fast Downward lama-first and seq-sat-lama-2011 as classical baselines, direct LLM planning (Claude Haiku 4.5), and agentic LLM planning via PyPDDLEngine. Fast Downward achieves 85.3% success. The direct and agentic LLM approaches achieve 63.7% and 66.7%, respectively, a consistent but modest three-percentage-point advantage for the agentic approach at $5.7\times$ higher token cost per solution. Across most co-solved difficulty blocks, both LLM approaches produce shorter plans than seq-sat-lama-2011 despite its iterative quality improvement, a result consistent with training-data recall rather than generalisable planning. These results suggest that agentic gains depend on the nature of environmental feedback. Coding agents benefit from externally grounded signals such as compiler errors and test failures, whereas PDDL step feedback is self-assessed, leaving the agent to evaluate its own progress without external verification.