Semantic Intelligence: Integrating GPT-4 with A Planning in Low-Cost Robotics

2025-05-03

AI Summary

Traditional robotic navigation relies on hand-coded finite-state machines and purely geometric planning, limiting its ability to interpret high-level semantic instructions. This work proposes a lightweight semantic navigation framework that integrates zero-shot semantic reasoning via GPT-4 with A* path planning on a low-cost Petoi Bittle robot (Raspberry Pi Zero 2W), implemented end-to-end in ROS2 Humble. Our key contributions are: (1) a novel prompt-driven large language model (LLM) task logic controller that replaces conventional finite-state machines; and (2) real-time encoding of semantic constraints—e.g., toxic zones, congestion, or low battery—into dynamic obstacle buffers embedded within the occupancy grid to guide A* replanning. Experiments demonstrate 96–100% success rates on multi-step, context-aware tasks, substantially outperforming geometry-only planning. To our knowledge, this is the first demonstration of rule-free, semantics–geometry co-navigating behavior on resource-constrained platforms.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Localization, Mapping, and Navigation
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Classical robot navigation often relies on hardcoded state machines and purely geometric path planners, limiting a robot's ability to interpret high-level semantic instructions. In this paper, we first assess GPT-4's ability to act as a path planner compared to the A* algorithm, then present a hybrid planning framework that integrates GPT-4's semantic reasoning with A* on a low-cost robot platform operating on ROS2 Humble. Our approach eliminates explicit finite state machine (FSM) coding by using prompt-based GPT-4 reasoning to handle task logic while maintaining the accurate paths computed by A*. The GPT-4 module provides semantic understanding of instructions and environmental cues (e.g., recognizing toxic obstacles or crowded areas to avoid, or understanding low-battery situations requiring alternate route selection), and dynamically adjusts the robot's occupancy grid via obstacle buffering to enforce semantic constraints. We demonstrate multi-step reasoning for sequential tasks, such as first navigating to a resource goal and then reaching a final destination safely. Experiments on a Petoi Bittle robot with an overhead camera and Raspberry Pi Zero 2W compare classical A* against GPT-4-assisted planning. Results show that while A* is faster and more accurate for basic route generation and obstacle avoidance, the GPT-4-integrated system achieves high success rates (96-100%) on semantic tasks that are infeasible for pure geometric planners. This work highlights how affordable robots can exhibit intelligent, context-aware behaviors by leveraging large language model reasoning with minimal hardware and no fine-tuning.

Problem

• Integrating GPT-4 with A* for semantic robot navigation
• Replacing hardcoded state machines with prompt-based reasoning
• Enabling low-cost robots to handle complex semantic tasks

Innovation

• Integrates GPT-4 with A* for hybrid path planning
• Uses prompt-based GPT-4 reasoning for task logic
• Dynamically adjusts occupancy grid via semantic constraints