LLMs for sensory-motor control: Combining in-context and iterative learning

2025-06-05

AI Summary

This work investigates the feasibility of large language models (LLMs) as end-to-end embodied controllers, addressing the core challenge of directly mapping continuous perceptual inputs (e.g., state vectors) to continuous action outputs. Methodologically, we propose a novel paradigm integrating symbolic reasoning with embodied perception–action closed-loop learning: an initial control policy is generated via textual prompting, then iteratively refined using sensory-motor feedback from Gymnasium/MuJoCo simulation environments, leveraging in-context learning and iterative prompt engineering. Our key contribution is the first demonstration of LLM-driven, gradient-free, iterative controller learning—without explicit policy networks or reinforcement learning updates. Experiments across multiple canonical control benchmarks achieve optimal or near-optimal performance, empirically validating LLMs’ efficacy and generalization capacity as universal embodied controllers.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Intelligent Robots: Embodied AI

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We propose a method that enables large language models (LLMs) to control embodied agents by directly mapping continuous observation vectors to continuous action vectors. Initially, the LLMs generate a control strategy based on a textual description of the agent, its environment, and the intended goal. This strategy is then iteratively refined through a learning process in which the LLMs are repeatedly prompted to improve the current strategy, using performance feedback and sensory-motor data collected during its evaluation. The method is validated on classic control tasks from the Gymnasium library and the inverted pendulum task from the MuJoCo library. In most cases, it successfully identifies optimal or high-performing solutions by integrating symbolic knowledge derived through reasoning with sub-symbolic sensory-motor data gathered as the agent interacts with its environment.

Problem

• Enabling LLMs to control agents via observation-action mapping
• Iteratively refining control strategies using feedback and data
• Validating method on classic and MuJoCo control tasks

Innovation

• LLMs map observation to action vectors
• Iterative refinement using feedback
• Combines symbolic and sensory data