Interactive LLM-assisted Curriculum Learning for Multi-Task Evolutionary Policy Search

2026-02-11

AI Summary

This work proposes an interactive large language model (LLM)-assisted framework for online curriculum generation in multitask policy search, addressing the limitations of poor generalization and the heavy reliance on manual design and domain expertise in traditional curricula. For the first time, the approach enables real-time collaboration between an LLM and an evolutionary optimizer: leveraging a genetic programming-driven policy evolution process, the LLM dynamically generates training tasks while incorporating multimodal feedback—including numerical metrics, progress plots, and behavioral visualizations—to enhance curriculum quality. Evaluated on 2D robotic navigation tasks, the method significantly outperforms static curriculum strategies and achieves performance comparable to expert-designed curricula, demonstrating the effectiveness and potential of LLMs in automating curriculum generation for reinforcement learning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Multi-task policy search is a challenging problem because policies are required to generalize beyond training cases. Curriculum learning has proven to be effective in this setting, as it introduces complexity progressively. However, designing effective curricula is labor-intensive and requires extensive domain expertise. LLM-based curriculum generation has only recently emerged as a potential solution, but was limited to operate in static, offline modes without leveraging real-time feedback from the optimizer. Here we propose an interactive LLM-assisted framework for online curriculum generation, where the LLM adaptively designs training cases based on real-time feedback from the evolutionary optimization process. We investigate how different feedback modalities, ranging from numeric metrics alone to combinations with plots and behavior visualizations, influence the LLM ability to generate meaningful curricula. Through a 2D robot navigation case study, tackled with genetic programming as optimizer, we evaluate our approach against static LLM-generated curricula and expert-designed baselines. We show that interactive curriculum generation outperforms static approaches, with multimodal feedback incorporating both progression plots and behavior visualizations yielding performance competitive with expert-designed curricula. This work contributes to understanding how LLMs can serve as interactive curriculum designers for embodied AI systems, with potential extensions to broader evolutionary robotics applications.

Problem

• multi-task policy search
• curriculum learning
• LLM-assisted design
• evolutionary optimization
• generalization

Innovation

• Interactive LLM
• Curriculum Learning
• Evolutionary Policy Search
• Multimodal Feedback
• Genetic Programming