DROM: A Language-Guided Diffusion Framework for Multi-Skill Robotic Manipulation

2026-09-29

AI Summary

This study addresses the challenge of learning diverse, long-horizon robotic manipulation policies from limited demonstrations. To this end, it proposes a language-guided diffusion framework that, for the first time, integrates Dynamic Movement Primitives (DMPs) with language-conditioned diffusion models for data augmentation. The approach leverages cross-attention mechanisms to enable multi-skill trajectory generation and employs large language models (LLMs) for task decomposition, thereby supporting both natural language interaction and autonomous execution. The primary contribution lies in achieving multi-skill composition and orientation-sensitive behavior control within a single generative policy. Validated across real-world robotic and simulated environments, the proposed method significantly outperforms baseline approaches, demonstrating robust multi-skill generalization from few-shot demonstrations and reliable execution of long-horizon tasks.

Abstract

Learning robust manipulation policies for diverse, long-horizon tasks from limited demonstrations remains a fundamental challenge in robotics. We present DROM, a language-guided diffusion framework that enables robots to learn, represent, and compose multiple manipulation skills within a single generative policy. DROM leverages Dynamic Movement Primitives (DMPs) to augment a small set of expert demonstrations into expressive multi-skill datasets, substantially reducing data collection while improving spatial generalization beyond the demonstrated workspace. Building upon Motion Planning Diffusion (MPD), we extend the diffusion architecture to support language-conditioned multi-skill trajectory generation through cross-attention, allowing a single model to generate skill-consistent motions for a diverse set of manipulation primitives, including orientation-sensitive behaviors that are difficult to design using conventional motion planning or hard-coded controllers. For long-horizon manipulation, a large language model decomposes high-level operator requests into executable sequences of skills, enabling natural language interaction and autonomous task execution. We validate DROM on a Franka Emika Panda robot, a FANUC CRX25ia robot, and in MuJoCo simulation across a wide range of manipulation tasks. Experimental results demonstrate that DROM outperforms Motion Planning Diffusion and Behavior Cloning baselines, achieves robust multi-skill generalization, and composes learned skills to reliably execute long-horizon manipulation tasks from natural language instructions using only a limited number of human demonstrations. Datasets, simulation environments, and more at https://github.com/automation-robotics-machines/drom.

Problem

• Robotic Manipulation
• Multi-Skill Learning
• Long-Horizon Tasks
• Limited Demonstrations
• Language-Guided Control

Innovation

• Diffusion Policy
• Dynamic Movement Primitives
• Language-Conditioned Generation
• Long-Horizon Manipulation
• Large Language Models