PhyRoGen: Synthetic Generation of Physical Robot Manipulation Puzzles Using Procedural Content Generation

2026-06-04

AI Summary

This work addresses the scarcity of large-scale, diverse datasets of physical manipulation puzzles that currently limits research in robotic manipulation. To bridge this gap, we introduce PhyRoGen, a novel framework that pioneers the application of procedural content generation (PCG) to physical robot manipulation by automatically synthesizing solvable puzzles featuring interlocked objects with dependency relationships. We design six distinct puzzle generators and validate our approach through integration with a sampling-based planning algorithm on the KUKA LBR iiwa simulation platform. Our experiments successfully produce 24 structurally diverse and physically feasible puzzles, all of which are efficiently solved within 1–300 seconds. This establishes a controllable and scalable new benchmark for training and evaluating robotic manipulation skills.

Abstract

Robot manipulation of physical puzzles is important for automatic assembly and disassembly tasks. However, to enable robots to solve physical puzzles, manipulation skills need to be learned, which requires large training datasets, the generation of which is often time consuming and tedious. To overcome this problem, we propose the Physical Robot Manipulation Puzzle Generation framework (PhyRoGen), which leverages procedural content generation (PCG) for automated generation of synthetic datasets of manipulation puzzles. PhyRoGen is a general-purpose puzzle generator, which can generate physical puzzles with interlocking object dependencies, where one articulated object must be manipulated before another can be moved. Based upon PhyRoGen, we define six concrete generators which we use to generate 24 physical puzzles. By using a benchmarking framework, we are able to solve all puzzles in 1 to 300 seconds using sampling-based planning algorithms. Finally, we demonstrate that every generated puzzle is manipulatable by using a KUKA LBR iiwa robot in a physical simulation. This shows that our framework is able to procedurally generate unique, solvable robot manipulation puzzles, which is a crucial ingredient to benchmark manipulation algorithms and to develop robust foundation models.

Problem

• robot manipulation
• physical puzzles
• training datasets
• data generation
• procedural content generation

Innovation

• Procedural Content Generation
• Robot Manipulation
• Physical Puzzles
• Synthetic Dataset
• Articulated Objects