Game-Guided Skill Discovery through Self-Play for Playable Agent Control

2026-09-30

AI Summary

This study addresses the challenge that existing unsupervised skill discovery methods struggle to simultaneously achieve semantic distinctiveness, interpretability, and expressiveness, rendering discovered skills unsuitable for direct human control. To overcome this, we propose the GGSD framework, which introduces a competitive self-play-based skill discovery mechanism within a hierarchical reinforcement learning architecture. By mapping high-level discrete actions to low-level continuous behaviors, our approach enables complex behavioral sequences to emerge from combinations of a few discrete skills, thereby transcending the expressiveness limitations of single primitives. Experiments in Ant and Franka environments demonstrate that the generated skills exhibit both high interpretability and strong expressiveness. Notably, humans can accomplish unseen tasks, such as maze navigation, through intuitive skill composition without requiring additional training.

Abstract

We present Game-Guided Skill Discovery (GGSD), a framework that uses self-play in games to discover motor skills that are directly playable by humans. Playable skills provide a compact abstraction for controlling embodied agents through a small set of learned behaviors rather than low-level actions. To be effective, these skills should be semantically distinct, interpretable, and expressive; properties that existing unsupervised skill-discovery methods often fail to achieve simultaneously. GGSD achieves these desiderata by grounding skill discovery in competitive gameplay. A hierarchical agent competes against its past selves, with a high-level policy selecting from a small discrete skill set and a skill-conditioned low-level policy learning the corresponding behaviors. After training, a human can replace the high-level policy and directly control the agent through the same discrete skills. Despite the small number of high-level actions, skill transitions give rise to emergent combo behaviors, expanding expressivity beyond individual primitives. Across Ant, Franka-arm, and Unitree G1 environments, we show that GGSD produces human-playable skills that humans can compose to solve unseen tasks, such as Maze and CubePush, without additional training. An interactive demo is available at https://ggsd-demo.github.io.

Problem

• skill discovery
• playable agent control
• unsupervised learning
• embodied agents
• motor skills

Innovation

• Self-Play
• Skill Discovery
• Hierarchical Reinforcement Learning
• Playable Agent
• Emergent Behaviors