COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Space

2026-05-30

AI Summary

This work addresses the challenge that unsupervised skill discovery often yields task-irrelevant or hazardous behaviors, while existing guidance approaches rely on additional model training or expert demonstrations and struggle under sparse online human feedback. The authors propose a training-agnostic guidance mechanism that constructs a semantically coherent latent skill space by leveraging dense unsupervised data to learn structured representations. This approach enables effective skill learning with only minimal online human feedback and eliminates the need for training auxiliary guidance models. Empirical results demonstrate that the method successfully acquires diverse, human-aligned, and safe skills across both state-based and pixel-based environments, achieving superior downstream task performance with remarkably few feedback signals.

Abstract

Unsupervised skill discovery (USD) aims to learn diverse behaviors without reward functions, but often results in task-irrelevant or hazardous behaviors due to uniform exploration. Guided skill discovery (GSD) addresses this issue by incorporating human intent to focus exploration on meaningful regions. However, existing GSD methods typically require training additional guidance models, and rely on pre-defined rules or expert demonstration, which can be ineffective under sparse, online-collected human feedback. To overcome this, we propose COLLIE, a GSD framework that leverages dense unsupervised data to construct a semantically coherent skill latent space. This latent space is well-structured, enabling reliable guidance with sparse online feedback. Moreover, its semantic coherence property enables training-free construction of guidance signals, eliminating the need for additional model training beyond skill learning. Theoretical analysis justifies the effectiveness of our training-free guidance signal, while experiments across diverse state-based and pixel-based tasks show that COLLIE learns diverse, human-aligned skills, avoids hazardous behaviors, and achieves superior downstream performance with minimal human feedback.

Problem

• guided skill discovery
• unsupervised skill discovery
• human feedback
• latent space
• semantic coherence

Innovation

• guided skill discovery
• semantically coherent latent space
• unsupervised skill discovery
• training-free guidance
• sparse human feedback