Self-Supervised Skill Optimization

2026-07-30

AI Summary

This work proposes the first fully unsupervised framework for reusable skill optimization that operates without access to ground-truth labels, rewards, or reliable evaluators. The method iteratively refines skills by executing them on unlabeled task instances, generating skill probes, and leveraging large language models as pairwise judges to compare execution outcomes. By integrating behavioral discrepancy extraction and evidence aggregation mechanisms, the framework updates skills in a self-supervised manner. Evaluated on both closed-ended and open-ended tasks, the approach significantly outperforms existing unsupervised prompt optimization methods and achieves performance on par with—or even surpassing—that of the strongest supervised optimizers on closed-domain benchmarks, thereby demonstrating, for the first time, effective skill enhancement for large language model agents under purely unsupervised conditions.

Abstract

Agent skills provide frozen large language model (LLM) agents with reusable procedural guidance, and recent work shows that such skills can be optimized with ground-truth (GT) feedback. Many applications, however, lack GT labels, task scores, rewards, or reliable task-specific evaluators. We therefore introduce Self-Supervised Skill Optimization (SSO), a comparative framework that learns a reusable skill from unlabeled task instances alone. At each step, SSO runs the current skill on an unlabeled batch, uses a subset of the resulting executions to generate complete skill probes, and runs the probes on the same batch. An LLM judge compares the resulting answers, trajectories, artifacts, or terminal states. A separate behavior extractor identifies behavioral differences without seeing the judge's decisions. SSO uses these decisions to aggregate evidence for and against the observed behaviors across instances. It then ranks the behaviors by the resulting evidence and renders a new complete skill from the highest-ranked behaviors. The update is accepted only if the new skill outperforms the current one on an unlabeled validation set. SSO outperforms existing GT-free prompt optimizers on both closed-ended and open-ended tasks. On closed-ended benchmarks, it approaches and sometimes exceeds the strongest GT-based skill optimizer without using any GT feedback.

Problem

• self-supervised
• skill optimization
• unlabeled data
• LLM agents
• ground-truth-free

Innovation

• Self-Supervised Skill Optimization
• LLM Agent
• Unlabeled Task Instances
• Behavior Extraction
• Comparative Framework