Training Skills Like Parameters via Self-Supervised Semantic Diffusion

2026-07-29

AI Summary

This work addresses the limitations of existing closed-source large language models in efficiently acquiring, updating, and reusing domain-specific skills, which often rely on costly human annotations or unreliable model-based evaluations. The authors propose an unsupervised self-evolving agent framework that, for the first time, adapts the destroy-and-reconstruct mechanism from diffusion models to skill learning. By contrasting agent-reconstructed text with original high-quality human-written text, the framework generates self-supervised signals to refine an external skill library—comprising skills that are readable, transferable, and composable—without updating the model’s internal weights. Integrating self-supervised contrastive learning with a semantic diffusion mechanism, the approach enables autonomous skill distillation and continuous iteration. Evaluated on short-form screenplay generation, it significantly improves output quality, demonstrating strong scalability and generalization in autonomously learning complex human artifacts.

Abstract

While Large Language Models (LLMs) demonstrate remarkable general instruction-following capabilities, they often fall short of human experts in highly specialized, open-ended domains such as creative screenwriting. Prior approaches typically adopt post-training, yet both supervised fine-tuning and reinforcement learning require weight access that closed-source frontier models do not offer, and demand heavy compute. Moreover, what is learned is tied to a single checkpoint and cannot be inspected by humans. Recent advancements in agentic continual learning instead attempt to bridge this gap by accumulating external textual skills. However, these methods heavily rely on costly human expert annotations or unreliable LLM-as-a-judge feedback for reflection. To overcome this bottleneck, we propose a novel, unsupervised self-evolving agent framework inspired by the corruption-and-reconstruction paradigm of diffusion models. Instead of relying on explicit external scoring, we leverage existing high-quality human artifacts to construct self-supervised signals. Training then follows the familiar loop of neural network training, forward, loss, and backward, with the loss coming from contrasting the agent's reconstruction against the human original. What is updated is not model weights but an external library of textual skills. We evaluate our framework on the challenging task of short drama screenwriting. Experimental results demonstrate that our method enables the agent to autonomously extract and internalize highly generalizable skills, significantly enhancing its domain-specific generation capabilities. Furthermore, this self-contrastive reflection paradigm offers a scalable pathway for agents to teach themselves the production of complex, high-quality human artifacts, without requiring external supervision.

Problem

• self-supervised learning
• textual skills
• agent learning
• diffusion models
• screenwriting

Innovation

• self-supervised learning
• semantic diffusion
• external skill library
• unsupervised agent evolution
• self-contrastive reflection