SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinement

2026-06-09

AI Summary

This work addresses the subpar quality of skill documentation generated by large language models (LLMs), which significantly lags behind human-authored counterparts. To bridge this gap, the paper introduces the first fully unsupervised framework for LLM-based skill self-optimization. By decomposing skill quality into four interpretable dimensions, the approach enables the model to self-evaluate, generate structured improvement briefs, and iteratively refine its skills—without relying on ground-truth labels, test suites, or environmental rewards. Evaluated on SkillsBench, the method improves task pass rates by 28%, closing 47–67% of the performance gap with human-written skills. On SpreadsheetBench, it boosts pass rates from 16.0% to 52.0% using only 22 skills, demonstrating its effectiveness as a continual learning engine in open-ended environments.

Abstract

Skill documents, structured natural-language instructions that guide Large Language Model (LLM) agents, are critical to modern agent frameworks, yet LLMs struggle to write skills that actually work. On SkillsBench, human-authored skills improve pass rates by 16.2 percentage points, while LLM-authored skills provide no measurable gain. We introduce SkillAxe, a fully unsupervised framework that enables LLMs to iteratively diagnose and refine their own skills. SkillAxe decomposes skill quality into four interpretable dimensions (quality impact, trigger precision, instruction compliance with fault attribution, and solution-path coverage), producing structured improvement briefs that require no ground-truth labels, test suites, or environment rewards. On SkillsBench, SkillAxe improves pass rates by 28\% relative over unimproved LLM skills and closes 47--67\% of the gap to human-authored skills. We validate the approach as a continuous improvement engine in the wild on SpreadsheetBench, where a SkillAxe-built skill library learns from past agent trajectories and raises pass rate from 16.0\% to 52.0\% using only 22 skills.

Problem

• LLM-authored skills
• skill effectiveness
• agent frameworks
• skill refinement
• performance gap

Innovation

• self-refinement
• unsupervised skill improvement
• interpretable skill evaluation
• LLM agent skills
• evaluation-guided learning