SkillApt: Learning When to Activate Agent Skills from Counterfactual Evidence

2026-09-22

AI Summary

为解决语言模型中技能激活不必要或有害的问题，提出SkillApt框架，通过对比执行证据决定是否加载技能，有效减少不必要的技能激活。

Abstract

Large language model agents increasingly retrieve reusable Skills and inject them into the active context. However, a retrieved Skill can be relevant yet unnecessary, costly, or even harmful in the current execution state. We present SkillApt, a post-retrieval activation framework that decides whether a retrieved Skill should actually be loaded. SkillApt builds execution evidence from matched WITH/WITHOUT runs and uses outcomes from similar historical states to make a LOAD/ABSTAIN decision for each candidate Skill. On the frozen confirmatory SRA-Bench evaluation, SkillApt-E achieved the same observed accuracy as BM25 Top-1 (0.838 vs. 0.838) while reducing the Skill activation rate from 100% to 31.5% and mean token usage by 74.3%. Further diagnostics show that both Skill utility and the learnability of its activation boundary vary across base models. These results suggest that Skill retrieval and Skill activation should be treated as separate decisions: retrieval identifies which Skill may be relevant, while SkillApt determines whether using it is worthwhile in the current state.

Problem

• Skill Activation
• Large Language Models
• Retrieval
• Execution State
• Cost

Innovation

• post-retrieval activation
• counterfactual evidence
• skill activation rate
• token usage