SkillContrast: Difference-Guided Text Selection for Agent Skill Reranking

2026-10-08

AI Summary

This study addresses the issue wherein shared instructions among similar agent skills cause retrieval systems to overlook critical distinctions. To mitigate this, we propose a training-free selector framework that contrasts retrieved candidate skills and preserves only their differentiating text segments. By leveraging relative differences among candidates to augment query relevance, the method supplies compact yet precise inputs to pretrained rerankers. Experimental results demonstrate that the proposed approach reduces token consumption by 51%–59% while significantly increasing the number of clean hits and improving overall reranking efficiency.

Abstract

Similar agent skills can share instructions but differ in their conditions of use. Query-based text selection may retain shared instructions and omit these distinctions. We introduce SkillContrast, a training-free selector that compares retrieved skills and retains their differing text with local context for a pretrained reranker. On 1,235 requests from SameCapRisk-Bench, it yields 54-72 more clean hits (requests that retrieve a helpful skill without its marked risky sibling) than TF-IDF query selection at identical per-candidate input lengths, across 2 retrievers and 2 reranker sizes. Length-matched component replacements identify differing text as the main contributor in the primary setting, with smaller, mixed context effects. Relative to full skill bodies, SkillContrast uses 51.1-58.8% fewer model-input tokens, with 10-18 fewer clean hits at 0.6B and matching or higher observed clean-hit counts at 4B. Candidate-relative differences thus complement query relevance in selecting compact reranking inputs.

Problem

• skill reranking
• text selection
• agent skills
• query-based retrieval
• risky sibling

Innovation

• Training-free selector
• Difference-guided text selection
• Agent skill reranking
• Compact input
• SameCapRisk-Bench