SkillTrace: Multi-Trace Provenance Auditing for LLM-Agent Skill Reuse

2026-08-05

AI Summary

Existing approaches struggle to effectively audit the provenance of skill reuse by large language model agents in multimodal, fragmented scenarios, as evidence is dispersed across textual, code, and operational structures. This work proposes SkillTrace, a novel framework that formulates skill reuse auditing as a multi-trajectory provenance problem. SkillTrace constructs a Skill Operation Graph (SOG) by extracting three types of trajectories—expressive, implementational, and operational—and leverages large language models solely during ingestion for efficient, deterministic trajectory matching. The method incorporates a negative-sample calibration mechanism, achieving an AUROC of 0.938 and an F1 score of 0.898 on SkillTrace-Bench. Evaluation on 36,446 real-world skills reveals actionable instances of skill reuse that surpass repository-level baselines.

Abstract

LLM-agent ecosystems are rapidly growing around reusable skills: mixed-modality packages of metadata, natural-language instructions, code, tools, references, and operational workflows. As skills become marketplace artifacts, auditing their reuse is no longer the same problem as ordinary code clone detection. Existing detectors target single-modality source code or whole-package similarity, yet skill reuse evidence is distributed across authored text, implementation fragments, and operational structure. As a result, they can miss reuse that preserves only one part of a skill. We present SKILLTRACE, a multi-trace provenance auditing framework for LLM-agent skill reuse. SKILLTRACE extracts three provenance traces: Expression, Implementation, and Operational. It represents the Operational Trace as a Skill Operational Graph (SOG) that captures activation, procedure, and resource-flow structure. An LLM assists only the Operational-trace extraction, once at ingestion; at audit time SKILLTRACE compares cached traces deterministically, calibrates each trace against same-function strict negatives, and reports which trace supports a reuse decision. On SKILLTRACE-BENCH, with 820 transformed reuse positives over 100 marketplace anchors and 751 negative controls, SKILLTRACE achieves AUROC 0.938 and F1 0.898. A 36,446-skill wild audit further shows that trace-attributed evidence surfaces actionable reuse review queues beyond repository-level baselines.

Problem

• skill reuse
• provenance auditing
• multi-trace
• LLM-agent
• operational graph

Innovation

• multi-trace provenance
• skill reuse auditing
• Skill Operational Graph
• LLM-agent ecosystem
• deterministic trace comparison