Behavioral Skill Reconstruction: Reconstructing Hidden Functionality from LLM Agent Skills

2026-08-04

AI Summary

Although closed-source large language model (LLM) agents conceal their internal implementations, their functionalities can still be reverse-engineered through normal interactions, rendering file-level confidentiality insufficient for protecting functional secrecy. This work proposes Behavioral Skill Reconstruction (BSR), a method that, under black-box settings, systematically enables functional cloning attacks by observing legitimate task requests and responses. We introduce SkillClone, a framework integrating interface hypothesis generation, structured probing, executable replica synthesis, and an iterative differential validation and repair mechanism. Evaluated on 30 diverse skills—spanning rules, tables, procedures, and algorithms—our approach achieves precise or partial functional recovery. Moreover, iterative re-querying significantly enhances reconstruction completeness, demonstrating that routine usage alone can lead to cumulative leakage of functional information.

Abstract

Closed source agent skills may encode proprietary instructions, scripts, constants, and data. Providers may offer their capabilities as services while keeping the underlying packages hidden. Prior work focuses on prompt injection attacks that directly disclose these artifacts, and existing defenses accordingly aim to prevent such leakage. However, preventing file disclosure does not prevent users from recovering the functionality those files implement. This raises a fundamental question: can a user reconstruct a skill's functionality through ordinary use while its files remain hidden? We study behavioral skill reconstruction (BSR), in which an attacker uses valid task requests and observed responses to build a functional clone of a hidden skill. We introduce SkillClone, a black-box attack that clones a target skill by forming an interface hypothesis from its public advertisement, issuing structured benign probes, synthesizing an executable replica, and iteratively repairing it through differential validation against the victim skill. Across 30 skills spanning rules, tables, procedures, and algorithms, SkillClone achieves exact or partial recovery on held-out inputs for several targets. Iterative requerying closes gaps missed by single-round reconstruction. Because SkillClone uses only legitimate interactions, disclosure-focused defenses provide limited coverage, and less detailed skill descriptions offer limited protection. These results show that file secrecy alone does not ensure functional secrecy. Defenses must also limit cumulative information leakage from ordinary use.

Problem

• Behavioral Skill Reconstruction
• LLM Agent Skills
• Functional Secrecy
• Black-box Attack
• Information Leakage

Innovation

• Behavioral Skill Reconstruction
• SkillClone
• black-box attack
• functional cloning
• information leakage