SkillGate: Cost Efficient Runtime Malicious Skill File Detection in Coding Agents

2026-07-28

AI Summary

This work addresses a critical security gap in widely adopted AI coding agents: their skill manifests lack safety screening, creating a novel supply-chain attack surface that adversaries can exploit to steal credentials, inject backdoors, or hijack operations. To mitigate this risk, the paper introduces SkillGate—the first runtime security gateway tailored for AI skill files. SkillGate employs a hybrid detection pipeline that efficiently screens skills prior to installation by combining regex-based pre-filtering with a large language model (LLM) discriminator, applying fine-grained analysis and lightweight interception only to suspicious code snippets. Evaluated on the SkillsBench benchmark, SkillGate achieves an F1 score of 0.817 and a false positive rate of 1.13%, reduces LLM input tokens by 77% compared to full-file scanning, and demonstrates a 5–6× improvement in AUPRC over existing tools—significantly lowering detection overhead and latency while maintaining robust security guarantees.

Abstract

Software engineering teams now deploy AI coding agents (Cursor, Claude Code, GitHub Copilot) as first-class productivity tools, installing domain-specific skill files to tailor agent behavior to project APIs, framework conventions, and organizational workflows. These complex Markdown files are easily downloaded from public registries with a single npx skills add command and no real security screening, representing a novel supply-chain attack surface: a malicious skill file can silently reprogram agent behavior, exfiltrating credentials, injecting backdoors into generated code, or redirecting agent actions to attacker-controlled endpoints. The threat is not hypothetical: recent reports document hundreds of malicious skill packages in public registries, including organized campaigns that distributed credential-stealing infostealers via fake productivity skills. No systematic toolchain defense exists for this attack surface. We present SkillGate, a deployable security gateway that screens AI skill packages before coding agent installation. SkillGate uses a hybrid regex-prefilter + LLM-judge pipeline: safe-signal files bypass the LLM entirely (skip savings); flagged files have only their matched snippet windows sent to the judge, not the full content (snippet savings). We answer four research questions covering detection effectiveness, screening cost, runtime overhead, and false positive behavior on the SkillsBench benchmark against two existing tools. On SkillsBench (n=1,650, 9.1% malicious), SkillGate achieves F1=0.817, FPR=1.13% while reducing LLM input tokens by 77% vs. full-file screening, and outperforming existing tools by 5-6x on threshold-independent AUPRC (0.830 vs. 0.144/0.162).

Problem

• AI coding agents
• skill files
• supply-chain attack
• malicious code
• security screening

Innovation

• SkillGate
• malicious skill detection
• hybrid regex-LLM pipeline
• cost-efficient LLM screening
• AI agent security