Adaptive Multi-Resolution Procedural Knowledge Compression for Large Language Models

2026-06-10

AI Summary

This work addresses the inefficiency of large language models when invoking reusable natural language skills, where repeatedly embedding full skill descriptions leads to excessively long contexts, high prefill costs, and significant latency. Existing compression techniques struggle to preserve the logical structure of procedural knowledge. To overcome this, the paper introduces SKIM, the first adaptive multi-resolution soft token compression framework tailored for procedural skills. SKIM dynamically generates a skill-specific number of soft tokens through offline optimization, aligning token allocation with skill complexity to maintain critical workflow logic and tool protocol dependencies while compressing context length. Experimental results demonstrate that SKIM reduces skill representations to 30%–60% of their original length, substantially lowering inference overhead while outperforming existing compression methods in task performance.

Abstract

Large language models (LLMs) are widely used to tackle complex tasks with autonomous workflows. Recently, reusable natural language skills have emerged as a popular paradigm to inject procedural knowledge into LLM applications. Since popular skills are often invoked repeatedly, placing their full text in every context significantly increases prefill cost and latency. While text compression techniques have the potential to solve this problem, most existing methods are designed to compress factual knowledge in documents instead of procedural knowledge, making them insufficient for skill compression. In this paper, we argue that an effective skill compression method should: 1) preserve logical dependencies among workflows and tool protocols, 2) enable lightweight, offline compression for frequently updated community skills, and 3) be adaptable to varying complexities across skills. To address this, we present SKIM (SKIll coMpression), an adaptive multi-resolution soft token compression framework for procedural skills. Depending on the complexity of each skill, SKIM creates different numbers of soft tokens that not only improve the efficiency of LLM inference, but also preserve the effectiveness of skill usage. Experiments indicate that SKIM compresses skills to 30 to 60 percent of their original token length while preserving task performance better than existing compression methods.We have released our code at https://github.com/bebr2/SKIM .

Problem

• procedural knowledge
• skill compression
• large language models
• text compression
• prefill cost

Innovation

• procedural knowledge compression
• soft token
• adaptive multi-resolution
• large language models
• skill compression