Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarios

2024-09-28

AI Summary

To address performance degradation in large language models (LLMs) under long-context scenarios—caused by redundant information interference and sensitivity to the positional placement of critical information—this paper proposes a training-free prompt compression framework. Methodologically, it leverages pure prompt engineering without introducing additional parameters or fine-tuning. Its key contributions are threefold: (1) a novel *instruction-aware retriever* that locates semantically critical text segments guided by user instructions; (2) a *dual-slope dynamic compression ratio allocator*, which adaptively adjusts the open-book retention rate versus compression intensity based on content importance; and (3) a *semi-guided iterative compression mechanism*, enabling token-level semantic filtering and precise removal of distracting tokens. Evaluated on major long-context benchmarks—including NaturalQuestions, LongBench, and MuSiQue—the framework consistently outperforms state-of-the-art methods, demonstrating both effectiveness and strong generalization across diverse tasks and domains.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Prompt Engineering / Prompting
• Data Mining & Knowledge Management: Data Compression

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large language models (LLMs) demonstrate exceptional capabilities in various scenarios. However, they suffer from much redundant information and are sensitive to the position of key information in long context scenarios. To address these challenges, we present Perception Compressor, a training-free prompt compression framework. It includes a perception retriever that leverages guiding questions and instruction to retrieve the most relevant demonstrations, a dual-slope ratio allocator to dynamically allocate compression ratios and open-book ratios, and a semi-guided iterative compression that retains key information at the token level while removing tokens that distract the LLM. We conduct extensive experiments on long context benchmarks, i.e., NaturalQuestions, LongBench, and MuSiQue. Experiment results show that Perception Compressor outperforms existing methods by a large margin, achieving state-of-the-art performance.

Problem

• Long Content Processing
• Information Compression
• Relevance Identification

Innovation

• Perception Compressor
• Example Selection Assistant
• Semi-Guided Compression Process