Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression

2026-08-05

AI Summary

This work identifies a paradigm-level flaw in hard prompt compression: by independently selecting high-scoring text spans, it often discards semantically dependent content, leading to dangling references—where retained passages lack definitions of removed entities and thus break multi-hop reasoning. To address this, the authors propose a lightweight, unsupervised method that scores text blocks using Qwen3 embeddings, employs a referential integrity classifier to detect critical missing sentences, and restores their semantic completeness via contextual reinsertion. On HotpotQA, automatically reinserting such key sentences improves accuracy by 4.7 points; when human-curated supporting paragraphs are reinserted, performance gains reach 29–34 percentage points, substantially narrowing the gap with full-context models.

Abstract

Hard prompt compression reduces long-context inference cost by independently scoring tokens, sentences, or chunks and retaining the highest-scoring units under a budget. We identify a structural failure in this procedure: independent selection can split dependent evidence pairs, retaining one member while deleting the other. When retained text contains an answer but deleted text defines the entity needed to interpret it, we call the result referential dangling. At a compression ratio of 0.30, Beaver, which ranks coherent chunks using Qwen3-0.6B embeddings, leaves the answer path incomplete in 34-54% of bridge examples across three multi-hop question answering datasets. On a shared HotpotQA bridge set, all six hard compressors we test exhibit dangling at rates up to 60%, and every document in LongBench-v2 Single-Document QA contains at least one dangling reference. On dangling examples evaluated with Qwen3-8B, reinserting the missing supporting paragraph while removing nonsupporting paragraphs to maintain the token budget improves accuracy by 29-34 percentage points (p < 0.0001), recovering at least 88% of the gap to contexts retaining both supporting paragraphs. Stronger answer models do not absorb the loss: on MuSiQue, GPT-5.5 is 8.8 points less accurate on compressed contexts than on contexts retaining both supporting paragraphs. Finally, we train a compact classifier to rank omitted sentences by whether they are needed to interpret retained text and reinsert the top-ranked candidates without support annotations at inference. On HotpotQA with Qwen3-8B, this automatic restoration improves accuracy by 4.7 points while changing the compression ratio only from 0.30 to 0.31. Hard compressors should optimize both relevance and referential completeness.

Problem

• hard prompt compression
• referential dangling
• multi-hop question answering
• context completeness
• evidence dependency

Innovation

• referential dangling
• hard prompt compression
• context compression
• multi-hop reasoning
• automatic restoration