MORSE: Multi-Context Ordering via Reverse Scoring for Evidence-Preserving Compression

2026-09-23

AI Summary

研究解决了压缩过程中证据保留受上下文顺序影响的问题，提出MORSE方法，通过反向评分优化上下文顺序以提高证据保留。

Abstract

Likelihood-based context compression can account for cross-context redundancy through sequential scoring, but this makes compression outcomes sensitive to context order. We show that different permutations of the same context collection can produce markedly different evidence-retention outcomes under an unchanged compressor. We attribute this sensitivity to information preemption: earlier partially relevant contexts can absorb credit for shared information, suppressing the incremental score of later, stronger evidence carriers and increasing their risk of removal. Controlled pair-swap interventions directly support this mechanism by showing that evidence-first ordering substantially improves supporting-evidence survival. To address this problem, we introduce MORSE, a compression-aware method for evidence-preserving context ordering. MORSE applies a common reverse query-evidence principle to both individual contexts and compressed candidate outputs, using the former to construct an evidence-first anchor and the latter to guide compression-aware permutation selection. Across multi-hop QA benchmarks, compression procedures, budgets, and scoring models, MORSE consistently improves evidence preservation over static reverse ordering and compute-matched random search, with corresponding overall improvements in downstream QA. Our code is available at https://github.com/tbn5pj/MORSE_code.

Problem

• context compression
• evidence preservation
• information preemption
• context order

Innovation

• MORSE
• evidence-preserving compression
• reverse scoring
• information preemption
• context ordering