Learning When to Trust via Selective Context Preference Optimization

2026-08-06

AI Summary

This work addresses the challenge that language models struggle to dynamically assess the reliability of external context, often oscillating between excessive skepticism and unwarranted trust. To tackle this issue, the study introduces SCOPE, the first approach to frame context evaluation as a selective trust task. Built upon the Direct Preference Optimization (DPO) framework, SCOPE leverages the newly curated MIST benchmark—comprising human-annotated examples across four context types: clean, misleading, correct, and irrelevant—and employs balanced preference optimization over paired samples to train models to adaptively judge context credibility. Evaluation using the proposed SC2W metric demonstrates that SCOPE substantially reduces errors induced by misleading contexts while preserving performance under other conditions, thereby enhancing model robustness and generalization.

Abstract

Language models increasingly condition their answers on external signals, and a single misleading one can turn a correct answer wrong. The obvious remedy, training models to resist such signals, hides a failure mode: a model that ignores all context looks robust yet is useless when the context is worth trusting. We recast the problem as selective trust and introduce MIST, a human-annotated benchmark that renders each reasoning item under four matched conditions (clean, misleading, correct-context, and irrelevant-context), together with SC2W, a paired metric counting how often a misleading signal flips a clean-correct answer to wrong. Across a comprehensive benchmark study, we observe that such a susceptibility is universal. We then propose SCOPE, which mines clean-correct/misleading-wrong failures and optimizes a standard Direct Preference Optimization (DPO) objective over matched preference pairs balanced equally across all four conditions, rather than over misleading items alone. Our approach substantially reduces SC2W on popular open-sourced models while preserving accuracy when the added context is clean, correct, or irrelevant. With this work, we argue that models should be judged on selective trust, not on resistance alone.

Problem

• selective trust
• context reliability
• misleading signals
• language models
• preference optimization

Innovation

• selective trust
• preference optimization
• context robustness
• DPO
• benchmarking