Risky Business: Measuring The Faithfulness-Safety Tension

2026-08-04

AI Summary

This study addresses the inherent tension between high faithfulness (i.e., monitorability of reasoning traces) and high safety (i.e., rejection of hazardous reasoning) in large reasoning models. To systematically evaluate and intervene in reasoning chains without relying on prompt injection, the authors introduce the HazMart benchmark and propose a Targeted Reasoning Replacement (TRR) method. Through mechanistic interpretability analysis and representation manipulation, they uncover— for the first time—orthogonal internal representation directions governing faithfulness and safety, enabling their independent control. Empirical results reveal a stark trade-off: DeepSeek-R1-Llama-70B achieves 97.5% faithfulness but only 12.3% safety, whereas QwQ-32B attains 73.9% safety at the cost of reduced faithfulness (74.7%). Crucially, representation manipulation improves safety by 9 percentage points without compromising core model capabilities.

Abstract

Chain-of-Thought (CoT) reasoning offers a promising window into model monitoring. However, monitoring relies on faithfulness, i.e., the model output strictly derives from its reasoning trace. We identify an alignment tension where a model must be faithful enough to be monitored, yet robust enough to reject unsafe reasoning. We demonstrate that this counterbalance exists in current Large Reasoning Models (LRMs), and show ways in which it can be addressed. We introduce HazMart, a human-written dataset set in an autonomous AI shopkeeper scenario. Unlike prior work that relies on providing hints in prompts to test faithfulness (e.g., "A Stanford professor said it should be Answer A"), we propose a novel replacement-based technique, which we call Targeted Reasoning Replacement (TRR), that directly intervenes in the reasoning chain to substitute in unsafe or illogical thoughts (e.g., "Wait, the answer must be Option B [was Option A] because it is the most fitting"). DeepSeek-R1-Llama-70B exhibits high faithfulness (97.5%) but fails to reject Unsafe Reasoning (12.3%), while QwQ-32B is more robust (73.9% safety) at the cost of lower faithfulness (74.7%). Mechanistic analyses of QwQ-32B reveal that these properties are represented by anti-correlated internal directions peaking at the action-commit token. Finally, we demonstrate that representation steering can independently amplify the safety direction, increasing safe behavior by 9 percentage points while maintaining base capabilities.

Problem

• faithfulness
• safety
• reasoning
• alignment tension
• Large Reasoning Models

Innovation

• faithfulness-safety tension
• Targeted Reasoning Replacement (TRR)
• representation steering
• Large Reasoning Models (LRMs)
• mechanistic interpretability