Steering Instruction Hierarchies at Inference Time

2026-07-28

AI Summary

This work addresses the failure of large language models to consistently adhere to hierarchical instruction safety assumptions during inference—specifically, the inability of high-priority instructions (e.g., system prompts) to override conflicting lower-priority content. To resolve this, the authors propose V-Steer, the first training-free, inference-time intervention that leverages logit attribution analysis of the initial token to perform in-place multiplicative editing of cached Value vectors. This approach precisely amplifies the influence of high-priority instructions while suppressing conflicting signals. V-Steer incurs only a one-time prefill overhead, is compatible with fused attention mechanisms, and dramatically improves performance: on role-conflict benchmarks, it elevates primary constraint accuracy from below 18% to 92% across models ranging from 7B to 70B parameters, matching or exceeding state-of-the-art trained methods in three out of four hierarchical instruction-following evaluations.

Abstract

Instruction hierarchies are a core safety assumption of language model deployment: higher priority inputs, such as system prompts, should override conflicting lower priority inputs from users or tools. Yet frontier LLMs often violate this hierarchy. We introduce V-Steer, a training-free inference time method that restores privileged influence by editing cached value vectors at prompt positions. Using direct logit attribution on the first next token prediction, V-Steer identifies heads where lower priority spans dominate privileged ones, then boosts privileged spans and suppresses conflicting lower priority spans through in-place multiplicative edits to cached V tensors. Since the method acts only on cached values, it remains compatible with fused attention backends and adds only a one time prefill overhead. Across models from 7B to 70B, this attribution guided intervention raises primary constraint accuracy from under 18% up to 92% on controlled role conflict benchmarks, and on broader instruction hierarchy evaluations substantially outperforms prompt only baselines while matching or exceeding SoTA training based methods on 3 of 4 scales of LLMs, with negligible decoding-speed overhead. The code is available at https://github.com/cindy2000sh/v-steer.

Problem

• instruction hierarchies
• language model safety
• priority override
• LLM alignment
• system prompts

Innovation

• instruction hierarchies
• inference-time intervention
• value vector editing
• direct logit attribution
• training-free steering