Investigating and Alleviating Harm Amplification in LLM Interactions

2026-06-01

AI Summary

This study addresses a critical gap in the safety mechanisms of large language models, which often overlook the cumulative and amplifying risks of harm across multi-turn dialogues. The work formally defines the phenomenon of “harm amplification” for the first time, establishing evaluation criteria that require substantive escalation, operational concreteness, and multi-turn necessity. To systematically assess this issue, the authors introduce HarmAmp, a novel benchmark for multi-turn harm amplification. Building upon this foundation, they propose TrajSafe, an active monitoring approach that proactively predicts harmful dialogue trajectories and intervenes by detecting user intent and guiding safe responses. Experimental results demonstrate that TrajSafe significantly reduces harmful outputs in multi-turn interactions while maintaining low over-refusal rates and preserving the model’s general capabilities.

Abstract

Large language models (LLMs) can serve as helpful assistants, yet they can equally function as harm amplifiers that enable malicious users to achieve harmful outcomes beyond their capabilities through extended interactions. This risk manifests along two axes, i.e., democratizing domain expertise that allows novices to produce specialized harmful content, and scaling harmful operations at volumes that manual effort cannot match. Existing works, however, often overlook how LLMs compound harm across multi-turn conversations. We introduce HarmAmp, a new benchmark for multi-turn harm amplification scenarios spanning twelve risk categories. Each scenario is grounded in real-world threats and satisfies rigorous criteria, i.e., substantive amplification, operational specificity, and multi-turn necessity. We further propose TrajSafe, a proactive monitor that anticipates harmful trajectories and intervenes through actions such as probing users' genuine intents and steering the models towards safer completion. Our extensive experiments demonstrate that TrajSafe significantly reduces the harmfulness incurred in multi-turn interactions while preserving a low over-refusal rate and the target model's general capabilities. Our work offers a promising paradigm to alleviate the nuanced safety risks in LLM interactions.

Problem

• harm amplification
• large language models
• multi-turn interactions
• safety risks
• malicious use

Innovation

• harm amplification
• multi-turn safety
• proactive monitoring
• LLM alignment
• TrajSafe