Adversarial Prompts for Acceptance Collapse in Speculative Decoding

2026-07-23

AI Summary

This work reveals a critical security vulnerability in speculative decoding stemming from misalignment between the draft and target models, which can cause verifier acceptance rates to collapse. To exploit this flaw, the authors propose ADSD, the first prompt-level adversarial attack tailored for speculative decoding. ADSD crafts adversarial suffixes that steer the draft model’s probability mass toward tokens unlikely to be accepted by the target model, thereby substantially degrading inference efficiency while preserving task output quality. The method introduces a novel Soft-Collapse surrogate objective jointly optimized with a task-fidelity constraint and employs an asymmetric acceptance rule to formulate a differentiable surrogate loss for gradient-based optimization. Experiments demonstrate that ADSD increases average sampling time by 62.3% on GSM8K and remains effective across diverse model architectures, domains, and speculation strategies, confirming the widespread nature of this vulnerability.

Abstract

Lossless acceleration schemes, such as speculative decoding, promise significant inference speedups by relying on dynamic token-level alignment between a draft and a target model. However, this guarantee of semantic equivalence masks a severe operational vulnerability: draft-target alignment can be systematically attacked. In this paper, we introduce ADSD, which, to the best of our knowledge, is the first prompt-suffix attack that collapses verifier acceptance by pushing draft probability mass toward tokens the target is unlikely to accept. ADSD uses Soft-Collapse, a verifier-aligned surrogate derived from the asymmetric speculative acceptance rule, together with a target-preservation objective that discourages obvious task corruption. ADSD successfully generates highly effective adversarial suffixes. On the GSM8K dataset, our attack increases the mean sample time by 62.3% while preserving the task quality. We further show that this vulnerability exists across different domains, speculative decoding strategies, and model architectures.

Problem

• adversarial prompts
• speculative decoding
• acceptance collapse
• verifier alignment
• inference acceleration

Innovation

• adversarial prompts
• speculative decoding
• acceptance collapse
• Soft-Collapse
• verifier-aligned surrogate