Borrowed Strength: Best-of-N Search over a Code EncodingBreaks Self-Check Jailbreak Defenses

2026-07-29

AI Summary

This work addresses a critical vulnerability in existing self-checking defense mechanisms, such as SAGE, which claim up to 99% defense success but rely fundamentally on the target model’s own judgment, rendering them inherently fragile. We propose a novel composite attack that integrates code-completion encoding with Best-of-N search, effectively jailbreaking models without altering the original attack structure or requiring modifications to the attack formulation. Our analysis uncovers the mechanism behind the reversal of attack efficacy across different defense types and introduces a diagnostic method to identify when Best-of-N fails in deterministic attacks. Empirical evaluation across three open-source models achieves jailbreak success rates of 67%, 22%, and 15%, respectively, with significant effectiveness persisting even on a 70B-parameter model. All findings are rigorously validated over 310,000 generation trials.

Abstract

A self-check defense asks the target model to assess a request before answering it; SAGE, the strongest published instance, reports an average 99% defense success rate. We show it can be breached by composing two attacks that are individually harmless against it: an established code-completion encoding and an established best-of-N search, neither of which exceeds 4.7% of behaviors alone. Composed, with the search budget spent on the encoding, they reach 67/22/15% across three open targets, and the effect persists on a 70B target. We then explain the composition rather than only reporting it. First, a self-check defense borrows its strength from the target: SAGE does not detect the attack, it asks the model to, and the four targets convert that request into an explicit refusal between 32% and 97% of the time, which orders the spread in defended coverage even though undefended reach is near-identical. Second, which attack survives is decided by the type of defense, and it inverts: against transform defenses the code encoding retains far more of its undefended reach than the character search, while against gate defenses the ordering flips. We account for this with the number of independent probes an attack delivers to a defense's decision boundary. Finally, we report a validity defect we found and repaired in our own pipeline, a deterministic attack under greedy decoding has no best-of-N variation channel at all, and give the one-line diagnostic that detects it. All claims rest on 310,000 generations scored by a human-validated judge.

Problem

• self-check defense
• jailbreak
• code encoding
• best-of-N search
• adversarial attack

Innovation

• best-of-N search
• code encoding
• self-check defense
• jailbreak
• defense composition