Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates

2026-07-30

AI Summary

This work addresses the prohibitive computational cost of adversarial training for large language models, which hinders scalability. To overcome this challenge, the authors propose a co-optimization framework that jointly enhances defense and attack efficiency. On the defense side, they employ representation fine-tuning (ReFT) and resolve its misalignment with adversarial objectives. On the attack side, they construct a lightweight proxy model by extracting critical circuits, thereby circumventing full-model forward and backward passes. This approach uniquely integrates low-rank defense with circuit-guided lightweight attack generation, achieving strong robustness at substantially reduced computational expense. Experiments demonstrate that, compared to standard latent adversarial training (LAT), the proposed method reduces FLOPs by 48.1% in single-step adversarial training while requiring only 0.0118% of trainable parameters.

Abstract

Adversarial training is one of the most effective defenses against adversarial attacks, yet the computational cost remains prohibitive at modern scales, especially for large language models (LLMs). While existing mitigation strategies, e.g., latent adversarial training (LAT), have been developed, they still incur a high computational cost. In this work, we comprehensively investigate computation-efficient strategies to speed up LAT from two complementary perspectives: (1) Defense-side optimization: We explore the representation fine-tuning (ReFT) within LAT, and reveal a potential issue if there is a mismatch on which tokens to apply ReFT and the attack. (2) Attack-side optimization: When computing adversarial attacks in each LAT iteration, we extract only the relevant circuits from the LLM to construct a lightweight surrogate model, avoiding the computation in the forward-backward passes through the full model during the attack generation. For both perspectives, we provide theoretical justifications and numerical evidence to illustrate the effectiveness of the proposed strategies. Ultimately, compared to standard LAT with full fine-tuning, our method on average reduces per-step adversarial-training FLOPs by 48.1% while requiring only 0.0118% trainable parameters.

Problem

• adversarial training
• large language models
• computational cost
• latent adversarial training
• efficiency

Innovation

• low-rank defense
• circuit-guided surrogates
• adversarial training
• representation fine-tuning
• large language models