CSULoRA: Closest Safe Update Low-Rank Adaptation

2026-05-28

AI Summary

This work addresses the vulnerability of Low-Rank Adaptation (LoRA) to minor adversarial or unsafe data during fine-tuning of large language models, which can compromise alignment. The authors propose a post-processing method that requires no additional training: by constructing a safety-aligned subspace based on weight displacement, the LoRA update is decomposed into distinct alignment components. A closed-form solution to a minimal-change optimization problem is then applied to smoothly suppress potentially unsafe directions. This approach achieves a significant reduction in adversarial attack success rates with only negligible adjustments, while preserving nearly all of LoRA’s task-specific performance gains, thereby effectively balancing safety and utility.

Abstract

Low-rank adaptation has become a standard method for parameter-efficient fine-tuning of large language models, but even small amounts of unsafe or adversarial fine-tuning data can substantially weaken the safety behavior of aligned models. Existing safety-preserving LoRA methods often rely on hard interventions such as projection, pruning, thresholding, or additional training objectives. While these methods can suppress unsafe update directions, they may also remove task-relevant information or require extra tuning. We introduce CSULoRA, a post-hoc method for correcting trained LoRA adapters through closest safe update estimation. CSULoRA estimates a safety-aligned subspace from the weight displacement between a safety-aligned model and its corresponding base checkpoint. It then decomposes each LoRA update into fully aligned, partially aligned, and off-subspace components. Instead of discarding components outside the estimated safety subspace, CSULoRA solves a closed-form penalized minimum-change problem that preserves the fully aligned component while smoothly attenuating potentially unsafe directions according to their relative energy. In adversarial fine-tuning experiments, CSULoRA substantially reduces attack success rate while preserving most of the utility gains obtained from standard LoRA fine-tuning.

Problem

• Low-rank adaptation
• Safety alignment
• Adversarial fine-tuning
• Parameter-efficient fine-tuning
• Unsafe updates

Innovation

• Low-Rank Adaptation
• Safety Alignment
• Post-hoc Correction
• Subspace Estimation
• Adversarial Robustness