Frequency-Aware Continual Learning for Smart Contract Vulnerability Detection with Large Language Models

2026-08-20

AI Summary

该研究针对智能合约漏洞检测中大型语言模型面临的参数高效适应、灾难性遗忘及多适配器整合问题，提出了一种包含频率感知低秩适应、遗忘感知重放和锚点保护逐步合并的三阶段解决方案。

Abstract

Smart contract vulnerability detection with Large Language Models (LLMs) faces three causally linked challenges. First, new vulnerability categories demand parameter-efficient adaptation, since full retraining is prohibitive for sequentially arriving tasks. Second, training per-task adapters on a shared backbone causes catastrophic forgetting of previously learned vulnerabilities. Third, the resulting multiplicity of adapters must be consolidated into a single model, since task identity is unknown at inference time. Each challenge arises directly from the solution to its predecessor, making an integrated framework essential. We propose a three-stage pipeline in which each stage addresses one challenge and feeds into the next. The adaptation stage uses Frequency-Aware Low-Rank Adaptation (FA-LoRA), which performs adaptation in the Fourier domain with per-frequency importance gates, requiring only 0.4% trainable parameters while outperforming standard LoRA and QLoRA. The continual learning stage applies Forget-Aware Replay (FAR), which uses these frequency gates to estimate per-sample forgetting risk via loss dynamics and prioritizes vulnerable knowledge for rehearsal, achieving an average Micro-F1 of 0.8022 across sequential tasks. The deployment stage employs Anchor-Protected Progressive Merging (APPM), which exploits the asymmetric generalization produced by FAR training to identify the strongest-generalizing adapter as an anchor and consolidates all adapters into a single model via anchor-protected weighted merging with frequency-domain gate competition. APPM achieves a Micro-F1 of 0.8085, within 2.7% of the independent per-task upper bound, at a merge cost of 156 ms and no additional runtime memory. Experiments on DIVE confirm the framework effectively addresses all three challenges for evolving blockchain ecosystems.

Problem

• Continual Learning
• Smart Contract Vulnerability Detection
• Large Language Models
• Catastrophic Forgetting
• Adapter Consolidation

Innovation

• Frequency-Aware Low-Rank Adaptation (FA-LoRA)
• Forget-Aware Replay (FAR)
• Anchor-Protected Progressive Merging (APPM)