LoRAX: LoRA eXpandable Networks for Continual Synthetic Image Attribution

2025-04-10

AI Summary

In the context of rapidly proliferating generative AI images, existing model attribution methods suffer from poor generalization to unseen architectures and impractical incremental updates. Method: This paper proposes a scalable class-incremental attribution framework based on Low-Rank Adaptation (LoRA), which trains a lightweight, architecture-specific feature extractor (≤3% parameters) for each newly generated model, enabling zero-forgetting continual learning. Contribution/Results: Evaluated on the Continual Deepfake Detection benchmark, our method achieves state-of-the-art or comparable performance across all training scenarios and memory constraints. It is the first to deeply integrate parameter-efficient fine-tuning with class-incremental learning, effectively addressing the core limitations of conventional attribution models—namely, weak generalization to unknown generator architectures and inability to dynamically evolve with emerging models—thereby significantly enhancing deployment sustainability and scalability.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

As generative AI image technologies become more widespread and advanced, there is a growing need for strong attribution models. These models are crucial for verifying the authenticity of images and identifying the architecture of their originating generative models-key to maintaining media integrity. However, attribution models struggle to generalize to unseen models, and traditional fine-tuning methods for updating these models have shown to be impractical in real-world settings. To address these challenges, we propose LoRA eXpandable Networks (LoRAX), a parameter-efficient class incremental algorithm that adapts to novel generative image models without the need for full retraining. Our approach trains an extremely parameter-efficient feature extractor per continual learning task via Low Rank Adaptation. Each task-specific feature extractor learns distinct features while only requiring a small fraction of the parameters present in the underlying feature extractor's backbone model. Our extensive experimentation shows LoRAX outperforms or remains competitive with state-of-the-art class incremental learning algorithms on the Continual Deepfake Detection benchmark across all training scenarios and memory settings, while requiring less than 3% of the number of trainable parameters per feature extractor compared to the full-rank implementation. LoRAX code is available at: https://github.com/mit-ll/lorax_cil.

Problem

• Attribution models struggle to generalize to unseen generative models
• Traditional fine-tuning methods are impractical for updating attribution models
• Need for parameter-efficient adaptation to new generative image models

Innovation

• Parameter-efficient class incremental algorithm
• Low Rank Adaptation for feature extraction
• Less than 3% trainable parameters needed