Who Made This? Fake Detection and Source Attribution with Diffusion Features

2025-10-31

AI Summary

The rapid advancement of generative diffusion models has intensified challenges in detecting deepfakes and tracing their origins, as existing supervised detectors suffer from poor cross-generator generalization, heavy reliance on labeled data, and frequent retraining requirements. To address these limitations, we propose FRIDA—a novel framework that leverages frozen internal activation features from pre-trained diffusion models, enabling universal deepfake detection and generator attribution without fine-tuning. FRIDA exploits inherent generator-specific patterns embedded in diffusion features: it employs a k-nearest neighbors classifier for binary forgery detection and a lightweight neural network for fine-grained source identification. Extensive experiments demonstrate that FRIDA achieves state-of-the-art performance on cross-generator detection and significantly outperforms existing supervised methods in generator attribution accuracy. Moreover, it exhibits strong generalization across unseen generators, offers interpretable decisions via feature-space proximity, and maintains deployment efficiency with minimal computational overhead.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The rapid progress of generative diffusion models has enabled the creation of synthetic images that are increasingly difficult to distinguish from real ones, raising concerns about authenticity, copyright, and misinformation. Existing supervised detectors often struggle to generalize across unseen generators, requiring extensive labeled data and frequent retraining. We introduce FRIDA (Fake-image Recognition and source Identification via Diffusion-features Analysis), a lightweight framework that leverages internal activations from a pre-trained diffusion model for deepfake detection and source generator attribution. A k-nearest-neighbor classifier applied to diffusion features achieves state-of-the-art cross-generator performance without fine-tuning, while a compact neural model enables accurate source attribution. These results show that diffusion representations inherently encode generator-specific patterns, providing a simple and interpretable foundation for synthetic image forensics.

Problem

• Detecting fake images created by generative diffusion models
• Attributing synthetic images to their source generators
• Overcoming generalization issues in existing deepfake detectors

Innovation

• Leverages pre-trained diffusion model activations for detection
• Uses k-nearest-neighbor classifier on diffusion features
• Employs compact neural model for source generator attribution