Uncertainty-Aware Deepfake Detection via Multi-View Structural Learning

2026-07-30

AI Summary

This work addresses the overconfidence of existing deepfake detection methods on out-of-distribution data and their inability to provide reliable confidence estimates, which limits their applicability in safety-critical scenarios. To overcome this, the authors propose a multi-view structural learning framework that integrates complementary visual, semantic, and structural evidence. By leveraging inconsistencies among evidential streams to identify forgeries, the method introduces an Inter-Branch Disagreement Calibration mechanism that explicitly links prediction uncertainty to conflicts across multiple evidence sources, enabling well-calibrated and selective predictions. Combining an enhanced CLIP encoder, differentiable semantic constraints, and class-dependent structural relationship modeling, the approach achieves state-of-the-art generalization performance across multiple out-of-distribution benchmarks after training on FaceForensics++, significantly improving both calibration quality and selective prediction accuracy.

Abstract

Security-critical biometric and forensic applications require accurate predictions and reliable confidence estimates, particularly under distribution shift. This challenge is especially acute for deepfake detection, where foundation-model-based detectors often exhibit overconfident predictions on out-of-distribution manipulations, which limits their suitability for operational deployment. We propose an uncertainty-aware deepfake detection framework that identifies manipulations through inconsistencies across complementary evidence sources. The framework integrates three streams: a visual stream based on an adapted CLIP encoder, a semantic stream that models consistency among facial attributes through differentiable constraints, and a structural stream that captures class-dependent dependency patterns between semantic and forensic features. To effectively combine these signals, we introduce Inter-Branch Disagreement Calibration (IBDC), a disagreement-aware uncertainty modeling mechanism that links predictive uncertainty to conflicts among evidence streams. Extensive cross-dataset experiments using FaceForensics++ as the training source demonstrate that the proposed framework achieves state-of-the-art generalization across multiple out-of-distribution benchmarks while consistently improving calibration and selective prediction performance. These results show that combining complementary evidence with disagreement-aware uncertainty provides a robust foundation for trustworthy and well-calibrated deepfake detection under distribution shift.

Problem

• deepfake detection
• distribution shift
• uncertainty estimation
• out-of-distribution
• model calibration

Innovation

• uncertainty-aware
• multi-view structural learning
• deepfake detection
• distribution shift
• Inter-Branch Disagreement Calibration