ModalFidelity: Routing Modalities for Deepfake Detection on a Budget

2026-09-29

AI Summary

This study addresses the computational inefficiency in deepfake detection caused by exhaustively reading entire audio-visual windows. To mitigate this, we propose a lightweight routing mechanism that pioneers a "route-before-detect" paradigm. By integrating multimodal attention routing with a hard computation budget control algorithm, our approach substitutes costly full-scale analysis with low-cost preview decisions, precisely identifying which modality streams require inspection under strict budget constraints to avoid redundant computation. Experimental results demonstrate that reading merely one-fifth of each window suffices to outperform conventional methods, reducing computational overhead by 15.9× while retaining over 96% of detection accuracy. This work achieves an exceptional balance between efficiency and performance for practical deepfake detection.

Abstract

Deepfakes no longer need to fake a whole video. Generators that read the transcript now alter only the few seconds in which a video's meaning turns, so a forgery hides in a small, unknown fraction of the video. Yet detectors still read every one-second window of both the audio and image streams, spending nearly all of their compute where nothing was altered. We observe that deciding where to look is far cheaper than looking. We present ModalFidelity, a lightweight router that previews each window and decides, before any forensic detector runs, which stream is worth reading, under a hard compute budget it can never exceed. On AV-Deepfake1M, reading at most a fifth of the windows, it is more accurate than gating after the detectors at 15.9x less compute, and retains over 96% of the accuracy of an oracle that knows where every forgery lies.

Problem

• Deepfake detection
• Compute budget
• Modality routing
• Video forensics

Innovation

• Deepfake Detection
• Modality Routing
• Compute Budget
• Lightweight Router
• Multimodal Forensics