ThinkOmni: A Reasoning-Driven Omni-Modal LLM Framework for Audio Forgery Detection and Localization

2026-07-29

AI Summary

This work addresses the limited generalizability of existing audio forgery detection methods, which often overfit to dataset-specific artifacts and struggle with subtle, localized, and previously unseen manipulations. To overcome this, the paper introduces the first inference-driven, full-modality large language model framework that jointly performs explicit forensic reasoning, forgery detection, and temporal localization. The proposed approach integrates a forensic-aware chain-of-thought mechanism, modality-progressive alignment learning, and a multi-task consistency loss to guide decision-making through structured forensic evidence, yielding interpretable and highly generalizable audio forgery analysis. Experimental results demonstrate that the method significantly outperforms existing approaches in cross-dataset settings, achieving superior detection and localization performance while exhibiting strong robustness and generalization capabilities.

Abstract

Existing audio forgery detection and localization (AFDL) methods often overfit dataset-specific low-level artifacts, limiting their generalization to subtle, localized, and unseen manipulations. Recent audio large language model (ALLM)-based approaches cast AFDL as question answering but still model forensic evidence implicitly, without linking manipulation cues to predictions. To bridge this gap, we propose ThinkOmni, a reasoning-driven omni-modal large language model that jointly performs explicit forensic reasoning, spoofing detection, and temporal manipulation localization. To enable explicit reasoning supervision, we construct Forensic-Aware Chain-of-Thought (FACoT), a 100K-sample dataset with structured forensic evidence and reasoning annotations. Leveraging FACoT, we introduce Forensic-Aware Modality-Incremental Learning (FMIL), which progressively aligns semantic, acoustic, and spectral-visual representations with the LLM backbone to capture complementary forensic cues. We further propose Forensic-Consistent Multi-task Loss (FCML), which combines weighted cross-entropy with an adaptive localization loss to coordinate reasoning generation, spoofing detection, and temporal localization. Extensive experiments show that ThinkOmni achieves strong cross-dataset generalization in both detection and localization. Code, models, data, and inference examples are available at https://beyond0814.github.io/ThinkOmni/.

Problem

• audio forgery detection
• localization
• generalization
• forensic reasoning
• manipulation cues

Innovation

• reasoning-driven
• omni-modal
• audio forgery detection
• forensic-aware learning
• multi-task localization