REIMU: Efficient Heterogeneous Hierarchical Reasoning for SSL-Based Speech Deepfake Detection

2026-08-01

AI Summary

This work addresses the growing threat of high-fidelity voice spoofing to media integrity and speaker verification systems by proposing a novel heterogeneous hierarchical recurrent inference architecture for deepfake detection based on self-supervised learning (SSL). The design integrates self-attention mechanisms in higher layers with linear attention in lower layers, achieving strong detection performance while reducing downstream parameters by 10.8%. Experimental results on the ASVspoof 2019 and 2021 evaluation sets demonstrate that the proposed method significantly outperforms conventional detection approaches with fewer parameters, thereby validating the efficacy and efficiency of the heterogeneous recurrent architecture.

Abstract

The increasing realism of speech generated by text-to-speech and voice conversion systems poses growing challenges to media integrity and voice authentication. Self-supervised learning (SSL) has substantially advanced speech deepfake detection, where downstream backbones conventionally process SSL representations through a single forward pass. This work investigates the practical effectiveness of recurrent hierarchical reasoning for this task. We term this controlled study REIMU and systematically compare conventional single-pass backbones, weight-shared recurrence, homogeneous HRM, and heterogeneous HRM across four Base-scale SSL frontends. We further examine heterogeneous high- and low-level modules that combine self-attention with linear attention. Experiments on the ASVspoof 2019 and 2021 evaluation sets show that recurrence and hierarchical decomposition do not inherently improve detection, whereas heterogeneous operator assignment provides a more competitive configuration. Notably, the heterogeneous design remains competitive while using 10.8\% fewer downstream parameters than the matched baseline, demonstrating its potential for parameter-efficient speech deepfake detection.

Problem

• speech deepfake detection
• self-supervised learning
• heterogeneous reasoning
• parameter efficiency
• hierarchical reasoning

Innovation

• heterogeneous hierarchical reasoning
• self-supervised learning
• speech deepfake detection
• parameter efficiency
• linear attention