speaker diarization

This work addresses the underexplored problem of anonymizing only the target speaker—e.g., the customer in call-center dialogues—within multi-speaker conversational speech. We propose the first fine-grained target-speaker anonymization framework for multi-speaker audio. Our method integrates speaker separation, target-directed voiceprint feature perturbation, and context-aware speech reconstruction to precisely conceal the identity of the specified speaker. Innovatively, we design a joint privacy-utility evaluation metric to overcome the limitations of existing metrics in multi-speaker scenarios. Experiments demonstrate that our approach significantly enhances target-speaker identity protection—reducing privacy leakage by over 60%—while preserving speech intelligibility and conversational coherence. Degradations in speech quality (measured by PESQ) and ASR accuracy remain within acceptable bounds, confirming a favorable privacy–utility trade-off.

To address the inconsistency and scene mismatch arising from the decoupled treatment of speaker extraction and diarization in complex overlapping speech, this paper proposes the first end-to-end jointly optimized framework that unifies frequency-domain speech separation with time-domain speaker activity annotation. The method integrates deep clustering, mask estimation, speaker activity detection, and waveform-level separation modules, supporting variable numbers of speakers and arbitrary overlap ratios. A bidirectional协同 mechanism enables mutual enhancement between extraction and diarization, breaking away from conventional cascaded pipelines. Evaluated on LibriMix, SparseLibriMix, and the real-world telephone conversation dataset CALLHOME, the approach achieves significant improvements over state-of-the-art methods on both tasks—marking the first demonstration of simultaneous gains in extraction quality (e.g., SI-SNRi) and diarization accuracy (e.g., DER).

To address dual privacy leakage—speaker identity and semantic content—in speech, this paper proposes the first end-to-end prompt-driven speech anonymization framework. Methodologically, it achieves speaker de-identification via controllable speaker descriptors, identifies and replaces sensitive semantic content using named entity recognition and large language models, and synthesizes high-fidelity anonymous speech via text-to-speech. Its key contributions are: (1) the first joint, controllable anonymization of both identity and semantics; (2) a descriptor modulation mechanism that explicitly models the privacy–utility trade-off; and (3) systematic analysis revealing how descriptor selection induces inherent biases between privacy protection and speech usability. Experiments demonstrate strong efficacy: speaker identification accuracy drops below 15%, while ASR accuracy remains above 92% and MOS for naturalness reaches ≈4.1—confirming both effectiveness and practicality.

Existing voice anonymization methods primarily target single-speaker scenarios and exhibit insufficient privacy protection in multi-speaker settings—especially under speech overlap. This work presents the first systematic study of multi-speaker voice anonymization, establishing the first dedicated benchmark encompassing task formalization, evaluation protocols, baseline systems, and privacy leakage analysis. We propose a dialogue-level dual-objective vector anonymization framework that simultaneously preserves inter-speaker relational structure and enhances individual speaker distinguishability. Technically, it integrates spectral-clustering-based speaker diarization, disentangled anonymization, selective anonymizers, and two dialogue-level speaker vector optimization strategies. Experiments on both non-overlapping simulated and real-world datasets demonstrate substantial reductions in privacy leakage risk, alongside improvements in speech intelligibility and speaker discriminability.

This study addresses severe emotional distortion in disentangled speaker anonymization. We propose a novel method that jointly ensures strong privacy protection and high emotional fidelity. Methodologically, we (1) design a pre-trained emotion encoder to explicitly disentangle emotion representations from speaker identity; (2) introduce the first SVM-based emotion boundary modeling and directional embedding-space compensation mechanism, enabling fine-grained adjustment of anonymized speaker embeddings along emotion gradients; and (3) integrate dual-path compensation—encoder-level feature fusion and post-hoc refinement. Experiments demonstrate that our approach achieves state-of-the-art speaker unidentifiability (ASR-based ID error >95%) while improving emotion recognition accuracy by 12.7% over existing disentanglement-based anonymization methods. Moreover, the framework is extensible to controllable preservation of other paralinguistic attributes, such as age and accent.

This work addresses the challenge of accurately modeling turn-taking in multi-party conversations, where overlapping speech and rapid speaker switches complicate dialogue understanding. The authors propose a purely audio-based, two-stage pipeline: a fast trigger first identifies candidate turn boundaries, followed by a lightweight verifier that determines whether the current speaker retains the floor or yields it, while also predicting the next speaker. A key innovation lies in decoupling turn-boundary detection from turn-holding decisions and introducing a label-preserving diffusion model to synthesize background-augmented training data. Evaluated on the VoxConverse dataset, the method significantly improves turn-yield detection performance, with the diffusion-based data augmentation strategy providing further measurable gains.

This work addresses the significant performance degradation of existing voice anonymization systems—primarily designed for adult speech—when applied to children’s voices, which struggle to balance privacy preservation with speech utility. To bridge this gap, the study proposes the first systematic self-supervised learning (SSL)-based anonymization framework tailored specifically for child speech. By performing domain adaptation on the MyST child corpus and extending the approach to both single-speaker and two-speaker mixture scenarios, the method integrates target speaker extraction with explicit modeling of child-specific vocal characteristics. This enables effective identity privacy protection while substantially improving speech intelligibility, perceptual quality, and conversational naturalness, thereby achieving child-centric, high-fidelity voice anonymization.

This work proposes the first streaming speaker anonymization framework for real-time speech applications that integrates a neural audio codec (NAC) with a causal language model (LM), addressing the dual challenges of low latency and privacy preservation. By leveraging pseudo-speaker representation sampling, speaker embedding mixing, and diverse prompt selection, the framework enables flexible trade-offs between privacy and latency under either dynamic or fixed delay constraints. The disentangled nature of quantized content codes effectively suppresses speaker information leakage. Evaluated under the VoicePrivacy 2024 protocol, the method reduces word error rate by 46% and improves unweighted average recall (UAR) for emotion recognition by 28% compared to DarkStream, while achieving an end-to-end latency of only 180 ms and offering strong privacy guarantees against lazy adversaries.

This study addresses the lack of effective cross-call speaker clustering methods for anonymized voice data in insurance fraud investigations. To bridge this gap, the authors propose DG⁢VoiC, a novel framework that, for the first time, integrates sensitive information-aligned voice anonymization with telephony-oriented speaker clustering. The approach leverages voice-guided preprocessing, sliding-window embedding extraction, and cosine similarity-based clustering to enable speaker identity linkage under strict anonymity constraints. Evaluated on a real-world dataset comprising 56 call segments from 22 speakers, DG⁢VoiC achieves exceptional performance—96% Adjusted Mutual Information (AMI), 95% Adjusted Rand Index (ARI), 98% completeness, 100% homogeneity, and 99% V-measure—demonstrating its significant potential to enhance fraud线索 detection capabilities in anonymized communication environments.

This work addresses the challenge of erroneous turn-taking prediction in multi-speaker scenarios caused by background speech interference. The authors propose a real-time architecture that integrates dominant speaker tracking with a hierarchical causal end-to-end turn-taking (EOT) prediction framework. The system leverages dominant speaker voice activity segmentation and multi-scale future state probability forecasting, enhanced by task-oriented knowledge distillation from wav2vec 2.0 to a lightweight MFCC-based student model. This approach achieves high performance while substantially reducing model size and latency. Experimental results demonstrate a frame-level F1 score of 82%, a response-word detection F1 of 70.6%, and a binary turn-end classification F1 of 69.3%. The end-to-end turn detection achieves a recall of 87.7% with a median latency of only 36 ms and a model size of merely 1.14 million parameters, outperforming existing Transformer-based baselines.