Tight Boundary Prediction in Speaker Diarization Using Causal-Anticausal Consistency

2026-06-10

AI Summary

This work addresses the challenge of obtaining precise speech boundaries in speaker diarization models trained under loosely annotated data, where annotations often exhibit imprecise temporal boundaries. To mitigate this issue, the authors propose a novel approach based on causal and anticausal consistency. Specifically, they construct paired causal and anticausal neural networks to generate compact pseudo-labels and introduce an iterative co-training framework that jointly optimizes both the model parameters and the pseudo-labels. This mechanism effectively prevents the model from learning the boundary slackness inherent in the original annotations. Notably, the method is the first to leverage causal–anticausal consistency to recover accurate boundaries solely from loose annotations, achieving approximately 70% of the performance attainable with ideal tight labels and yielding significant improvements in downstream tasks.

Abstract

Multi-talker conversational automatic speech recognition data are often used to train speaker diarization models. Because such data prioritize semantic continuity, pauses and boundary margins are included within speech segments, resulting in loose annotations. Models trained on such data tend to internalize mechanisms that reproduce this looseness, although tight speech intervals are sometimes preferable for downstream applications. In this paper, we address the novel task of enabling models to produce tight predictions using loose labels. Our method generates tighter pseudo labels using causal and anticausal models, which are inherently incapable of learning loosening behavior. We further propose a co-training scheme that iteratively tightens labels and updates both models for more progressive refinement. Experimental results show that the proposed method recovers about 70 % of the tightening effect achieved by ideal tight-label training and improves downstream performance.

Problem

• speaker diarization
• tight boundary prediction
• loose labels
• speech segmentation
• boundary refinement

Innovation

• causal-anticausal consistency
• tight boundary prediction
• pseudo-label refinement
• speaker diarization
• co-training