forced alignment

Existing forced alignment tools produce only point estimates of segment boundaries without quantifying uncertainty. To address this limitation, we propose the first confidence interval estimation method for forced alignment based on model ensembling and order statistics: ten independently trained segment classification neural networks are aggregated; boundary predictions are centered at the median, and 97.85% confidence intervals are constructed via order statistics. The method supports Praat TextGrid point-tier output and provides an interpretable boundary diagnostic table. This work is the first to integrate neural network ensembling with order statistics for uncertainty modeling in forced alignment. Evaluated on the Buckeye and TIMIT corpora, our approach achieves marginally higher boundary accuracy than single-model baselines while enabling uncertainty-aware linguistic analysis. It has been deployed in real-world speech processing pipelines, facilitating robust, uncertainty-informed phonetic and phonological modeling.

In speech emotion recognition, severe temporal misalignment between ASR transcripts and speaker diarization (SD) timestamps critically undermines the reliability of multimodal systems in conversational settings. To address this, we propose an end-to-end timestamp alignment pipeline that dynamically calibrates temporal boundaries between ASR and SD outputs, integrating cross-attention fusion and a gating mechanism to achieve precise inter-modal synchronization. The method leverages pre-trained Wav2Vec (audio), RoBERTa (text), and an SD model, requiring no additional annotations for fine-grained temporal alignment. Experiments on IEMOCAP demonstrate a 3.2% absolute improvement in emotion classification accuracy over unaligned baselines. This work constitutes the first systematic empirical validation of the critical role of temporal synchronization in multimodal emotion analysis, establishing a new benchmark for robust, time-aware multimodal modeling in dialogue scenarios.

This study investigates the trade-off between efficiency and accuracy in semi-automatic transcription for spoken language corpus construction. Through a two-stage experiment, it compares the performance of expert and novice transcribers on three types of Italian conversational data under both manual and ASR-assisted conditions. The work proposes an integrated analytical framework combining word-level alignment, quality evaluation metrics, and statistical modeling to systematically quantify behavioral differences across transcription workflows. Results demonstrate that ASR substantially increases transcription speed, yet its impact on accuracy varies depending on dialogue type, transcriber expertise, and workflow configuration. The findings provide empirical support for the development of the KIParla corpus, showing that a fine-tuned and optimized semi-automatic pipeline can effectively accelerate annotation while maintaining high transcription quality.

Current TTS systems suffer from limitations in speech naturalness, duration modeling, and audio coding—particularly exhibiting poor robustness to out-of-vocabulary (OOV) words and noisy text. To address these issues, we propose TTS-Transducer: the first end-to-end TTS framework integrating a robust neural transducer (RNN-T) with a neural audio codec featuring residual vector quantization (RVQ) and a monotonic alignment mechanism. This design enables implicit alignment between text and multi-codebook discrete speech tokens without explicit duration prediction. Furthermore, we introduce non-autoregressive residual codebook prediction and enable joint end-to-end training of the codec and transducer. Experiments demonstrate that TTS-Transducer achieves speech quality and naturalness comparable to state-of-the-art TTS systems, significantly improves robustness to OOV words and noisy input text, and eliminates the need for a separate duration model.

To address the high manual annotation cost and fragmented nature of existing automatic singing annotation (ASA) methods in constructing high-quality singing voice synthesis (SVS) datasets, this paper proposes the first unified ASA framework that jointly performs phoneme alignment, note transcription, vocal technique identification, and global style labeling. We introduce a novel non-autoregressive local acoustic encoder integrated within a hierarchical modeling architecture—spanning frames, phonemes, words, notes, and sentences—to learn structured, multi-granularity acoustic representations. Experiments demonstrate consistent superiority over state-of-the-art ASA methods across all annotation tasks. The fine-grained annotations generated by our framework significantly improve the naturalness and stylistic controllability of downstream SVS models. This work establishes both a high-quality annotated dataset and a robust methodological foundation for controllable singing voice synthesis.

This work addresses the challenge of conducting fine-grained, part-of-speech (PoS)-level error analysis in automatic speech recognition (ASR) for non-Latin script languages, where reliable word-level alignment is often unattainable. To overcome this limitation, the authors propose a language-agnostic automatic alignment framework that, for the first time, uniformly supports the three major writing systems—Abugida, Alphabetic, and Abjad. By integrating a general-purpose sequence alignment algorithm with standard PoS taggers, the framework enables a scalable and reproducible pipeline for PoS-level ASR error analysis. This approach effectively removes linguistic barriers in ASR diagnostics for non-Latin scripts and demonstrates practical utility: in multilingual experiments, insights derived from the analysis were successfully fed back into ASR training, yielding significant reductions in word error rate (WER).

Traditional automatic speech recognition evaluation metrics, such as word error rate (WER) and character error rate (CER), fail to capture human perception of errors and neglect linguistic and semantic influences. This work proposes a novel paradigm that embeds any perception-oriented evaluation metric into the minimum edit distance (minED) framework to produce an intuitively interpretable equivalent error rate. For the first time, this approach translates human perceptual modeling into a comprehensible error rate format, enabling quantification of error severity from the perspective of human understanding. The resulting metric not only aligns closely with human judgments but also effectively identifies recognition errors that critically impact semantic comprehension.

This work addresses the limitations of traditional phonetic annotation tools, which often require extensive manual correction or task-specific training data to accurately estimate key phonetic parameters such as voice onset time (VOT), closure duration, and burst realization. To overcome these challenges, the paper introduces wav2VOT, the first approach to directly leverage large-scale pretrained speech models like wav2vec 2.0 for fine-grained phonetic feature annotation. By employing segment-level regression and classification fine-tuning strategies, wav2VOT enables joint, high-precision estimation of these parameters. The method substantially reduces reliance on human intervention and labeled task-specific data, demonstrates strong generalization across unseen datasets, and maintains high predictive fidelity across varying voicing contrasts and places of articulation.

This study addresses the challenge posed by imprecise note onset annotations in weakly aligned score–audio data, which significantly limits the performance of automatic music transcription. It presents the first systematic analysis of the impact of onset “snapping” during cross-instrument transcription training and introduces a global context–aware optimization method. The approach formulates snapping as a pitch-wise assignment problem, constructing a bipartite graph from neural network posteriorgrams and dynamic time warping alignments, and replaces conventional greedy strategies with optimal matching within context-sensitive temporal windows. Evaluated on piano, chamber, and orchestral datasets, the method substantially improves both onset alignment accuracy and overall transcription performance, with particularly pronounced gains under wide snapping windows or coarse initial alignments.

This study addresses the challenge of achieving robust phonetic transcription in low-resource scenarios involving non-standard dialects and atypical speech—such as non-native and post-stroke dysarthric utterances—where expert phonetic annotation is prohibitively expensive. The authors systematically investigate the interplay between grapheme-to-phoneme (G2P) models and human supervision, revealing on an 80-hour multi-variety speech benchmark that G2P augmentation benefits performance only when fewer than 20–30 hours of manual transcriptions are available; beyond this threshold, it degrades cross-dialect generalization. To overcome this limitation, they propose replacing G2P with ASR-based pretraining and introduce a weighted phonetic feature error rate for evaluation. This approach yields substantial gains on both non-native and aphasic speech, reducing error rates by a factor of 2.3 compared to prior systems.