pause detection

Traditional clinical rating scales for assessing formal thought disorder (FTD) in schizophrenia-spectrum disorders are resource-intensive and difficult to scale. Existing automated speech analysis approaches predominantly rely on single-modality features and fail to jointly model temporal dynamics (e.g., pause patterns) and semantic coherence. To address this, we propose the first systematic multimodal framework integrating ASR-derived pause dynamics—such as pause frequency and duration distribution—with semantic coherence metrics computed via pretrained language models. The framework is designed to be robust across diverse clinical contexts. We employ support vector regression (SVR) for feature fusion and evaluation on the TOPSY dataset yields a correlation coefficient of ρ = 0.649 for FTD severity prediction and an AUC of 83.71% for detecting severe cases—both significantly outperforming unimodal baselines. This work establishes a scalable, objective, and quantitatively grounded methodology for automated assessment of speech disorganization in psychosis.

This paper addresses the lack of an interdisciplinary-compatible annotation framework for conversational turn-taking. We propose a two-layer temporal alignment framework—comprising Inter-Pausal Units (IPUs) and Prosodic Compositional Units (PCOMPs)—that jointly satisfies sequential requirements from conversation analysis, temporal precision for speech modeling, and computational tractability for machine learning. Applied to 95 minutes of naturalistic dialogue from the GRASS corpus, the framework employs Praat for millisecond-level temporal alignment, integrates hierarchical manual annotation with sequence-based rules, and achieves high inter-annotator agreement (Cohen’s κ > 0.95 for IPUs; κ ≈ 0.75 for PCOMPs). To our knowledge, this is the first systematic integration of theoretical rigor, phonetic-analytic fidelity, and automatic classification feasibility into a standardized, highly consistent, reusable, and extensible protocol. Accompanying open-source annotated data and a detailed annotation guide further support cross-disciplinary research and applications at the intersection of linguistics and artificial intelligence.

This study addresses the underrepresentation of prosodic phrasing in spontaneous speech synthesis by systematically investigating the impact of manual versus automatic prosodic segmentation on non-autoregressive Brazilian Portuguese speech synthesis (FastSpeech 2). Using an open-source dataset licensed under CC BY-NC-ND 4.0, it presents the first comparative evaluation of these two annotation approaches regarding intonation modeling, pause control, and fluency enhancement. Results show that explicit prosodic segmentation yields modest improvements in intelligibility and acoustic naturalness. Both methods successfully reproduce core accent patterns; however, manual annotation—by preserving greater prosodic variability—significantly outperforms automatic segmentation in nuclear pitch contour fidelity and prosodic diversity. This work provides empirical evidence and methodological guidance for fine-grained prosodic modeling in spontaneous speech synthesis.

This study addresses the lack of effective deep learning approaches for automatic prosodic boundary detection in Brazilian Portuguese by proposing the first application of the Whisper large-v3 model to this task. Through fine-tuning, the model enables end-to-end simultaneous prediction of speech transcription and prosodic boundary labels. The approach integrates multimodal cues and incorporates a test-time filtering strategy alongside n-gram and acoustic visualization analyses to systematically evaluate the model’s sensitivity to syntactic, semantic, and prosodic features. Evaluated on the NURC-SP test set, the method achieves an F1 score of 0.731, and demonstrates strong generalization with an F1 score of 0.796 on the out-of-domain MuPe-Diversidades dataset, significantly outperforming existing approaches.

This work addresses the insufficient syntactic sensitivity of text-to-speech (TTS) systems in prosodic phrase boundary prediction—particularly for syntactically ambiguous constructions such as garden-path sentences—where models over-rely on punctuation and neglect latent syntactic cues. To rigorously assess TTS models’ syntactic awareness, we introduce a psycholinguistic evaluation paradigm. We propose a punctuation-agnostic fine-tuning strategy that compels models to infer implicit syntactic structure. Our methodology integrates controlled fine-tuning of pretrained TTS models, construction of a syntactically annotated prosodic boundary dataset, development of a human-validated prosody labeling protocol, and design of a contrastive ambiguity analysis framework. Results demonstrate significantly improved syntactic consistency in prosodic boundary placement for complex sentences: fine-tuned models better reflect constituent-level syntactic structure and markedly reduce punctuation dependency. This work provides both a novel methodological framework and empirical evidence for enhancing TTS naturalness and alignment with linguistic structure.

This study investigates the distribution of silent gaps in human and AI-generated spoken interactions and their role in turn-taking. Employing a distant viewing methodology, the authors analyze 30 American sitcoms and 51 synthetic podcasts generated by Google NotebookLM—the first application of this approach to compare silence thresholds between human and AI speech. Using Praat, speaker gender is automatically annotated based on fundamental frequency, and silence intervals in audiovisual materials are detected and statistically analyzed. The findings reveal that the duration distribution of silences in AI-generated content significantly differs from that in human dialogue, and that both speaker gender and production context systematically influence silence thresholds, thereby uncovering distinctive characteristics of generative audio in conversational turn structure.

Current spoken dialogue systems lack interpretable prosodic evaluation methods that adapt to speaker characteristics and interaction states. This work addresses this gap by proposing a condition-matched human reference benchmark and a percentile-based evaluation protocol, leveraging over 4,000 hours of dyadic English conversational data to enable fine-grained analysis of acoustic prosodic features such as fundamental frequency, speech rate, and pause duration. By incorporating hierarchical matching and out-of-bounds flagging mechanisms, the method significantly enhances behavioral plausibility and interpretability of prosodic assessments. Validation on held-out human data demonstrates an anomaly flagging rate close to the nominal 10%, outperforming conventional aggregate statistics while clearly indicating the direction of deviation, thereby providing an effective behavioral plausibility check for synthetic speech.

This study addresses key challenges in real-world speech prosody collection—namely semantic confounding, privacy risks, and low participant compliance—by introducing the first field-deployable protocol that simultaneously ensures content control, privacy preservation, and scalability. The proposed approach employs standardized scripts calibrated for emotional valence to constrain semantic content, leverages on-device smartphone processing to extract prosodic features in real time, and immediately discards raw audio after feature derivation, uploading only the anonymized features. In a large-scale deployment involving 560 participants, the protocol successfully yielded 9,877 high-quality recordings. These data effectively supported accurate prediction of speaker gender and momentary affective states (valence and arousal), thereby validating both the data quality and practical utility of the proposed framework.

This study addresses the challenge of early Alzheimer’s disease (AD) detection in settings where neuroimaging or speech transcription tools are unavailable. The authors propose the first lightweight, speaker-independent screening method that operates solely on raw spontaneous speech without requiring transcriptions. Their approach employs WebRTC for voice activity detection and manually extracts a 99-dimensional acoustic-temporal feature set encompassing pauses, fluency, spectral characteristics, prosody, and MFCCs along with their dynamic derivatives. Using an RBF kernel SVM classifier evaluated on the DementiaBank Pitt Corpus under 30 speaker-independent splits, the method achieves an average AUC of 0.674, with a best single-split AUC of 0.742 (accuracy: 0.657), thereby demonstrating the feasibility of detecting AD using only untranscribed audio.

Existing evaluation methods for audio captioning struggle to accurately assess the fidelity of multimodal semantics and acoustic attributes in structured audio descriptions. This work proposes the first multi-axis evaluation framework tailored for structured audio captioning, integrating large language model (LLM)-based semantic judgments with deterministic acoustic metrics across five orthogonal dimensions: label sets, descriptive content, logical reasoning, numerical measurements, and spectral contours. The framework incorporates a controlled perturbation protocol to validate its ability to distinguish between semantic preservation and acoustic distortion. Experiments on the AudioCards dataset demonstrate that the proposed approach effectively differentiates semantically consistent paraphrases from genuine errors, significantly outperforming existing methods in both reliability and sensitivity.