Analyzing Speech Condition Effects in Dysarthric ASR: A Layer-wise Probing Study

2026-08-03

AI Summary

This study addresses the significant degradation in automatic speech recognition (ASR) performance caused by dysarthric speech and investigates how such impairments affect internal representations within Transformer-based ASR encoders—a mechanism previously unclear. Focusing on Mandarin dysarthric speech, the work establishes, for the first time, a link between encoder representation depth and task difficulty through layer-wise probing analyses, revealing that higher-layer representations remain relatively robust to dysarthria. Building on this insight, the authors propose a layer-aware, parameter-efficient fine-tuning strategy: fine-tuning only the 7th layer with LoRA or layers 5–8 achieves 96.5% and 97.5%, respectively, of full-encoder fine-tuning performance while substantially reducing computational cost. The approach is validated via zero-shot TTS resynthesis and tone-sensitive evaluation, demonstrating the marginal yet meaningful benefit of adapting higher layers for dysarthric speech recognition.

Abstract

Automatic speech recognition (ASR) performance degrades sharply on dysarthric speech, yet how disordered articulation reshapes a model's internal representations is underexplored. We present a layer-wise probing analysis of a transformer ASR encoder on Mandarin dysarthric speech under three transcript-matched conditions: original dysarthric speech, speaker conditioned zero-shot TTS resynthesis, and unconditioned TTS. The probes reveal a task-dependent hierarchy: phoneme boundary information stays weak for dysarthric speech at every layer, phoneme identity becomes recoverable toward the upper layers, and recognition difficulty is encoded in the deepest layers. Tone-sensitive evaluation shows Mandarin lexical tone is a persistent error source. Cross-condition similarity divergence grows with depth, indicating that disordered speech affects high-level representations more than low-level acoustic features. Guided by these findings, single-layer LoRA at layer 7 and adaptation on subset layers 5-8 achieve performance within 3.5% and 2.48% relative margins of full encoder adaptation, respectively, while upper-layer adaptation is less effective for dysarthric speech. These findings link representation analysis to parameter-efficient fine-tuning and motivate layer-aware adaptation for low-resource Mandarin dysarthric ASR.

Problem

• dysarthric speech
• automatic speech recognition
• layer-wise probing
• Mandarin tone
• representation analysis

Innovation

• layer-wise probing
• dysarthric ASR
• parameter-efficient fine-tuning
• representation analysis
• Mandarin speech