When Can You Prune Your Network? A Study of Intermediate Neurons in Multilingual Speech Parsing

2026-10-08

AI Summary

This study investigates the role of intermediate neural networks in multilingual speech parsing, which are commonly regarded as essential for bridging the representational gap between pretrained encoders and downstream tasks. Through systematic analysis, we reveal that such intermediate layers are beneficial only when the pretrained encoder remains frozen. Accordingly, this work proposes a streamlined end-to-end architecture that eliminates the intermediate network entirely, validated across multilingual scenarios including French and Slovenian. Experimental results demonstrate that the proposed architecture matches or surpasses existing methods on both automatic speech recognition and syntactic parsing tasks, proving particularly effective for low-resource languages. Notably, it achieves a 12% parameter reduction while simultaneously improving performance. These findings establish a new paradigm for constructing efficient multilingual speech parsing models.

Abstract

End-to-end speech parsing, a task recently proposed, consists in predicting both the transcription and the syntactic tree for a spoken utterance. Existing architectures for speech parsing often utilise intermediate neural networks. In this work, we examine the effectiveness of intermediate neural networks (NN) for parsing, and, specifically, what role do they play. We introduce a simpler end-to-end architecture for speech parsing, where we remove these intermediate NN units, reducing the parameters by 12%, while achieving comparable or better performance than prior method on both automatic speech recognition (ASR) and parsing. We demonstrate that intermediate NN units help reduce the representational gap when the pre-trained encoder is frozen. We do a comprehensive evaluation of speech parsing on French, and medium-low resource languages Slovenian and Naija. We further investigate the impact of the training data size and intermediate layers of the pretrained speech encoder on speech parsing.

Problem

• speech parsing
• intermediate neurons
• multilingual
• end-to-end
• pruning

Innovation

• End-to-end speech parsing
• Network pruning
• Intermediate neurons
• Multilingual
• Representational gap