Pretrained ASR Pseudo-labeling for Noisy Police Audio

2026-09-24

AI Summary

This study addresses the performance degradation of pretrained ASR models on noisy police audio and the challenge of filtering low-quality pseudo-labels. To this end, we propose an external filtering paradigm leveraging LLM-as-a-Judge alongside a cross-model pseudo-label generation mechanism. Specifically, we evaluate the pseudo-label adaptability of Whisper and Qwen3-ASR on the BPC dataset, employing large language model-based evaluation to discard poor-quality transcriptions and introducing a cross-fine-tuning strategy for model optimization. Experimental results demonstrate that the proposed approach significantly reduces the word error rate (WER). Furthermore, our findings validate the superiority of the external filtering scheme over conventional internal metrics, establishing an effective pathway and identifying future research directions for robust speech recognition in highly noisy scenarios.

Abstract

Pretrained ASR systems perform poorly on noisy Broadcast Police Communication (BPC), hindering efforts to understand police decision-making. Pseudo-labeling offers an unsupervised path to improve ASR without expensive human labels, but the efficacy of this approach on very noisy domains is not known. In this work, we systematically assess the opportunities and limits of pseudo-labeling to adapt foundation ASR models (Whisper and Qwen3-ASR) to noisy BPC domain corpora from Baltimore and Chicago. We demonstrate that existing internal confidence metrics (log-probabilities and STAR scores) fail to distinguish between high and low quality BPC pseudo-labels, and we introduce an external LLM-as-a-judge filtering paradigm that leverages parametric knowledge to discard contextually implausible transcripts. Our LLM-judging filters more aggressively than internal metrics and significantly reduces WER of the pseudo-labeled training sets across the Baltimore and Chicago BPC corpora, though a substantial gap remains relative to an oracle filter. We also introduce a new cross-model pseudo-labeling paradigm where one model is finetuned with pseudo-labels from the other, and we identify this method as a promising direction for future pseudo-labeling work.

Problem

• Automatic Speech Recognition
• Pseudo-labeling
• Noisy Audio
• Broadcast Police Communication
• Foundation Models

Innovation

• Pseudo-labeling
• LLM-as-a-judge
• Cross-model pseudo-labeling
• Automatic Speech Recognition
• Noisy audio