Automatic Speech Recognition Biases in Newcastle English: an Error Analysis

2025-06-19

AI Summary

This study identifies a significant structural bias in automatic speech recognition (ASR) systems toward Newcastle English—a prototypical regional dialect—moving beyond prior bias research focused primarily on sociodemographic attributes (e.g., race, gender). Using expert linguistic annotation, systematic phonological, morphological, and syntactic contrastive analysis, and real-world corpus-driven error diagnosis, we demonstrate that misrecognition stems predominantly from inherent dialectal features (e.g., pronouns *yous*/*wor*), not speaker demographics. Quantitative analysis reveals a strong correlation between dialect feature density and word error rate. Our work fills a critical empirical gap in regional dialect bias research and advances ASR evaluation and debiasing frameworks by formally integrating dialectal diversity—providing both methodological rigor and actionable evidence for developing more inclusive speech technologies.

Technology Category

Natural Language Processing: Ethics — Bias, Fairness, Transparency & PrivacyMachine Learning: Ethics, Bias, and FairnessComputer Vision: Bias, Fairness & Privacy

Application Category

Search and Retrieval-Augmented AI: Web evaluation methodologies and metricsSocial Networks and Social Media: Fairness and bias in social network and social media analysisUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Automatic Speech Recognition (ASR) systems struggle with regional dialects due to biased training which favours mainstream varieties. While previous research has identified racial, age, and gender biases in ASR, regional bias remains underexamined. This study investigates ASR performance on Newcastle English, a well-documented regional dialect known to be challenging for ASR. A two-stage analysis was conducted: first, a manual error analysis on a subsample identified key phonological, lexical, and morphosyntactic errors behind ASR misrecognitions; second, a case study focused on the systematic analysis of ASR recognition of the regional pronouns ``yous''and ``wor''. Results show that ASR errors directly correlate with regional dialectal features, while social factors play a lesser role in ASR mismatches. We advocate for greater dialectal diversity in ASR training data and highlight the value of sociolinguistic analysis in diagnosing and addressing regional biases.

Problem

• ASR systems perform poorly on regional dialects like Newcastle English
• Regional bias in ASR lacks research compared to other biases
• Study analyzes phonological and lexical errors in ASR dialect recognition

Innovation

• Manual error analysis identifies ASR misrecognitions
• Case study on regional pronouns yous and wor
• Advocates dialect diversity in ASR training data