🤖 AI Summary
This work addresses hallucination errors in end-to-end speech recognition for low-resource languages, which often arise from rare entities and domain-specific terms. Existing retrieval-augmented approaches struggle to balance recognition robustness with inference efficiency. To overcome this, the authors propose a lightweight, purely lexical framework that explicitly models speech recognition errors within a sparse retrieval mechanism. By integrating symmetric text normalization, an error-aware TF-IDF algorithm, and a sparse diagonal penalty matrix derived from historical errors, the method prioritizes the retrieval of correction documents containing high-risk misrecognized terms, thereby suppressing both phoneme-level and recurrent hallucination errors. Evaluated on the FLEURS Persian subset, the approach increases error-aware recall from 53.7% to 90.9% and reduces word error rate from 23.06% to 18.83%, all while introducing negligible inference latency.
📝 Abstract
End-to-end automatic speech recognition systems frequently hallucinate rare entities and domain-specific terms, especially in low-resource languages. While retrieval-augmented generation frameworks can mitigate these errors using large language models, current architectures face significant challenges. They either rely on standard sparse retrieval that ignores phonetic misrecognitions or utilize heavyweight cross-modal embeddings that introduce high latency. This letter proposes a highly efficient, purely lexical error-aware framework designed to explicitly resolve phonetic and loop hallucinations. Our approach integrates a symmetric text normalization module with a novel error-aware term frequency-inverse document frequency algorithm. By constructing a sparse diagonal penalty matrix based on historical errors, the retriever mathematically prioritizes corrective documents containing specific high-risk misrecognitions. Evaluated on the Persian subset of the FLEURS dataset, our method increased the error-aware hit rate from 53.7% to 90.9%. In end-to-end evaluations, the integrated framework reduced the final word error rate from 23.06% to 18.83%, achieving significant accuracy gains with near-zero inference latency.