Ontology Memory-Augmented ASR Correction for Long Text-Speech Interleaved Conversations

2026-06-11

AI Summary

This work addresses the challenge of effectively leveraging sparse, redundant, and noisy contextual information in long-form speech-text interleaved dialogues for automatic speech recognition (ASR) error correction. The authors propose an ontology memory–enhanced ASR correction framework that introduces, for the first time, a dynamically updatable structured ontology memory to organize and retrieve entities, domain-specific terms, surface variants, confusable items, and their semantic relationships from dialogue history. This enables context-aware retrieval-augmented generative correction, advancing ASR error correction from localized hypotheses toward holistic contextual understanding. Evaluated on the newly constructed RAMC-Corr dataset, the method significantly outperforms direct correction across nine out of ten backbone model configurations, particularly improving accuracy in correcting context-dependent errors.

Abstract

Automatic speech recognition (ASR) correction has traditionally focused on isolated utterances or short local contexts. However, as text and speech become increasingly interleaved in long interactions, ASR correction requires conversation-level contextual evidence. Existing ASR correction methods often rely on the current hypothesis or concatenate raw dialogue history. In such contexts, sparse correction evidence can be difficult to locate amid redundancy and noise. Addressing these challenges, we propose an ontology memory-augmented ASR correction framework for long text-speech interleaved conversations. The framework organizes preceding interaction history into a dynamically updatable ontology memory, where entities, terminology, surface variants, potential ASR confusions, and semantic relations are stored as retrievable nodes for context-grounded correction. To evaluate this setting, we construct RAMC-Corr, a dataset derived from MAGIC-RAMC for long-range ASR correction with grounded context. Experiments on RAMC-Corr show that our method improves over direct correction in 9 out of 10 paired backbone-setting combinations and encourages more selective and evidence-grounded corrections for context-dependent ASR errors.

Problem

• ASR correction
• long conversations
• text-speech interleaved
• contextual evidence
• ontology memory

Innovation

• ontology memory
• ASR correction
• long-context conversation
• text-speech interleaving
• context-grounded correction