Medical Spoken Named Entity Recognition

2024-06-19

arXiv.org

AI Summary

The scarcity of real-world annotated data for Vietnamese medical spoken-language named entity recognition (NER) hinders progress in this domain. Method: We introduce VietMed-NER, the first large-scale, publicly available Vietnamese medical spoken-language NER dataset, covering 18 fine-grained entity types—the most diverse spoken-language NER resource globally to date. We formally define the medical spoken-language NER task and propose an end-to-end framework based on multilingual pretrained encoders (XLM-R/mT5), jointly modeling ASR outputs, human transcripts, and cross-lingual translations. Contribution/Results: Empirical evaluation shows that multilingual encoder-based models significantly outperform sequence-to-sequence baselines on both reference texts and ASR-postprocessed inputs. Moreover, the models demonstrate strong cross-lingual transferability to medical NER tasks in other languages. All code, data, and trained models are publicly released to foster reproducible research and community advancement.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Spoken Named Entity Recognition (NER) aims to extract named entities from speech and categorise them into types like person, location, organization, etc. In this work, we present VietMed-NER - the first spoken NER dataset in the medical domain. To our knowledge, our Vietnamese real-world dataset is the largest spoken NER dataset in the world regarding the number of entity types, featuring 18 distinct types. Furthermore, we present baseline results using various state-of-the-art pre-trained models: encoder-only and sequence-to-sequence; and conduct quantitative and qualitative error analysis. We found that pre-trained multilingual models generally outperform monolingual models on reference text and ASR output and encoders outperform sequence-to-sequence models in NER tasks. By translating the transcripts, the dataset can also be utilised for text NER in the medical domain in other languages than Vietnamese. All code, data and models are publicly available: https://github.com/leduckhai/MultiMed/tree/master/VietMed-NER.

Problem

• Extract medical named entities from Vietnamese speech
• Create largest spoken NER dataset with 18 entity types
• Evaluate multilingual vs monolingual models for medical NER

Innovation

• First medical spoken NER dataset
• Multilingual models outperform monolingual
• Encoders better than sequence-to-sequence models