Instruction-tuned Large Language Models for Machine Translation in the Medical Domain

2024-08-29

arXiv.org

AI Summary

To address inconsistent and inaccurate terminology translation by large language models (LLMs) in medical machine translation, this work proposes explicitly injecting structured medical terminologies—sourced from authoritative ontologies such as UMLS and SNOMED CT—into instruction-tuning datasets, enabling the first instance of controllable, terminology-guided instruction fine-tuning. Methodologically, we employ efficient LoRA-based fine-tuning on open-source LLMs to construct high-quality medical instruction data. Experiments on multiple Chinese–English and German–English medical terminology translation benchmarks demonstrate substantial improvements: +9.2 BLEU points and +37.5% absolute gain in terminology accuracy (TERMScore), significantly outperforming both baseline LLMs and conventional neural machine translation models. This work overcomes the critical limitation of generic LLMs—terminological inaccuracy in low-resource, domain-specific settings—and establishes a novel paradigm for domain-adaptive instruction tuning grounded in structured knowledge integration.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) have shown promising results on machine translation for high resource language pairs and domains. However, in specialised domains (e.g. medical) LLMs have shown lower performance compared to standard neural machine translation models. The consistency in the machine translation of terminology is crucial for users, researchers, and translators in specialised domains. In this study, we compare the performance between baseline LLMs and instruction-tuned LLMs in the medical domain. In addition, we introduce terminology from specialised medical dictionaries into the instruction formatted datasets for fine-tuning LLMs. The instruction-tuned LLMs significantly outperform the baseline models with automatic metrics.

Problem

• Improving medical domain machine translation with LLMs
• Enhancing terminology consistency in medical translations
• Instruction-tuning LLMs for specialized medical dictionaries

Innovation

• Instruction-tuned LLMs enhance medical translation
• Incorporates medical dictionaries for terminology accuracy
• Outperforms baseline models in automatic metrics