MAATS: A Multi-Agent Automated Translation System Based on MQM Evaluation

2025-05-20

AI Summary

Current single-agent automatic translation systems exhibit significant limitations in self-correction capabilities, particularly in ensuring semantic accuracy, localization adaptability, and translation quality for distant language pairs. To address these challenges, we propose the Multi-Agent Automatic Translation System (MAATS), the first MQM-driven modular multi-agent architecture. MAATS decomposes translation refinement into specialized LLM agents—each dedicated to distinct MQM dimensions (e.g., accuracy, fluency, terminology)—enabling multi-perspective error detection, cross-dimension omission identification, and context-aware iterative refinement. This design enhances interpretability and tightly integrates with human post-editing workflows. Experiments across diverse language pairs and LLM backbones demonstrate that MAATS significantly outperforms zero-shot and single-agent baselines. Improvements are consistently observed in semantic accuracy, localization fidelity, and distant-language-pair translation quality, validated by both human evaluation and automated metrics.

Technology Category

• Multiagent Systems: Multiagent Learning
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We present MAATS, a Multi Agent Automated Translation System that leverages the Multidimensional Quality Metrics (MQM) framework as a fine-grained signal for error detection and refinement. MAATS employs multiple specialized AI agents, each focused on a distinct MQM category (e.g., Accuracy, Fluency, Style, Terminology), followed by a synthesis agent that integrates the annotations to iteratively refine translations. This design contrasts with conventional single-agent methods that rely on self-correction. Evaluated across diverse language pairs and Large Language Models (LLMs), MAATS outperforms zero-shot and single-agent baselines with statistically significant gains in both automatic metrics and human assessments. It excels particularly in semantic accuracy, locale adaptation, and linguistically distant language pairs. Qualitative analysis highlights its strengths in multi-layered error diagnosis, omission detection across perspectives, and context-aware refinement. By aligning modular agent roles with interpretable MQM dimensions, MAATS narrows the gap between black-box LLMs and human translation workflows, shifting focus from surface fluency to deeper semantic and contextual fidelity.

Problem

• Detects translation errors using MQM framework
• Improves semantic accuracy and locale adaptation
• Bridges gap between LLMs and human translation

Innovation

• Multi-agent system for MQM-based error detection
• Specialized AI agents refine translations iteratively
• Outperforms single-agent methods in semantic accuracy