🤖 AI Summary
This work addresses the challenge that current machine translation systems struggle to preserve alignment between textual content and visual layout—such as tables and mathematical formulas—when processing PDF documents, often resulting in structural distortion. To tackle this issue, the authors construct a multimodal parallel corpus comprising 3,956 PDFs across 15 language pairs, meticulously retaining original layout metadata. They introduce, for the first time, a 45-dimensional geometric feature space to guide K-Medoids sampling, prioritizing visual structural diversity. This dataset enables layout-aware translation that jointly leverages textual and visual context, exposing critical limitations of existing systems in spatial localization and geometric synchronization. It thus establishes a high-fidelity, evaluable benchmark for document translation and reconstruction that faithfully preserves complex layouts.
📝 Abstract
We present ForMaT (Format-Preserving Multilingual Translation), a parallel corpus of 3,956 PDFs across 15 language pairs that preserves original layout metadata proposed for multimodal machine translation. To ensure structural diversity in the dataset, we employ K-Medoids sampling over 45 geometric features, capturing complex elements like nested tables and formulas to focus only on visually diverse PDF documents. Our evaluation reveals that current MT systems struggle with spatial grounding and geometric synchronization, often losing the link between text and its visual context. ForMaT provides a benchmark for developing layout-aware translation models that integrate visual and textual context for high-fidelity document reconstruction.