AMALIA-VL: A Native European Portuguese Open-Source Vision and Language Model

2026-06-17

AI Summary

This work addresses the longstanding neglect of European Portuguese (pt-PT) in existing open-source multimodal models, where it is often conflated with Brazilian Portuguese. To bridge this gap, we propose the first open-source vision-language model natively supporting pt-PT, integrating a high-resolution visual encoder, dynamic image tiling, and a language model optimized specifically for pt-PT. Our approach employs a three-stage training paradigm encompassing multimodal alignment, instruction fine-tuning, and preference optimization. We further introduce a novel multimodal data mixing strategy tailored to pt-PT, combining carefully curated, translated, and newly constructed datasets. The resulting model significantly outperforms current baselines. We release the model weights, training pipeline, data, and a machine translation evaluation benchmark to establish a foundation for future pt-PT multimodal research.

Abstract

Large Vision and Language Models (LVLMs) have advanced rapidly, yet European Portuguese (pt-PT) remains systematically underserved by existing open-source multimodal models, which either conflate it with Brazilian Portuguese or severely under-represent it in their training data mixes. We introduce AMALIA-VL, the first open-source instruction-tuned LVLM built natively for pt-PT, pairing a high-resolution vision encoder with dynamic image tiling and a fully open pt-PT-optimized language model via a learned connector. We contribute with a purposefully designed three-stage training process - vision-language alignment, general visual instruction tuning, and preference optimization - together with a pt-PT-centric multimodal data mix combining curated and translated public datasets with novel datasets that address the near-total absence of European Portuguese multimodal resources. Our evaluation shows that AMALIA-VL establishes a strong baseline for open-source pt-PT LVLMs.We will release model weights, training data, and construction pipelines along with machine-translated pt-PT evaluation benchmarks to help democratize pt-PT LVLM development.

Problem

• European Portuguese
• Vision and Language Models
• multimodal models
• language representation
• open-source

Innovation

• vision-language model
• European Portuguese
• dynamic image tiling
• instruction tuning
• multimodal dataset