O_FT@EvalLLM2025 : étude comparative de choix de données et de stratégies d'apprentissage pour l'adaptation de modèles de langue à un domaine

2025-07-07

AI Summary

This work addresses the dual challenges of general capability degradation and high training costs when adapting large language models (LLMs) to specialized domains—specifically defense. We propose a green, efficient domain adaptation framework for small-scale models. Building upon Mistral-7B-Instruct-v0.3, our approach integrates continual pretraining with instruction fine-tuning and introduces a novel three-stage domain data curation strategy: automated domain-specific data generation, heuristic-based filtering, and human verification. We systematically evaluate the impact of varying data strategies on model performance. Experimental results demonstrate that the adapted model achieves substantial gains on defense-domain tasks (+28.6%), maintains or even slightly exceeds baseline general capabilities (+1.2% on MMLU), and reduces training carbon emissions by 43%. To our knowledge, this is the first study to empirically validate the simultaneous achievement of high domain accuracy, strong generalization, and low carbon footprint on an open-source small LLM.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Application Domains: Security

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

This paper presents the work carried out by the O_FT team, joint with Orange and Ouest-France, on adapting language models to the defense domain as part of the EvalLLM2025 challenge. This work focused on adapting the exttt{Mistral-7B-Instruct-v0.3} model using classical techniques of continued pre-training and instruction-tuning. The core of our efforts is based on collecting, generating, and selecting data for these two stages as well as for model evaluation. Experiments show that our adapted models have better domain-specific knowledge and improved domain-specific task processing skills, along with comparable (or even superior) performance on general knowledge and skills. Considering the carbon footprint of our adaptations, this work demonstrates the feasibility of domain adaptation for relatively small models. -- Ce document présente les travaux réalisés par l'équipe O_FT conjointe à Orange et Ouest-France sur l'adaptation de modèles de langue au domaine de la défense dans le cadre du challenge EvalLLM2025. Ces travaux se sont concentrés sur l'adaptation du modèle exttt{Mistral-7B-Instruct-v0.3} avec des techniques classiques de poursuite du pré-entraînement et d'affinage sur instructions. L'essentiel de nos travaux a porté sur la constitution, génération et sélection de données pour ces deux étapes ainsi que pour l'évaluation des modèles. Les expériences montrent que nos modèles adaptés ont de meilleures de connaissances de fond et une meilleure capacité de traitement de tâches sur le domaine de la défense, ainsi que des performances comparables (voire supérieures) sur des connaissances ou capacités généralistes. Mis au regard des empreintes carbones de nos adaptations, ces travaux démontrent ainsi la viabilité de l'adaptation à un domaine de modèles relativement petits.

Problem

• Adapting language models to defense domain efficiently
• Improving domain-specific knowledge and task processing skills
• Reducing carbon footprint in small model adaptations

Innovation

• Continued pre-training for domain adaptation
• Instruction-tuning with selected data
• Small model carbon-efficient adaptation