A Holistic Assessment of the Carbon Footprint of Noor, a Very Large Arabic Language Model

2026-09-22

AI Summary

This study addresses the limitation of existing carbon footprint assessments for large language models, which predominantly focus on the training phase. Using the 13-billion-parameter Arabic model Noor as a case study, this work proposes a comprehensive life-cycle environmental impact assessment framework encompassing data processing, research and development, training, inference, and exogenous factors such as cross-border collaboration. By integrating zero-shot generalization, instruction fine-tuning, and distributed computing techniques, the authors conduct an end-to-end carbon emission accounting. The results demonstrate that inference and exogenous costs significantly influence the total carbon budget. This research transcends conventional assessment boundaries, providing empirical evidence and optimization pathways for mitigating the carbon footprint of extremely large-scale models.

Abstract

As ever larger language models grow more ubiquitous, it is crucial to consider their environmental impact. Characterised by extreme size and resource use, recent generations of models have been criticised for their voracious appetite for compute, and thus significant carbon footprint. Although reporting of carbon impact has grown more common in machine learning papers, this reporting is usually limited to compute resources used strictly for training. In this work, we propose a holistic assessment of the footprint of an extreme-scale language model, Noor. Noor is an ongoing project aiming to develop the largest multi-task Arabic language models -- with up to 13B parameters -- leveraging zero-shot generalisation to enable a wide range of downstream tasks via natural language instructions. We assess the total carbon bill of the entire project: starting with data collection and storage costs, including research and development budgets, pretraining costs, future serving estimates, and other exogenous costs necessary for this international cooperation. Notably, we find that inference costs and exogenous factors can have a significant impact on total budget. Finally, we discuss pathways to reduce the carbon footprint of extreme-scale models.

Problem

• carbon footprint
• large language models
• holistic assessment
• inference costs
• environmental impact

Innovation

• Carbon Footprint
• Large Language Model
• Holistic Assessment
• Inference Costs
• Arabic NLP