🤖 AI Summary
This study addresses the absence of standardized carbon accounting and environmental assessment frameworks in artificial intelligence research. Through an automated literature review, it reveals the prevalent lack of environmental reporting in NeurIPS publications. Methodologically, this work proposes SMAJ, a novel framework designed to balance model accuracy with computational efficiency, thereby challenging conventional state-of-the-art paradigms. Furthermore, it develops a heuristic algorithm for carbon emission estimation alongside carbonbenchmark, a drop-in software tool for practical implementation. The primary contributions lie in defining standardized sustainability metrics for AI evaluation and providing deployable carbon tracking utilities. Ultimately, this research advocates for an approach wherein AI development simultaneously pursues performance breakthroughs and environmental responsibility.
📝 Abstract
As the capabilities and ubiquity of Large Language Models (LLMs) grow, so does their environmental footprint. Despite calls for responsible AI, the machine learning community lacks standardised practices for carbon accounting. Our automated literature review of the 5,285 papers accepted to NeurIPS 2025 reveals that reporting of environmental impact is nearly non-existent. To catalyse a shift toward sustainable AI, we define standardised sustainability metrics for evaluating model training efficiency, accompanied by simple heuristics to estimate the carbon cost of LLM inference. We implement these metrics in carbonbenchmark, a drop-in software solution for tracking and reporting emissions.
Finally, to combat the pursuit of marginal accuracy gains at disproportionate environmental costs, we formalise the `Smallest Model that Achieves the Job' (SMAJ), a framework which challenges the field to prioritise computational efficiency and environmental accountability alongside traditional `State-of-the-Art' (SotA) accuracy.