Prompt engineering and its implications on the energy consumption of Large Language Models

2025-01-10

AI Summary

Large language models (LLMs) incur substantial energy consumption during code generation, raising environmental and operational concerns. Method: This work systematically investigates the impact of prompt engineering on carbon efficiency during inference—using Llama 3 on the CodeXGLUE benchmark—via hardware-level energy measurement jointly with accuracy evaluation. Contribution/Results: We identify, for the first time, that structured prompt labeling (e.g., role-tagged segmented prompts) significantly reduces energy consumption without degrading output quality. We propose a novel “prompt design–carbon efficiency” co-optimization paradigm. Experimental results demonstrate that the optimal prompt variant achieves a 12.7% reduction in inference energy while maintaining stable BLEU and CodeBLEU scores. Our approach offers a deployable, zero-parameter, lightweight optimization pathway toward green AI.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Efficient ML / Green AI
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Reducing the environmental impact of AI-based software systems has become critical. The intensive use of large language models (LLMs) in software engineering poses severe challenges regarding computational resources, data centers, and carbon emissions. In this paper, we investigate how prompt engineering techniques (PETs) can impact the carbon emission of the Llama 3 model for the code generation task. We experimented with the CodeXGLUE benchmark to evaluate both energy consumption and the accuracy of the generated code using an isolated testing environment. Our initial results show that the energy consumption of LLMs can be reduced by using specific tags that distinguish different prompt parts. Even though a more in-depth evaluation is needed to confirm our findings, this work suggests that prompt engineering can reduce LLMs' energy consumption during the inference phase without compromising performance, paving the way for further investigations.

Problem

• Prompt Engineering
• Large Language Models
• Environmental Impact

Innovation

• Energy-efficient AI
• Code Generation
• Environmental Sustainability