Evaluating MFU as a Proxy for GPU Power for Energy-Aware Simulation of LLM Training

2026-08-04

AI Summary

This work addresses the inability of existing AI system simulators to accurately predict GPU power consumption for large model training prior to execution. The authors propose modeling power consumption using Model FLOPs Utilization (MFU) as a hardware-agnostic software metric, enabling a linear power model that operates without hardware performance counters. Through systematic experiments spanning diverse GPUs, data types, and batch sizes, the study demonstrates for the first time that MFU reliably predicts power consumption across platforms during compute-bound LLM training. By constructing fine-grained models at the granularity of (GPU, dtype, batch size), the approach achieves an average prediction error of approximately 1%—approaching the lower bound imposed by measurement noise—and substantially enhances the practicality and accuracy of energy-aware simulation.

Abstract

High-fidelity performance simulators are essential for designing and configuring efficient AI systems, yet today's tools lack the ability to predict power consumption. Established GPU power models rely on hardware utilization counters, which do not exist until the workload has actually run. This work evaluates whether Model FLOPs Utilization (MFU)-an analytical, software-defined metric relating achieved throughput to peak hardware capability-can serve as a portable, software-defined predictor of GPU power for LLMs. We benchmark almost 3000 single-device training runs across six GPUs, covering different model families, numerical precisions, batch sizes, and context-window lengths. We find that a linear MFU-based power model fits every tested GPU as long as the workload is compute-bound, as in production LLM training. Fitting per-(GPU, dtype, batch size) instead of per-GPU drops the within-cell mean error from around 10% to around 1%, matching the cross-repeat measurement-noise floor.

Problem

• GPU power prediction
• energy-aware simulation
• Model FLOPs Utilization
• LLM training
• hardware utilization counters

Innovation

• Model FLOPs Utilization
• GPU power modeling
• energy-aware simulation
• LLM training
• software-defined metrics