DEFT: Joint Task Placement and DVFS for Energy-Efficient Multi-GPU Runtimes

2026-08-03

AI Summary

Existing multi-GPU runtime systems lack coordinated optimization among task placement, dynamic voltage and frequency scaling (DVFS), and inter-GPU communication, making it challenging to balance energy efficiency and performance. This work proposes DEFT, a novel framework that, for the first time, jointly models task assignment and DVFS at the task granularity. DEFT employs a cost model that holistically accounts for execution time, data migration overhead, and DVFS transition costs to enable fine-grained, energy-aware cooperative scheduling. The framework incorporates slack-aware and throughput-aware mechanisms and explicitly models both communication and DVFS switching overheads, integrating seamlessly into the CUDASTF runtime. Experimental results on NVIDIA L40S and L4 GPUs demonstrate average energy savings of 14.8% and 4.8%, respectively, with energy-delay product (EDP) reductions of 9.9% and 3.7%, while keeping performance degradation within 1.5%.

Abstract

Energy efficiency has become a first-order concern in modern high-performance computing systems, as it directly determines achievable throughput under fixed power budgets. Although Dynamic Voltage and Frequency Scaling (DVFS) provides an effective mechanism for reducing GPU energy consumption, existing runtime systems decouple DVFS from task placement and inter-GPU communication, focus on single-GPU execution, or cannot adapt frequency to task granularity and runtime contention in multi-GPU environments. Consequently, current schedulers fail to capture the tight coupling between task placement, frequency selection, and inter-GPU data movement that fundamentally governs energy-performance trade-offs on multi-GPU systems. This paper presents DEFT, an energy-aware scheduling framework that jointly optimizes task-to-device assignment and per-GPU DVFS configuration for task-based multi-GPU applications. DEFT employs a cost-model-driven strategy that integrates slack awareness, throughput awareness, and explicit modeling of task execution cost, inter-GPU data movement, and DVFS transition overheads, enabling coordinated placement and frequency decisions at task granularity under dynamic runtime conditions. We prototype DEFT within the CUDASTF runtime and demonstrate its effectiveness across five optimization objectives. The evaluation shows that DEFT reduces energy consumption by 14.8% and 4.8% on average on NVIDIA L40S and L4, and reduces EDP by 9.9% and 3.7%, respectively, while maintaining performance within 1.5% of the fastest baseline.

Problem

• multi-GPU
• energy efficiency
• task placement
• DVFS
• inter-GPU communication

Innovation

• joint optimization
• task placement
• DVFS
• multi-GPU scheduling
• energy efficiency