Exploring the Potential of Carbon-Aware Execution for Scientific Workflows

2025-03-17

AI Summary

Scientific workflows are energy-intensive and carbon-heavy, yet their delay tolerance, interruptibility, and scalability present unique opportunities for carbon-aware computing. This paper proposes a carbon-aware execution framework for scientific workflows, integrating time-aware scheduling, dynamic pause/resume, and elastic resource scaling. Leveraging a linear power model and real-world regional marginal carbon intensity (MCI) data—not average grid intensity—the framework enables fine-grained temporal carbon optimization within the Nextflow engine. It constitutes the first systematic evaluation of carbon-aware strategies for scientific workflows, quantifying their emission-reduction potential and identifying time-shifting and elastic scaling as key levers for carbon footprint reduction. Empirical evaluation across two geographically distinct regions demonstrates significant reductions in both end-to-end workflow and per-task carbon emissions, validating the effectiveness and generalizability of MCI-driven scheduling.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Hardware-aware ML
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Scientific workflows are widely used to automate scientific data analysis and often involve processing large quantities of data on compute clusters. As such, their execution tends to be long-running and resource intensive, leading to significant energy consumption and carbon emissions. Meanwhile, a wealth of carbon-aware computing methods have been proposed, yet little work has focused specifically on scientific workflows, even though they present a substantial opportunity for carbon-aware computing because they are inherently delay tolerant, efficiently interruptible, and highly scalable. In this study, we demonstrate the potential for carbon-aware workflow execution. For this, we estimate the carbon footprint of two real-world Nextflow workflows executed on cluster infrastructure. We use a linear power model for energy consumption estimates and real-world average and marginal CI data for two regions. We evaluate the impact of carbon-aware temporal shifting, pausing and resuming, and resource scaling. Our findings highlight significant potential for reducing emissions of workflows and workflow tasks.

Problem

• Reducing carbon emissions in scientific workflows
• Optimizing energy consumption in data-intensive workflows
• Implementing carbon-aware methods for workflow execution

Innovation

• Carbon-aware temporal shifting reduces emissions.
• Pausing and resuming workflows cuts carbon footprint.
• Resource scaling optimizes energy use efficiently.