Casper: Carbon-Aware Scalable Processing in Elastic Clusters

The Casper project investigates how the execution of scalable batch processing applications can be aligned with the availability of low-carbon energy. More specifically, we aim to pioneer how individual processing steps of large-scale cluster applications can be managed dynamically based on continuously updated estimates of application performance, resource availability, and carbon intensity.

About Our Research

The three main goals towards our project aim are:

Develop the methods needed to reliably reduce the emissions of large scalable batch data processing applications on elastic compute clusters significantly
Design and implement a prototype for elastic Kubernetes clusters and scalable batch data processing (such as Nextflow workflows and Spark dataflows)
Evaluate the prototype system in use cases covering (1) a commercial cluster service of a leading public cloud provider and (2) a similar service running on the private cloud resources of a research organisation

Latest News

New pre-print extending our study of carbon-aware scientific workflows

August 23, 2025

We prepared a new article that systematically assesses the potential of carbon-aware temporal shifting, pausing and resuming, and resource scaling. For this, we studied various real Nextflow workflows, compute clusters, and energy grid data.

Short paper on Carbon-Aware Scientific Workflows at CCGrid 2025

May 23, 2025

We are happy to share that our short paper on “Exploring the Potential of Carbon-Aware Execution for Scientific Workflows” was accepted for publication in the proceedings of the “25th IEEE International Symposium on Cluster, Cloud and Internet Computing”, and was presented in person, in Tromsø in May 2025.

Full workshop paper on workflow footprint estimation accepted at LOCO 2024

April 25, 2025

We are happy to share that our full workshop paper on “Ichnos: A Carbon Footprint Estimator for Scientific Workflows” has been accepted for publication in the post-proceedings of the 1st International Workshop on Low Carbon Computing (LOCO 2024), which was held as a hybrid event in Glasgow on December 3rd last year.

View all news

Recent Publications and Open-Source Software

Energy-Aware Workflow Execution: An Overview of Techniques for Saving Energy and Emissions in Scientific Compute Clusters

Lauritz Thamsen, Yehia Elkhatib, Paul Harvey, Syed Waqar Nabi, Jeremy Singer, and Wim Vanderbauwhede

To appear in a textbook on Workflow Systems for Large-Scale Scientific Data Analysis, 2025