Casper: Carbon-Aware Scalable Processing in Elastic Clusters

The Casper project investigates how the execution of scalable batch processing applications can be aligned with the availability of low-carbon energy. More specifically, we aim to pioneer how individual processing steps of large-scale cluster applications can be managed dynamically based on continuously updated estimates of application performance, resource availability, and carbon intensity.

About Our Research

The three main goals towards our project aim are:

Develop the methods needed to reliably reduce the emissions of large scalable batch data processing applications on elastic compute clusters significantly
Design and implement a prototype for elastic Kubernetes clusters and scalable batch data processing (such as Nextflow workflows and Spark dataflows)
Evaluate the prototype system in use cases covering (1) a commercial cluster service of a leading public cloud provider and (2) a similar service running on the private cloud resources of a research organisation

Latest News

Presentations at FONDA workshop at HU Berlin

November 06, 2025

Youssef, Kathleen, and Lauritz will be speaking at the FONDA workshop on Reducing the Carbon Emissions of Large-Scale Computational Data Analysis, which will take place on 25 and 26 November at Humboldt University of Berlin.

New pre-print extending our study of carbon-aware scientific workflows

August 23, 2025

We prepared a new article that systematically assesses the potential of carbon-aware temporal shifting, pausing and resuming, and resource scaling. For this, we studied various real Nextflow workflows, compute clusters, and energy grid data.

Short research paper on Carbon-Aware Scientific Workflows at CCGrid 2025

May 23, 2025

We are happy to share that our short paper on “Exploring the Potential of Carbon-Aware Execution for Scientific Workflows” was accepted for publication in the proceedings of the “25th IEEE International Symposium on Cluster, Cloud and Internet Computing”, and was presented in person, in Tromsø in May 2025.

View all news

Recent Publications and Open-Source Software

Energy-Aware Workflow Execution: An Overview of Techniques for Saving Energy and Emissions in Scientific Compute Clusters

Lauritz Thamsen, Yehia Elkhatib, Paul Harvey, Syed Waqar Nabi, Jeremy Singer, and Wim Vanderbauwhede

To appear in a textbook on Workflow Systems for Large-Scale Scientific Data Analysis, 2025