Measure and track the carbon footprint of training and inferencing machine learning models

[Stijnp]

Proposal Summary

Add a mechanism to measure and track the carbon footprint of training and inferencing machine learning models.

This feature would integrate a carbon tracking feature, allowing users to monitor and report the carbon emissions associated with their machine learning experiments. The feature could leverage existing tools such as CodeCarbon to compute the carbon footprint based on model training and inferencing details, such as GPU/CPU usage, duration, energy consumption, and regional power grid carbon intensity.

It would be great to see on a per experiment basis as well as the entirety of a certain model (including failed experiments, earlier experiments in the fine-tuning lifecycle, etc) as this final number is often what needs to be reported.

Motivation

Companies and governments are becoming increasingly aware of the impact of AI on our energy grid and GHG emissions and need to report these numbers. For other practitioners it could be a great way to increase the transparency of their ML training strategies.

Related Discussion

Not yet

Other resources

Blogpost on why and how data roots implemented something similar in MLFlow

[ainoam]

Thanks for suggesting @Stijnp. ClearML is all about adding visibility - We'll definitely consider how this can addressed in a future release.