GenAI Carbon Footprint calculator for LLM Inference

This repository contains a Go library (github.com/greenscale-ai/genai-carbon-footprint) designed to estimate the energy usage and carbon emissions associated with running Large Language Model (LLM) inferences, taking into account various factors such as the carbon intensity of electricity, device Thermal Design Power (TDP), memory consumption, and inference timing.

Features

• Carbon Intensity Calculation: Calculate the carbon emissions based on user-provided or default values for electricity carbon intensity, device TDP, memory usage, and inference timing.
• Flexible Time Calculation: Supports specifying total inference time directly or calculating it based on first token latency and subsequent token latency times.
• CLI Tool: A command-line interface (CLI) tool for easy interaction with the library without needing to integrate it into other Go applications.

Getting Started

Prerequisites

Go version 1.15 or higher

Installing

Clone the repository to your local machine:

git clone https://github.com/greenscale-ai/genai-carbon-footprint.git
cd genai-carbon-footprint

Build the CLI tool:

go build -o genai-carbon-footprint-cli cli/main.go

Usage

Run the CLI tool with the necessary flags to calculate the carbon emissions. For example:

./genai-carbon-footprint-cli --tdp 350 --mem 8000 --total-inference-latency 16000 --carbon-intensity 450

Flags

• -tdp: Model TDP in Watts (required).
• --total-inference-latency: Total time of one inference procedure in milliseconds.
• --first-latency: First token latency in milliseconds.
• --next-latency: Next token latency in milliseconds.
• -mem: Memory consumption in MB.
• --token-size: Output token number in one inference (default: 32).
• --carbon-intensity: Carbon intensity of electricity (default: 475 gCO2e/kWh - world average).

Contributing

We welcome contributions to improve this project. Please follow these steps to contribute:

1. Fork the repository.
2. Create a new branch (git checkout -b feature/AmazingFeature).
3. Commit your changes (git commit -m 'Add some AmazingFeature').
4. Push to the branch (git push origin feature/AmazingFeature).
5. Open a Pull Request.

License

This project is subject to the MIT License. A copy of this license can be found in LICENSE at the root of this repo.

Acknowledgments

Intel Corporation for the initial Python version of the carbon calculator (link).