Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs #98

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
2 changes: 1 addition & 1 deletion docs/co2footprint.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ This includes failed tasks as well.

## Used data

If the `location` parameter is specified, the plugin makes use of [location-specific CI data](../../plugins/nf-co2footprint/src/resources/CI_aggregated.v2.2.csv) that was copied from the Green Algorithms project [green-algorithms-tool/data](https://github.com/GreenAlgorithms/green-algorithms-tool/tree/master/data).
If the `location` parameter is specified, the plugin makes use of [location-specific CI data](https://github.com/nextflow-io/nf-co2footprint/blob/1.0.0-beta/plugins/nf-co2footprint/src/resources/CI_aggregated.v2.2.csv) that was copied from the Green Algorithms project [green-algorithms-tool/data](https://github.com/GreenAlgorithms/green-algorithms-tool/tree/master/data).
The CPU TDP data from the Green Algorithms project is used to retrieve model-specific CPU power draw values.


Expand Down
6 changes: 1 addition & 5 deletions docs/contributing/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,16 @@ To test with Nextflow for development purpose:
cd .. && git clone https://github.com/nextflow-io/nextflow
cd nextflow && ./gradlew exportClasspath
```

2. Append the following line to the `settings.gradle` in this project:

```bash
includeBuild('../nextflow')
```

3. Compile the plugin code

```bash
./gradlew compileGroovy
```

4. Run nextflow with this command:

```bash
Expand All @@ -49,7 +46,6 @@ The plugin can be tested without using a local Nextflow build using the followin
./gradlew compileGroovy
make install
```

2. Run nextflow with this command, specifying the plugin version:

```bash
Expand All @@ -68,7 +64,7 @@ To compile and run the tests use the following command:

## Change and preview the docs

The docs are generated using [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/). To change the docs, edit the files in the [docs/](docs/) folder and run the following command to generate the docs (after installing mkdocs via `pip install mkdocs-material`):
The docs are generated using [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/). To change the docs, edit the files in the [docs/](https://github.com/nextflow-io/nf-co2footprint/tree/master/docs) folder and run the following command to generate the docs (after installing mkdocs via `pip install mkdocs-material`):

```bash
mkdocs serve
Expand Down
76 changes: 76 additions & 0 deletions docs/usage/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
title: Configuration
description: Configuration of the nf-co2footprint plugin
---

# Configuration of the nf-co2footprint plugin

## General usage
To test if the plugin works on your system please follow the quick start guide ([Quick Start](https://nextflow-io.github.io/nf-co2footprint/#quick-start)) on a small pipeline like [nextflow-io/hello](https://github.com/nextflow-io/hello).

You can adjust the nf-co2footprint plugin parameters in your config file as follows:

```groovy title="nextflow.config"
plugins {
id 'nf-co2footprint@1.0.0-beta'
}

def co2_timestamp = new java.util.Date().format( 'yyyy-MM-dd_HH-mm-ss')

co2footprint {
traceFile = "${params.outdir}/co2footprint/co2footprint_trace_${co2_timestamp}.txt"
reportFile = "${params.outdir}/co2footprint/co2footprint_report_${co2_timestamp}.html"
summaryFile = "${params.outdir}/co2footprint/co2footprint_summary_${co2_timestamp}.txt"
ci = 300
pue = 1.4
}
```

Include the config file for your pipeline run using the `-c` Nextflow parameter, for example as follows:

```bash
nextflow run nextflow-io/hello -c nextflow.config
```

In this case the plugin will create 3 outputfiles within the outdir, which is set for the pipeline run. Also a custom carbon intensity (CI) is set and a Power Usage Efficiency (PUE) factor. To check all available parameters you can visit: [Parameters](https://nextflow-io.github.io/nf-co2footprint/usage/parameters/).
Bastian-Eisenmann marked this conversation as resolved.
Show resolved Hide resolved
The CI and PUE values will influence the CO<sub>2</sub> footprint the most and are dependent on where your pipeline is running. The CI reflects the used energy sources, while the PUE describes how efficiently the Power is used in regard to computing power.
Bastian-Eisenmann marked this conversation as resolved.
Show resolved Hide resolved

## Cloud computations

At the moment the nf-co2footprint can not natively support cloud computations.

!!! warning

This is not tested and might not work as intended, but cloud native support will be implemented soon.

If you are still keen to get insights into your CO<sub>2</sub> you could try to find out the following and append it to your config:

- The location and hereby CI of your instance.
- The PUE of the data center, where the instance is located
- The power draw per core of your selected instance.
- If available the power draw of the memory per GB.
skrakau marked this conversation as resolved.
Show resolved Hide resolved

Your configuration could look something like:

```groovy title="nextflow_cloud.config"
plugins {
id 'nf-co2footprint@1.0.0-beta'
}

def co2_timestamp = new java.util.Date().format( 'yyyy-MM-dd_HH-mm-ss')

co2footprint {
traceFile = "${params.outdir}/co2footprint/co2footprint_trace_${co2_timestamp}.txt"
reportFile = "${params.outdir}/co2footprint/co2footprint_report_${co2_timestamp}.html"
summaryFile = "${params.outdir}/co2footprint/co2footprint_summary_${co2_timestamp}.txt"
ci = 300
pue = 1.4
ignoreCpuModel = true
powerdrawCpuDefault = 8
powerdrawMem = 0.3725
}
```

## GPU computations

So far tracking of GPU driven computations are not implemented, and functionality might be impaired.
15 changes: 15 additions & 0 deletions docs/usage/output.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
title: Output
description: Output of the nf-co2footprint plugin.
Bastian-Eisenmann marked this conversation as resolved.
Show resolved Hide resolved
---

## Output

The nf-co2footprint plugin creates three output files:

- `traceFile`: The trace file includes calculations for each task, similar to the Nextflow trace file. Within this file you can find resource usage details of specific tasks and also the hardware information of your CPU.
- `summaryFile`: The summary file includes the total CO<sub>2</sub> footprint of the workflow run and the configuration used for the plugin.
- `reportFile`: The HTML report contains information about the carbon footprint of the whole pipeline run as well as plots showing the distributions of the CO<sub>2</sub> emissions for the different processes. Additionally, it contains a table with the metrics for all individual tasks.



28 changes: 5 additions & 23 deletions docs/usage/parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,6 @@ description: Customising parameters for the CO2e calculation.

## Customising parameters

You can adjust the nf-co2footprint plugin parameters in your config file as follows:

```groovy title="nextflow.config"
def co2_timestamp = new java.util.Date().format( 'yyyy-MM-dd_HH-mm-ss')

co2footprint {
traceFile = "${params.outdir}/co2footprint_trace_${co2_timestamp}.txt"
reportFile = "${params.outdir}/co2footprint_report_${co2_timestamp}.html"
ci = 300
pue = 1.4
}
```

Include the config file for your pipeline run using the `-c` Nextflow parameter, for example as follows:

```bash
nextflow run nextflow-io/hello -c nextflow.config
```

The following parameters are currently available:

- `traceFile`: Name of the TXT carbon footprint report containing the energy consumption, the estimated CO<sub>2</sub> emission and other relevant metrics for each task.
Expand All @@ -32,22 +13,23 @@ Default: `co2footprint_trace_<timestamp>.txt`.
Default: `co2footprint_summary_<timestamp>.txt`.
- `reportFile`: Name of the HTML report containing information about the entire carbon footprint, overview plots and more detailed task-specific metrics.
Default: `co2footprint_report_<timestamp>.html`.
- `ci`: carbon intensity of the respective energy production. Mutually exclusive with the `location` parameter.
- `ci`: Carbon intensity of the respective energy production. Available regions can be found [here](https://github.com/nextflow-io/nf-co2footprint/blob/1.0.0-beta/plugins/nf-co2footprint/src/resources/CI_aggregated.v2.2.csv), otherwise you can use data from [ElecricityMaps](https://app.electricitymaps.com/map/24h), to find out locally more accurate CI.
Mutually exclusive with the `location` parameter.
Default: 475.
- `location`: location code to automatically retrieve a location-specific CI value.
For countries, these are [ISO alpha-2 codes](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2).
For regions, it’s the ISO alpha-2 code for the country, followed by an identifier for the state, e.g. US-CA for “California, USA”.
You can find the available data [here](../../plugins/nf-co2footprint/src/resources/CI_aggregated.v2.2.csv).
You can find the available data [here](https://github.com/nextflow-io/nf-co2footprint/blob/1.0.0-beta/plugins/nf-co2footprint/src/resources/CI_aggregated.v2.2.csv).
Mutually exclusive with the `ci` parameter.
Default: `null`.
- `pue`: power usage effectiveness, efficiency coefficient of the data centre.
- `pue`: Power usage effectiveness, efficiency coefficient of the data centre. For local cluster you can usually find out your specific PUE at the system administrators or system managers. Also the current [yearly worldwide average](https://www.statista.com/statistics/1229367/data-center-average-annual-pue-worldwide/) could be used.
Default: 1.67.
- `powerdrawMem`: power draw from memory.
Default: 0.3725.
- `customCpuTdpFile`: Input CSV file containing custom CPU TDP data.
This should contain the following columns: `model`,`TDP`,`n_cores`,`TDP_per_core`.
Note that this overwrites TDP values for already provided CPU models.
You can find the by default used TDP data [here](../../plugins/nf-co2footprint/src/resources/TDP_cpu.v2.2.csv).
You can find the by default used TDP data [here](https://github.com/nextflow-io/nf-co2footprint/blob/1.0.0-beta/plugins/nf-co2footprint/src/resources/TDP_cpu.v2.2.csv).
Default: `null`.
- `ignoreCpuModel`: ignore the retrieved Nextflow trace `cpu_model` name and use the default CPU power draw value.
This is useful, if the cpu model information provided by the linux kernel is not correct, for example, in the case of VMs emulating a different CPU architecture.
Expand Down
4 changes: 3 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,16 @@ site_name: nf-co2footprint
repo_name: nextflow-io/nf-co2footprint
repo_url: https://github.com/nextflow-io/nf-co2footprint
site_url: https://nextflow-io.github.io/nf-co2footprint/
edit_uri: edit/dev/docs/ # TODO: change on release to edit/main/docs/
edit_uri: edit/master/docs/

nav:
- Home:
- index.md
- co2footprint.md
- Usage:
- usage/configuration.md
- usage/parameters.md
- usage/output.md
- Contributing:
- contributing/setup.md

Expand Down
Loading