diff --git a/docs/co2footprint.md b/docs/co2footprint.md index 9fb7a00..320338a 100644 --- a/docs/co2footprint.md +++ b/docs/co2footprint.md @@ -35,7 +35,7 @@ This includes failed tasks as well. ## Used data -If the `location` parameter is specified, the plugin makes use of [location-specific CI data](../../plugins/nf-co2footprint/src/resources/CI_aggregated.v2.2.csv) that was copied from the Green Algorithms project [green-algorithms-tool/data](https://github.com/GreenAlgorithms/green-algorithms-tool/tree/master/data). +If the `location` parameter is specified, the plugin makes use of [location-specific CI data](https://github.com/nextflow-io/nf-co2footprint/blob/1.0.0-beta/plugins/nf-co2footprint/src/resources/CI_aggregated.v2.2.csv) that was copied from the Green Algorithms project [green-algorithms-tool/data](https://github.com/GreenAlgorithms/green-algorithms-tool/tree/master/data). The CPU TDP data from the Green Algorithms project is used to retrieve model-specific CPU power draw values. diff --git a/docs/contributing/setup.md b/docs/contributing/setup.md index 53f15f9..9710306 100644 --- a/docs/contributing/setup.md +++ b/docs/contributing/setup.md @@ -15,19 +15,16 @@ To test with Nextflow for development purpose: cd .. && git clone https://github.com/nextflow-io/nextflow cd nextflow && ./gradlew exportClasspath ``` - 2. Append the following line to the `settings.gradle` in this project: ```bash includeBuild('../nextflow') ``` - 3. Compile the plugin code ```bash ./gradlew compileGroovy ``` - 4. Run nextflow with this command: ```bash @@ -49,7 +46,6 @@ The plugin can be tested without using a local Nextflow build using the followin ./gradlew compileGroovy make install ``` - 2. Run nextflow with this command, specifying the plugin version: ```bash @@ -68,7 +64,7 @@ To compile and run the tests use the following command: ## Change and preview the docs -The docs are generated using [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/). To change the docs, edit the files in the [docs/](docs/) folder and run the following command to generate the docs (after installing mkdocs via `pip install mkdocs-material`): +The docs are generated using [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/). To change the docs, edit the files in the [docs/](https://github.com/nextflow-io/nf-co2footprint/tree/master/docs) folder and run the following command to generate the docs (after installing mkdocs via `pip install mkdocs-material`): ```bash mkdocs serve diff --git a/docs/usage/configuration.md b/docs/usage/configuration.md new file mode 100644 index 0000000..5a9afb7 --- /dev/null +++ b/docs/usage/configuration.md @@ -0,0 +1,73 @@ +--- +title: Configuration +description: Configuration of the nf-co2footprint plugin +--- + +# Configuration of the nf-co2footprint plugin + +## General usage +To test if the plugin works on your system please follow the quick start guide ([Quick Start](https://nextflow-io.github.io/nf-co2footprint/#quick-start)) on a small pipeline like [nextflow-io/hello](https://github.com/nextflow-io/hello). + +You can adjust the nf-co2footprint plugin parameters in your config file as follows: + +```groovy title="nextflow.config" +plugins { + id 'nf-co2footprint@1.0.0-beta' +} + +def co2_timestamp = new java.util.Date().format( 'yyyy-MM-dd_HH-mm-ss') + +co2footprint { + traceFile = "${params.outdir}/co2footprint/co2footprint_trace_${co2_timestamp}.txt" + reportFile = "${params.outdir}/co2footprint/co2footprint_report_${co2_timestamp}.html" + summaryFile = "${params.outdir}/co2footprint/co2footprint_summary_${co2_timestamp}.txt" + ci = 300 + pue = 1.4 +} +``` + +Include the config file for your pipeline run using the `-c` Nextflow parameter, for example as follows: + +```bash +nextflow run nextflow-io/hello -c nextflow.config +``` + +## Cloud computations + +At the moment the nf-co2footprint can not natively support cloud computations. + +!!! warning + + This is not tested and might not work as intended, but cloud native support will be implemented soon. + +If you are still keen to get insights into your CO2 you could try to find out the following and append it to your config: + +- The location and hereby CI of your instance. +- The PUE of the data center, where the instance is located +- The power draw per core of your selected instance. +- If available the power draw of the memory per GB. + +Your configuration could look something like: + +```groovy title="nextflow_cloud.config" +plugins { + id 'nf-co2footprint@1.0.0-beta' +} + +def co2_timestamp = new java.util.Date().format( 'yyyy-MM-dd_HH-mm-ss') + +co2footprint { + traceFile = "${params.outdir}/co2footprint/co2footprint_trace_${co2_timestamp}.txt" + reportFile = "${params.outdir}/co2footprint/co2footprint_report_${co2_timestamp}.html" + summaryFile = "${params.outdir}/co2footprint/co2footprint_summary_${co2_timestamp}.txt" + ci = 300 + pue = 1.4 + ignoreCpuModel = true + powerdrawCpuDefault = 8 + powerdrawMem = 0.3725 +} +``` + +## GPU computations + +So far tracking of GPU driven computations are not implemented, and functionality might be impaired. \ No newline at end of file diff --git a/docs/usage/output.md b/docs/usage/output.md new file mode 100644 index 0000000..60b9fb7 --- /dev/null +++ b/docs/usage/output.md @@ -0,0 +1,15 @@ +--- +title: Output +description: Output of the nf-co2footprint plugin. +--- + +## Output + +The nf-co2footprint plugin creates three output files: + +- `traceFile`: The trace file includes calculations for each task, similar to the Nextflow trace file. Within this file you can find resource usage details of specific tasks and also the hardware information of your CPU. +- `summaryFile`: The summary file includes the total CO2 footprint of the workflow run and the configuration used for the plugin. +- `reportFile`: The HTML report contains information about the carbon footprint of the whole pipeline run as well as plots showing the distributions of the CO2 emissions for the different processes. Additionally, it contains a table with the metrics for all individual tasks. + + + diff --git a/docs/usage/parameters.md b/docs/usage/parameters.md index 5edd3e1..d08e8eb 100644 --- a/docs/usage/parameters.md +++ b/docs/usage/parameters.md @@ -5,25 +5,6 @@ description: Customising parameters for the CO2e calculation. ## Customising parameters -You can adjust the nf-co2footprint plugin parameters in your config file as follows: - -```groovy title="nextflow.config" -def co2_timestamp = new java.util.Date().format( 'yyyy-MM-dd_HH-mm-ss') - -co2footprint { - traceFile = "${params.outdir}/co2footprint_trace_${co2_timestamp}.txt" - reportFile = "${params.outdir}/co2footprint_report_${co2_timestamp}.html" - ci = 300 - pue = 1.4 -} -``` - -Include the config file for your pipeline run using the `-c` Nextflow parameter, for example as follows: - -```bash -nextflow run nextflow-io/hello -c nextflow.config -``` - The following parameters are currently available: - `traceFile`: Name of the TXT carbon footprint report containing the energy consumption, the estimated CO2 emission and other relevant metrics for each task. @@ -32,22 +13,23 @@ Default: `co2footprint_trace_.txt`. Default: `co2footprint_summary_.txt`. - `reportFile`: Name of the HTML report containing information about the entire carbon footprint, overview plots and more detailed task-specific metrics. Default: `co2footprint_report_.html`. -- `ci`: carbon intensity of the respective energy production. Mutually exclusive with the `location` parameter. +- `ci`: Carbon intensity of the respective energy production. Available regions can be found [here](https://github.com/nextflow-io/nf-co2footprint/blob/1.0.0-beta/plugins/nf-co2footprint/src/resources/CI_aggregated.v2.2.csv), otherwise you can use data from [ElecricityMaps](https://app.electricitymaps.com/map/24h), to find out locally more accurate CI. +Mutually exclusive with the `location` parameter. Default: 475. - `location`: location code to automatically retrieve a location-specific CI value. For countries, these are [ISO alpha-2 codes](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2). For regions, it’s the ISO alpha-2 code for the country, followed by an identifier for the state, e.g. US-CA for “California, USA”. -You can find the available data [here](../../plugins/nf-co2footprint/src/resources/CI_aggregated.v2.2.csv). +You can find the available data [here](https://github.com/nextflow-io/nf-co2footprint/blob/1.0.0-beta/plugins/nf-co2footprint/src/resources/CI_aggregated.v2.2.csv). Mutually exclusive with the `ci` parameter. Default: `null`. -- `pue`: power usage effectiveness, efficiency coefficient of the data centre. +- `pue`: Power usage effectiveness, efficiency coefficient of the data centre. For local cluster you can usually find out your specific PUE at the system administrators or system managers. Also the current [yearly worldwide average](https://www.statista.com/statistics/1229367/data-center-average-annual-pue-worldwide/) could be used. Default: 1.67. - `powerdrawMem`: power draw from memory. Default: 0.3725. - `customCpuTdpFile`: Input CSV file containing custom CPU TDP data. This should contain the following columns: `model`,`TDP`,`n_cores`,`TDP_per_core`. Note that this overwrites TDP values for already provided CPU models. -You can find the by default used TDP data [here](../../plugins/nf-co2footprint/src/resources/TDP_cpu.v2.2.csv). +You can find the by default used TDP data [here](https://github.com/nextflow-io/nf-co2footprint/blob/1.0.0-beta/plugins/nf-co2footprint/src/resources/TDP_cpu.v2.2.csv). Default: `null`. - `ignoreCpuModel`: ignore the retrieved Nextflow trace `cpu_model` name and use the default CPU power draw value. This is useful, if the cpu model information provided by the linux kernel is not correct, for example, in the case of VMs emulating a different CPU architecture. diff --git a/mkdocs.yml b/mkdocs.yml index 6169abb..b796168 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -2,14 +2,16 @@ site_name: nf-co2footprint repo_name: nextflow-io/nf-co2footprint repo_url: https://github.com/nextflow-io/nf-co2footprint site_url: https://nextflow-io.github.io/nf-co2footprint/ -edit_uri: edit/dev/docs/ # TODO: change on release to edit/main/docs/ +edit_uri: edit/master/docs/ nav: - Home: - index.md - co2footprint.md - Usage: + - usage/configuration.md - usage/parameters.md + - usage/output.md - Contributing: - contributing/setup.md