Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix snakemake dependencies in carbon budget runs #228

Closed
wants to merge 6 commits into from

Conversation

energyLS
Copy link
Contributor

@energyLS energyLS commented Mar 8, 2022

This PR should resolve issues when running the model with myopic foresight with a carbon budget defined by the 'cb' option. The main issue was that the Snakemake object was not passed to imported functions in prepare_sector_network.py.

The following steps have been done:

  1. move the snakemake dependencies of the affected functions to
    compulsory function arguments (in build_energy_totals.py)
  2. enter snakemake object values in the function calls
    (build_energy_totals and prepare_sector_network)
  3. add the new explicit file dependencies to the Snakefile at rule
    prepare_sector_network

Following other errors have occured and should be resolved:

  • The folder created for carbon_budget_distribution.csv included the csv file in the path itself when creating the folders, hence an error occured. Solved in commit
    78d1c92

  • The functions build_eurostat_co2 and build_co2_totals from build_energy_totals were missing the 'countries' argument. Solved in commit
    e9e0f3f

  • The function build_eurostat_co2 expects the country argument as pd.Index to run the line 25638e3#diff-132a8f9b549899b7eee1fc97ad04baa7c72a349df45041ba0bc1266e58813cacR640 successfully. Solved in commit 25638e3

Remaining questions:

  • Should the file dependencies to the Snakefile at rule
    prepare_sector_network added optionally if 'cb' keyword is in
    {sector_opts} wildcard)?
  • When running the full snakemake workflow, an error in plot_summary occurs: It tries to read 'results/run_name/csvs/countries.csv' which does not exist. This is no file specified as output in the Snakefile. Where should the 'countries.csv' (originally) be created?

Once the remaining questions are resolved, this PR could be transformed from Draft to ready. Coding improvements are highly welcome. Sorry for the long text ;)

@energyLS energyLS marked this pull request as draft March 8, 2022 18:10
@energyLS
Copy link
Contributor Author

energyLS commented Apr 4, 2022

Reason for error of missing "countries.csv" identified, the file creation was deleted in 1fc1d2a#diff-d838d3f5e300ee6371584a10781b96424387d2186d355c6cf601433ac3e189cbL126

Copy link
Member

@fneum fneum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @energyLS, and sorry for taking so long to review. Like the compulsory function argument for snakemake. Nice workaround.

Could you add a brief release note to doc/release_notes.rst?

I'd try not to re-introduce the countries.csv. It's just a list of included countries. Maybe one can retrieve it from the PyPSA-Eur config.yaml or n.buses.country.unique().

@energyLS
Copy link
Contributor Author

Hey @fneum thanks for approving. I will integrate your suggestions (release notes & retrieving countries from config.yaml) in approx. two weeks time.

@lisazeyen
Copy link
Collaborator

this can be closed since solved with #256

@lisazeyen lisazeyen closed this Aug 9, 2022
LukasFrankenQ pushed a commit to LukasFrankenQ/pypsa-eur-sec that referenced this pull request Apr 12, 2023
In some cases, in networks with DC links, a non-existing column would
be referenced in an empty Dataframe. This commit adds a check for this
case.

Co-authored-by: Koen van Greevenbroek <koen@vesoldo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants