Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eurostat 2023 data for energy totals #947

Merged
merged 19 commits into from
Mar 5, 2024
Merged

Eurostat 2023 data for energy totals #947

merged 19 commits into from
Mar 5, 2024

Conversation

toniseibold
Copy link
Contributor

@toniseibold toniseibold commented Feb 23, 2024

Changes proposed in this Pull Request

Reads in the latest eurostat report from 2023 and uses it for building energy totals.
Since the JRC IDEES data is only available for 2015, the energy balances are scaled by the factor of eurostat[report_year]/eurostat[2015] data as well.
The latest swiss energy totals are added to ensure consistent energy totals.

To complement the PR, please add the eurostat data from 2023 to the zenodo data.

Missing is only the passenger car efficiency and the number of passenger cars that are initially in the JRC IDEES but not in the eurostat data set.

Comparison between 2023 and 2016 reports for the year 2013 show no difference except for Norway and Switzerland.
energy_totals_comparison.csv

Checklist

  • I tested my contribution locally and it seems to work fine.
  • Code and workflow changes are sufficiently documented.
  • Changed dependencies are added to envs/environment.yaml.
  • Changes in configuration options are added in all of config.default.yaml.
  • Changes in configuration options are also documented in doc/configtables/*.csv.
  • A release note doc/release_notes.rst is added.

@toniseibold toniseibold marked this pull request as ready for review February 23, 2024 12:57
@lisazeyen
Copy link
Contributor

@toniseibold great! Just a few remarks:

  1. in the long-term we would like to move away from the Zenodo sector-bundle repository. Can you add to the retrieve_sector_databundle.py some lines which download the necessary files from Eurostat directly?
  2. The new data set is still also containing the previous years and the UK, so there is no need to keep the old ones, right? This simplifies the script, you can use as an input only the new one an remove the if-else statement depending on which Eurostat input you use.
  3. Can you do some comparison of the output files between previous Eurostat and the new?

Copy link
Contributor

@FabianHofmann FabianHofmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just some stylistic comments

scripts/build_energy_totals.py Outdated Show resolved Hide resolved
scripts/build_energy_totals.py Outdated Show resolved Hide resolved
scripts/build_energy_totals.py Outdated Show resolved Hide resolved
@FabianHofmann
Copy link
Contributor

  1. in the long-term we would like to move away from the Zenodo sector-bundle repository. Can you add to the retrieve_sector_databundle.py some lines which download the necessary files from Eurostat directly?

I think, a separate rule retrieve_eurostat would be better to have a clear distinction

@lisazeyen
Copy link
Contributor

@toniseibold
Have you done some comparison of the output files between previous Eurostat and the new? Can you plot it here or confirm that the outputs are similar? The CI is failing for the script build_energy_totals.py can you make sure that it is running through? It seems to be connected with the swiss data and the year 2019

@fneum
Copy link
Member

fneum commented Mar 5, 2024

Checked output of energy_totals.csv before and after and value changes are in an acceptable range.

@fneum fneum enabled auto-merge March 5, 2024 18:23
@fneum fneum merged commit b1074e4 into master Mar 5, 2024
6 checks passed
@toniseibold toniseibold deleted the energy_totals branch June 18, 2024 07:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants