Update IBTrACS to version 4.1

[ChrisFairless]

Changes proposed in this PR:

This year IBTrACS updated from version 4.0 to 4.1 and on May 28th the last v4.0 dataset was posted to their site.

That means anyone using CLIMADA is using IBTrACS data that hasn't been updated since then, even if they're regularly redownloading the file.

This PR points everything to the location of the new download and updates the tests because some of the tested values have changed.

Changes in v4.1:

• There's a new provider, KMA, the South Korean agency, with associated columns. I haven't updated CLIMADA for that: this would require someone to find out their wind averaging period and add it to tc_tracks.py
• It looks like a lot of values have been changed but I haven't looked into what and how much. Many appear to be due to a change in rounding somewhere in the data creation (lat, lon, pressure, wind). Since IBTrACS only gather and summarise data from other bodies, there shouldn't be big changes. Either way, NOAA consider this an improvement on previous versions of the data and we don't have a choice about using it. I had to update some of the tests because of this
• The TRACK_TYPE column now includes 'PROVISIONAL' as a possible value, which labels provisional US data (A-Deck). These are updated to 'Best Tracks' (B-Deck) by HURDAT after each season ends. It's information only, pointing out something that IBTrACS did already. Nothing in CLIMADA Core uses this column

The (rewritten) tests for the unit all pass.

Note:

• Users who have previously downloaded the v4.0 data will be left with it on their machine and might not be aware. It's ~25 MB so not tiny but not huge.

PR Author Checklist

• Read the Contribution Guide
• Correct target branch selected (if unsure, select develop)
• Descriptive pull request title added
• Source branch up-to-date with target branch
• Documentation updated
• Tests updated
• Tests passing
• No new linter issues
• Changelog updated

PR Reviewer Checklist

• Read the Contribution Guide
• CLIMADA Reviewer Checklist passed
• Tests passing
• No new linter issues

[peanutfun]

Thanks a lot for this PR, @ChrisFairless! 🙌 I am currently on parental leave and will not be able to review until January. Maybe somebody else would be willing? @spjuhel @NicolasColombi

[spjuhel]

Happy to give this a look next week!

[NicolasColombi]

Happy to have a look as well!

[spjuhel]

Everything looks in order for me.

👍 for replacing the hard-coded filenames!

[NicolasColombi]

Thank you @ChrisFairless! All good from my side.
Only one thing, the TC tutorial is failing in several cells due to different reasons (even on develop, so not because of this PR). One reason though is that it can't download the v04r01.nc tracks, but the problem seems to be in the noaa archive itself, as it can't be reached even by navigating to it manually. I would propose to merge this PR and we address the tutorial issues later on.

[chahank]

Please do NOT merge before solving the problem in the tutorial. This is the entire point of this PR. Also, we really do not want to push things that we know are failing, this is the perfect recipe for accumulating errors that will never be addressed. Not addressing an error is only acceptable if we know the reason for the error.

[ChrisFairless]

Yeah it looks like https://www.ncei.noaa.gov/data/ is down today. I get a 503 'Service unavailable' regardless of what I'm accessing. They had some issues earlier in the year too. We can wait and re-run the builds once it's back online.

[chahank]

Thanks for checking! Let's wait for NOAA to be available again, and if it then runs let's merge.

[chahank]

Ok, I fixed some minor issues with TCTracks and Hazard on the go. The unit tests are currently failing, not so clear why. Part of it is due to changes in petals, part is issues with the pre-commit hook that is failing with code 1, and part is internet connections. Let's try again later and see which ones are temporary.

[chahank]

Compatibility test with petals failing because the tests used the 4.0 version. @ChrisFairless please check CLIMADA-project/climada_petals#152

[emanuel-schmid]

@chahank @ChrisFairless @NicolasColombi : time to merge - right?

[chahank]

I just updated the changelog to make it clearer. For me it is good to go.

[emanuel-schmid]

🙌 to all

)I manually re-ran the failing compatibility tests on jenkins: apparently just a temporary glitch.)