release v0.18.0

[dcherian]

As discussed in the meeting, we should issue a release soon with the new backend refactor and the new docs theme.

Here's a list of blockers:

• open_mfdataset: Raise if combine='by_coords' and concat_dim=None #5231
• lock kwarg needs a deprecation cycle? #5073
• Fix exception when display_expand_data=False for file-backed array. #5235

Would be nice and look done:

• Add GitHub action for publishing artifacts to PyPI #5244
• Bump versions #5258
• Surface plots #5101
• Refactor line plotting #4866 (we should let this sit on master for a while to find bugs)
• also apply combine_attrs to the attrs of the variables #4902
• Automatic duck array testing - reductions #4972 (this should probably also sit on master for a while)
• coarsen: better keep_attrs #5227
• Faster interp #4740
• Convert attribute and dimension names to strings when generating HTML repr #5149

Somewhat important, but no PR yet:

• Feature to retrieve the underlying store #5175 (as pointed out by @shoyer, this is really a new feature, not a regression, it can wait)

@TomNicholas and @alexamici volunteered to handle this. I can be online at release time to help with things if needed.

Release instructions are here: https://github.com/pydata/xarray/blob/master/HOW_TO_RELEASE.md

IIRC they'll need to be added to the PyPI list and RTD list.

[Illviljan]

May I suggest #4740, the only thing this needs I think is a rebase. I recently got a new computer and was reminded how slow the interpolation is without these changes.

[alexamici]

I propose the following timeline that is the fastest I can personally try:

• Monday May the 3rd, feature freeze, please refrain from merging new features unless they are discussed here
• during feature freeze everybody please help reviewing the PR in the list above
• Thursday the 6th, target release day, the best would be to schedule a window for a couple of hours (?) for @TomAugspurger and me to going through the release steps together with @dcherian.

@TomAugspurger and others does it work for you all?

[TomNicholas]

@alexamici I think you tagged the wrong Tom, but yes that works for me.

[max-sixty]

I think @alexamici 's plan is reasonable. I've had less of a process — no freeze etc — and that's worked fine — we do have a good test suite, and I've been really trying to make the release process a normal thing anyone can do in an hour.

But it's your call (and there's not much practical difference).

Thanks again!

[dcherian]

IIRC they'll need to be added to the PyPI list

We can avoid this by using the pypi github action thing to automatically build and upload when tagging a release on github. It uses a repo-level secret. Just a thought.

and RTD list.

I added @alexamici . @TomNicholas you need to sign up on RTD before I can add you there as a maintainer

[dopplershift]

We can avoid this by using the pypi github action thing to automatically build and upload when tagging a release on github. It uses a repo-level secret. Just a thought.

$0.02 from an outsider is that this has served us exceedingly well on MetPy. Our release process has become:

1. Close milestone
2. Adjust the auto-generated draft GitHub release (summary notes)
3. Click "publish release" -> packages uploaded to PyPI
4. Merge conda-forge update from their bots

It's almost more secure this way because the token from PyPI only has upload permissions--no need to store someone's password.

[TomNicholas]

(@dcherian I've signed up on RTD now)

[keewis]

great, you're a maintainer now (for pint-xarray, too, btw)

[dcherian]

It'd be nice to bump our minimum dependency versions if someone has the time.

[alexamici]

Form what I can see the #5258 "bump versions" looks ready, someone with knowledge about minimum versions please merge it.

After that is in, @dcherian you can probably uncomment the var._meta bit in #4740 "faster interp" and we can merge it as well. Again I'm not a user of that part of code, and I would feel more conformable if someone with more knowledge would click on the "merge" button.

This leaves only #5235 as a blocker that has a failing test, a proposed fix by @keewis. but the OP hasn't responded yet. How should we handle it? This looks ready to merge now. Sorry @tomwhite I read the timestamps wrong. Great job!

[alexamici]

Any thoughts on merging #5244 and doing the v0.18.0 release via GitHub Actions?

I'm using GA in a couple of repositories and I find them very handy, but I usually had to make a couple of releases before getting all the details right, which looks awkward to do with a high profile project like xarray.

We could try release one or more release candidates and go for v0.18.0 only after we are sure the GA release procedure is working though.

[keewis]

we could also temporarily switch the uploads to test.pypi.org to verify, create debug versions (rc candidates?) and switch back to pypi.org once we're confident

[kmuehlbauer]

we could also temporarily switch the uploads to test.pypi.org

Why not always upload pushes to master to test.pypi.org?

[keewis]

I think that's because nobody had that idea, yet.

Edit: that would also help with the idea in #5244 (comment). Let's continue that discussion in #5244

[alexamici]

@TomNicholas & @dcherian I'm having a look at the what's new, shall we set a time for going through the release process? do we want do the steps while in a call?

[TomNicholas]

@alexamici yes sure - I'm free anytime today except for 2-3pm EST.

[alexamici]

@TomNicholas I'm in Europe, with a meeting at 17:30-18:30 CEST (11:30-12:30 EST) we can probably start now-ish and finish after 3pm EST. I propose to hijack the community meeting zoom :)

[dcherian]

We're still waiting on #5235.

Let's do 12.30 Eastern at https://columbiauniversity.zoom.us/j/953527251

[alexamici]

@dcherian 11:30-12:30 EST I'm meeting with Stephan and Joe, I hope to have them merge #5235 :)

12:30 EST is fine by me. I cannot make it :/

[keewis]

#5235 shouldn't be that much of an issue. There are two decisions to check: never collapse the repr of MemoryCachedArray and keep the array used to check that small (we have a limit of 1e5, after which the repr changes from array([...]) to [xyz values with dtype=...])

Edit: essentially: I don't want to merge it myself

[dcherian]

I don't have permissions to generate a PyPI token... I think either @jhamman or @shoyer need to do that.

[keewis]

uploading to TestPyPI works so uploading to PyPI should work, too, once we have the token

[alexamici]

@dcherian I'm sorry, I have cannot make it by 12:30 EST. I think you and @TomNicholas can go on without me or I will be back later.

[dcherian]

OK I think we can go ahead. 3 people is pretty overkill for the release process :P The token has been added and I'm in the Zoom room @TomNicholas

[alexamici]

Deleting and re-creating the release should work (I've done it on one of my projects)

[TomNicholas]

Okay let me try that then

[andersy005]

I accidentally closed this via Github's keywords... The bug in the CI workflow should be fixed.

[TomNicholas]

hmm should I also delete the tag? It appears to be also linked to one of @keewis ' PRs (#5272)

[andersy005]

hmm should I also delete the tag?

Yes...

[alexamici]

hmm should I also delete the tag? It appears to be also linked to one of @keewis ' PRs (#5272)

yes, sorry: delete release AND tag. Then re-create release.

[keewis]

yes, please. The command to create the tag should be

# empty commit unless you update the release date there
git commit --empty -am 'Release v0.18.0'
git tag -a v0.18.0 -m 'v0.18.0'

[andersy005]

@keewis, do we need to do anything with the release on test.pypi.org? (since the v0.18 is already published there)

[keewis]

errr... we can't do anything about that, PyPI and TestPyPI don't allow republishing the same version

[keewis]

maybe we should make the "upload to TestPyPI" a separate step?

[TomNicholas]

oh - I just deleted both and then re-released just now, but v0.18.0 again...

[TomNicholas]

Looks like it did push to PyPI this time though? https://github.com/pydata/xarray/runs/2521179293?check_suite_focus=true

[dcherian]

Yup, nice work everyone!

https://pypi.org/project/xarray/

[keewis]

ah, right, only the "push tag" workflow run failed

[alexamici]

I'm testing after pip install -U xarray just to be on the safe side 😄

[TomNicholas]

Alright I'll carry on with the other steps then I guess? 😅

[TomNicholas]

How many xarray devs does it take to do a release? Answer: at least 5 apparently

[alexamici]

How many xarray devs does it take to do a release? Answer: at least 5 apparently

🤦

But next time it will be sooooooo smooth!

[andersy005]

maybe we should make the "upload to TestPyPI" a separate step?

Or we could just set

skip_existing: true

on the upload to TestPyPI step.

[TomNicholas]

Okay that should be everything except emailing the mailing lists, which I will try now

[alexamici]

12899 passed, 1145 skipped, 211 xfailed, 63 xpassed, 321 warnings on my Mac.

[alexamici]

@TomNicholas feel free to close the issue once you are done.

[TomNicholas]

My email got bounced from pyaos@lists.johnny-lin.com because I'm not a member, but otherwise I think this is done. (I'm also not subscribed to numpy-discussion@scipy.org or scipy-user@scipy.org so maybe it's about to get bounced from there too).

[TomNicholas]

I updated the release guide in #5274 by the way.

[dcherian]

Nice work, @TomNicholas

[shoyer]

Thanks everyone, great work!

…

On Thu, May 6, 2021 at 1:06 PM Deepak Cherian ***@***.***> wrote: Nice work, @TomNicholas <https://github.com/TomNicholas> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5232 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJJFVVMYMHDRMUEOMZWR3LTMLZD3ANCNFSM43XW74VQ> .