Add support for mechanized deployments as implemented in importlib_metadata. - [merged]

[jaraco]

In GitLab by @jaraco on Sep 11, 2018, 21:09

Merges feature/mechanized-releases -> master

Based on the work in python-devs/importlib_metadata#8, follow the same pattern here.

I'm marking this a WIP while I work out issues and consider python-devs/importlib_metadata!9.

[jaraco]

In GitLab by @jaraco on Sep 11, 2018, 21:12

added 1 commit

• a6fce685 - Add support for mechanized deployments as implemented in importlib_metadata.

Compare with previous version

[jaraco]

In GitLab by @jaraco on Sep 11, 2018, 21:15

added 1 commit

• 66669b7 - Add support for mechanized deployments as implemented in importlib_metadata.

Compare with previous version

[jaraco]

In GitLab by @codecov on Sep 11, 2018, 21:18

Codecov Report

Merging #67 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master    #67   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files           1      1           
  Lines         179    179           
  Branches       25     25           
=====================================
  Hits          179    179

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e6e4e46...2ebce63. Read the comment docs.

[jaraco]

In GitLab by @jaraco on Oct 31, 2018, 14:46

unmarked as a Work In Progress

[jaraco]

In GitLab by @jaraco on Oct 31, 2018, 14:48

I've decided it's important to get the mechanized releases out, especially to ensure that wheels are published with each release. I can separate the concern about version bumping, and follow up here if importlib adopts SCM-based versioning. I suggest incorporating these changes before the next release.

[jaraco]

In GitLab by @warsaw on Nov 1, 2018, 14:08

@jaraco Does this mean that each commit will create a new release? Or do I need to do something explicit to do the release? I definitely don't want the former.

Given that tox -e release (without this change) creates the sdist and wheel artifacts, is the problem just as "easy" as doing twine upload artifacts/*.{tar.gz,whl}?

[jaraco]

In GitLab by @jaraco on Nov 1, 2018, 15:03

Each tagged commit will produce a release (and only tags that match the regex in the .gitlab.yml). So basically, instead of a release process:

1. Bump version
2. Update changelog (commit)
3. Tag commit
4. Find a machine with bash
5. Ensure release dependencies. Run release.sh
6. Push
7. Upload dists, making sure you have credentials

You get this workflow:

1. Bump version
2. Update changelog (commit)
3. Tag commit
4. Push

There are several benefits to this approach:

• Releases depend on passing tests (in the general case).
• The tox tooling ensures that the right packages are available to cut the release.
• The upload username and password is configured once in the infrastructure instead of needing to be resolved on whichever developer is doing the release.
• It limits the possibility of a release being cut without pushing the code (since the incentive is to just tag and push).
• This process mirrors the one already implemented in importlib_metadata, which has worked well for several releases.
• Builds are made consistently in one environment (GitLab CI/CD) instead of by individuals where environments vary.
• Technique runs cross-platform (no bash dependency).
• Saves on (presumed) unnecessary steps building in one directory and moving to another.

is the problem just as "easy" as doing twine upload?

Yes, perhaps, but having a process that relies entirely on tox instead of tox+bash seems superior to me. If you wish to keep the pip wheel invocation instead of setup.py bdist_wheel, I could go for that. I'd probably still build with -w dist to build to the same target and then upload from there, rather than building to separate targets and moving them.

Whatever we decide, I'd like to see tagged commits being automatically released and importlib_* to be harmonized to that process.

[jaraco]

In GitLab by @warsaw on Nov 1, 2018, 18:09

Thanks for the detailed description @jaraco - I agree that is much better.

A couple of comments/questions remain:

• Do we need to set TWINE_USERNAME and TWINE_PASSWORD somewhere in the GitLab settings? And do we need to add more collaborators to importlib_resources on PyPI? I honestly don't remember what we needed to do to make it work in importlib_metadata.

I have no opinion on using pip wheel vs setup.py bdist_wheel. You're the expert on that so I'll totally defer to you. :)

The tag pattern restriction wasn't obvious from looking at the .gitlab.yml file, but maybe it would be if I looked up the semantics of the only key. Is it possible and worthwhile to add a comment?

Should we capture the steps needed to do a release in a DEVELOPER.rst top level file or something?

[jaraco]

In GitLab by @warsaw on Nov 1, 2018, 18:09

assigned to @warsaw

[jaraco]

In GitLab by @jaraco on Nov 1, 2018, 20:35

Should we capture the steps needed to do a release in a DEVELOPER.rst top level file or something?

I was thinking the same thing. I can do that.

Do we need to set TWINE_USERNAME and TWINE_PASSWORD somewhere in the GitLab settings?

I don't remember if we set it on the project or the group. If we set it on the group, then we won't need to set it again for this project. I'll check the project history.

Confirmed - passwords are already configured for python-devs. But it does require that tags be configured to be protected. That is now done and can be seen in this page.

And do we need to add more collaborators to importlib_resources on PyPI?

Yes. It's my credentials that are currently configured, so you have two options - add me as a maintainer on importlib_resources (pypi) or update the credentials with those of someone who has maintainer access on both.

That should be it.

[jaraco]

In GitLab by @jaraco on Nov 1, 2018, 20:38

added 1 commit

• 2ebce63 - Add DEVELOPER.rst

Compare with previous version

[jaraco]

In GitLab by @warsaw on Nov 1, 2018, 20:44

I've updated the ownership records on PyPI for this project to match importlib_metadata, where @warsaw, @jaraco, and @brettcannon are all owners.

I would like to continue to cut importlib_resources releases for now. I guess we can't have multiple values for the TWINE_* environment variables.

[jaraco]

In GitLab by @warsaw on Nov 1, 2018, 20:49

@jaraco Man, I really don't like exposing my (or really, any person's) PyPI credentials in those variables. It looks like there's no other way to do it. What do you and @brettcannon think about creating a shared account on PyPI and using it to do mechanized uploads?

[jaraco]

In GitLab by @warsaw on Nov 1, 2018, 20:51

FWIW, we'd need a unique contact address and username. I can request one on @python.org if you're amenable to this approach.

[jaraco]

In GitLab by @jaraco on Nov 1, 2018, 21:17

Looks like warehouse is tantalizingly close to having support for access tokens. It may be worth waiting to see how that change pans out and if it will address our use case. In fact, I'll update that ticket to ask.

[jaraco]

In GitLab by @warsaw on Nov 1, 2018, 21:48

Oh nice. I'll subscribe to that issue. We're in no hurry (importlib_resources being pretty stable at this point), so I agree we can wait a bit and see how it goes.

[jaraco]

In GitLab by @jaraco on Nov 3, 2018, 11:38

In the meantime, my password is already exposed, so there's no reason to delay acceptance of this MR.

[jaraco]

In GitLab by @warsaw on Nov 5, 2018, 16:27

approved this merge request

[jaraco]

In GitLab by @warsaw on Nov 5, 2018, 16:27

merged

[jaraco]

In GitLab by @warsaw on Nov 5, 2018, 16:27

mentioned in commit 02766ca

[jaraco]

In GitLab by @warsaw on Nov 5, 2018, 16:27

Thanks @jaraco !