Pipy release

[alan-barzilay]

Is there a reason as to why the newest pyparted version was not pushed to pypi?

[Torxed]

In spirit of other Red Hat communications (molecule#4067), as a packager I would very much appreciate if we could get v3.13.0 pushed to PyPi.

[dcantrell]

I no longer understand how to publish things in PyPI. I could figure it out [again], but I just haven't had time. At one point I noticed some random person had published something under the name 'pyparted' which was a release I made, but it wasn't done by me and that prevented me from using the name 'pyparted'. If I remember correctly.

But I'll put this on the to do list of things to figure out, hopefully before PyPI is deprecated and is replaced.

FWIW, For releases I make signed tags in git and attach the release tarball and signature to a Release announcement here. I don't retag anything. If something needs changing, I make a new release. Checking for new releases here can also be done with git:

git ls-remote --tags https://github.com/dcantrell/pyparted.git

[dcantrell]

I have reset my PyPI account and updated everything that was outdated on it and enabled 2FA. Next I need to read through how to set up GitHub publishing, but I won't be able to get to that until I return to work. I am currently on PTO/company shutdown and have family obligations.

[alan-barzilay]

wait, Im confused. You are the owner of this package right? https://pypi.org/project/pyparted/
Are you saying this package wasnt always under your control?

And you dont need to publish a package to pypi through github, you can do it in the CLI. If you already have a tarball and all this should be pretty straightforward

[Torxed]

Tbh setting up a GitHub runner that does PyPi publishing for you is the smarter way from what I'm hearing.

Our runner isn't the gold standard, but it works:
https://github.com/archlinux/archinstall/blob/master/.github/workflows/python-publish.yml

You'll need to go to your project settings on PyPi, generate a auth token.
Then go to GitHub poroject settings -> "Secrets and variables" -> "Actions" -> In the "Repository secrets" press "Add new repository secret".

Then add two variables:

• PYPI_USERNAME
• PYPI_PASSWORD

This is if you want a copy paste friendly example and instructions that match.
Any time you release a new version (not a tag, but a release), it will publish to pypi for you.

Runner in action: https://github.com/archlinux/archinstall/actions/runs/8187426692
PyPi result: https://pypi.org/project/archinstall/#history

[alan-barzilay]

@dcantrell while you get the automation of publishing to pypi sorted out, how about manually pushing version 3.13 to pypi? I' m still confused as to why this hasnt happened yet, if the newest version isn't in pypi your users wont ever use it (most wont even realise it exists, we only found it out do to the still unexplained downgrade that broke my pipeline)

[Torxed]

The pypi docs for local publishing was a bit meh to follow. Perhaps it's tricky to get the local conf sorted?

[svartkanin]

Any updates on this?

[Torxed]

Seeing as setting up the automation didn't really stick.
Could we perhaps get a manual update instead?
Here's a real simple walkthrough:

1. $ python setup.py sdist bdist_wheel
2. $ pip install twine
3. $ twine upload --username '__token__' dist/*
   (It should actually be __token__, as it indicates you're using a project token to login, when asked for a password, enter your access token for the project, looks like this: pypi-AgEIcH..)

If you don't have access to your __token__, and you didn't store it in ~/.pypirc - then just log in to https://pypi.org/manage/account/ and scroll down to API tokens.

Press Add API token and enter:

1. Token name: pyparted-pypi
2. Scope: Project: pyparted

And proceed to compile + upload as instructed above.

The reasoning

The reason why we need this, is that we do as you point out:

For releases I make signed tags in git and attach the release tarball and signature to a Release announcement here.

We look for new releases (often using nvchecker), we get a notification about it, and begin testing our applications against the new pyparted version. We then lock the pyparted version that we just tested - such as in archinstall pyparted==3.13.0.

However, when people do pip install archinstall they will fail because there's no 3.13.0 on pypi.org. And that's a real, honest blocker. And has been for 6 months now.
Downgrading is really not an option either, because if i recall correctly 3.12.0 had some issues that made integration a hell..

---

We're really trying our best here to help out, and short of offering to jump on a jitsi call, I believe 2-3 commands to get this out of the way - is the simplest and quickest solution to this.

No code/changes needed, we just need the 3.13.0 tag to be pushed to pip

[Torxed]

@dcantrell here's a step-by-step guide!

We're still in dire need of that release.
Using pyparted as a dependency in workarounds does not work for pypi.org - only for 'pip install`:

step-by-step guide

$ pip install setuptools build installer twine
$ git clone https://github.com/dcantrell/pyparted.git /tmp/pyparted
$ cd /tmp/pyparted
$ git checkout v3.13.0
$ python -m build . --wheel
$ TWINE_USERNAME=__token__ twine upload dist/*
Enter your API token:       <-- Paste your access token here

Generate a new token here: https://pypi.org/manage/account/token/

After you're done, remove the token if you're unsure of how to store it safely. And then there's no harm done.

[dcantrell]

Wow, you guys are really unable to use pyparted unless it's on PyPI? That is extremely surprising to me. Getting PyPI publishing automation or whatever set up has been one of the lowest priority things for me in the last 6 months, I just haven't looked at doing it. I guess if people are incapable of getting source from here and must need it on PyPI, I will give it higher priority.

It is worth noting that pyparted is not really like other Python modules. It is really dependent on the version of GNU parted you have and pip doesn't really have a way to tell you that information about the system. Distribution vendors are better equipped to ensure pyparted and parted are built and linked together for their platform.

On the other hand, I'm not a cop. Use it however you want. Sorry I didn't give PyPI higher priority. I will work on this next week and see if I can get it working.

[Torxed]

As you mentioned, to make sure any distribution is mapping pyparted against the correct GNU parted we need to package against immutable upstream python artifacts (to have known states of packages). That eliminates the use github for sourcing artifacts (again, even Red Hat which you are familiar with strongly advise against it).

Packagers like myself therefore have to rely on PyPi for python artifacts. With known versions that work with the systems GNU parted.

This is why this is so darn important to get this to PyPi so we can lock it against the latest stable that is known to work with in my case the systems GNU parted.

So I would greatly appreciate it if you manage to find time to work on this the next week, and send a ping if we can help in any capacity.

[dcantrell]

I disagree that immutable upstream release artifacts are not available via GitHub. All releases are tagged and you can pull source from that specific tag. I also sign all of the tags if you want an additional step for verification. But beyond that, I've always uploaded the generated tarball release when I make a release. Sure, GitHub could vanish and those would all go away, but this is no different than any other download site vanishing. If I lost all the tarballs anyway, I would recreate them from the signed git tags.

But that's just my personal take on it. I know there are a lot of processes out there. I can't do it right now, but on Monday I should be able to work on getting the entry on PyPI updated.

[dcantrell]

I have published v3.13.0 to PyPI after resetting all of my PyPI login information. Future releases will once again be posted to PyPI when I make the release on GitHub. Thank you, everyone, for the pointers and such on how to fix my PyPI account.

I only uploaded the source archive as a prebuilt wheel does not make sense to me. In fact, PyPI wouldn't take it and instead wanted a "manylinux" wheel. But again, I don't think anyone actually wanted that. Just the source archives, right?

[Torxed]

Just the source archives, right?

Correct, sources are fine. I think most distros build during packaging.
At least Arch Linux runs:

• check()
• build()
• package()

And all we need is sources, except in some very rare occations.

Well done, and thank you!