Workflows failing in main

[andrea-pasquale]

After merging #270, there is an error in the workflows when creating the project.
See for example here. It seems to be related to furo.
@Edoardo-Pedicillo @alecandido could you please have a look?

[alecandido]

It's not necessarily related to "the workflow".

Of course is the workflow being failing, but as it would fail if the tests were failing. The problem is Poetry, in particular poetry install: it seems like the files inside furo wheel (the compressed archive you download from PyPI) are "corrupted", in the sense that they do not correspond to those who were downloaded when you created the lockfile (for the exact same version).

[alecandido]

Since they are classified as corrupted, Poetry is failing the installation.

Easiest potential solution: attempt lockfile regeneration with poetry lock, and push the new lockfile.

[andrea-pasquale]

Thanks for the explanation @alecandido.
I will proceed as you suggested.

[andrea-pasquale]

It seems that he didn't work @alecandido.
Any other suggestions?

[alecandido]

poetry update, it will try to get a new version. Maybe, since the dependencies were still consistent, it didn't anything in practice.

Btw, which version of Poetry are you using? You moved all the hashes at the bottom of the lockfile, but I believe this is not the behavior of the current Poetry, i.e. 1.4, but I'm not sure is even the behavior of the former 1.3...

EDIT: for me poetry update is a noop wrt to poetry lock, so in principle it should make no difference (I just pushed another commit with this version)

[andrea-pasquale]

Btw, which version of Poetry are you using? You moved all the hashes at the bottom of the lockfile, but I believe this is not the behavior of the current Poetry, i.e. 1.4, but I'm not sure is even the behavior of the former 1.3...

Yeah, my bad I'm still using poetry 1.2.2 :(
I've just updated to poetry 1.4.1.

[alecandido]

Nothing changed: I reproduced the problem locally, and without Furo there is no issue (just after running poetry remove furo), so we need to further investigate why Furo is now causing this issue.

The version of Poetry is just a detail, to avoid future inconvenients, but it seems unrelated to this specific problem.

[alecandido]

The problem is confirmed also with the latest version of Furo, 2022.12.7

[alecandido]

Furo uses a peculiar build backend https://github.com/pradyunsg/furo/blob/55b32f0dd5ac9a055629a8fbb8914e9c62a8b072/pyproject.toml#L1-L3 (by the same author, that is a senior contributor to Python packaging in general), it might have some incompatibilities with Poetry, I'm investigating

https://github.com/pradyunsg/sphinx-theme-builder

[alecandido]

It seems that there is a known problem with the hashes, but it is known since more than one year, not sure why we're affected only now
pradyunsg/sphinx-theme-builder#13

pypa/pip#11762 (comment)

[alecandido]

A hypothesis is that, since we are running the Poetry installation action in the workflow with default options, we just stepped into usage of Poetry 1.4.

Poetry 1.4 introduces the new installation "back-end" (i.e. a packaging front-end, but never mind), replacing the former usage of pip with direct usage of https://github.com/pypa/installer
https://python-poetry.org/blog/announcing-poetry-1.4.0/#faster-installation-of-packages

Now, this in principle should have no relation with the hashes, but you never know...

[Edoardo-Pedicillo]

It seems we have two possibilities:

• force the workflows to install Poetry 1.3
• Install Poetry 1.4 and disabling the new installer.
  @alecandido let me know what do you think about it.

[alecandido]

I tested yesterday evening, and indeed is working with 1.3.2 (until this it was only an hypothesis).

Actually, I'd still like to test it with 1.4.0, since it is not clear to me why we started failing only now, while 1.4.0 rolled out 20 days ago (and latest Furo is from December, but we are also using a version from September).

So, the first option I can tell it's working, but I'm not yet sure the second is actually an option.
In case it is, an alternative might be pushing a Poetry config file to Qibocal itself, and drop it the moment Furo will have fixed the issue.

In any case, I haven't checked yet Poetry issues, to see if other people encountered the same problem and have a more detailed explanation. Has someone done it?

[alecandido]

As (partially) expected from the timing, the installation works even with Poetry 1.4.0. I'm still not sure which one is the problem caused by 1.4.1, but it was reasonable it should've been related to that, since it rolled out yesterday https://github.com/python-poetry/poetry/releases/tag/1.4.1

Reading the changelog, the likeliest change responsible for the break should be:

Fix an issue where the hash of sdists was neither validated nor written to the direct_url.json during installation (python-poetry/poetry#7594).

So, my current hypothesis is that there is something broken with Furo sdist hashes, most likely because of sphinx-theme-builder, and it became checked only with Poetry 1.4.1.

Current advice: stick to Poetry 1.4.0.

The options are:

1. hard-code Poetry 1.4.0 in the reusable workflow
2. expose a further input to set the Poetry version

Given that the problem will be the same for all packages using Poetry and Furo, so all, I would rather choose 1., and update it as soon as the problem will be solved.
@Edoardo-Pedicillo can you take care of that?

[alecandido]

Actually, I forgot, but a better/alternative solution might be to deactivate the modern installation in the same workflow (instead of changing version), with a global config.

Just run

poetry config installer.modern-installation false

before poetry install.

python-poetry/poetry#7686 (comment)

[alecandido]

Here the issue (actually a discussion) on Furo

pradyunsg/furo#639