Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Projects that depend on lxml fail to build in newer Pythons #75

Open
jaraco opened this issue Dec 9, 2022 · 3 comments
Open

Projects that depend on lxml fail to build in newer Pythons #75

jaraco opened this issue Dec 9, 2022 · 3 comments

Comments

@jaraco
Copy link
Owner

jaraco commented Dec 9, 2022

I support a number of projects that depend on lxml (svg.charts, openpack, jaraco.site, ...). Every time a new Python is released, lxml lags in supplying binary artifacts for that Python, meaning that builds will fail unless those platforms are excluded from the build matrix, requiring a per-project disablement of new platforms and then a later re-enablement. Furthermore, the rollout of binary artifacts for lxml is irregular, often landing on some platforms before others. I've tried building my own lxml artifacts, but found that the process isn't readily replicable and is often broken. It wouldn't be so bad if these projects could be configured with a PyPI mirror that had advance builds of lxml available, but supporting that is infeasible.

Maybe there's some way that the builds can be configured to fail quietly when lxml isn't available. It would need to be a tox plugin or wrapper, as the failure occurs before the test suite is run. Ideally, this step would be cheap enough that it could be run in all projects and not just the ones that depend on lxml.

@jaraco
Copy link
Owner Author

jaraco commented Dec 9, 2022

I looked at extending tox, but it doesn't seem to have a hook that would allow for suppressing an error during an install operation.

I also [searched](https://www.google.com/search?q=pip install --ignore failed dependencies) for a way to ignore failed dependencies in the installer, but there seems to be no such option.

Now I'm considering other options:

fake package

Consider a proxy package (lenient-lxml) that will always install but will only depend on lxml when it's available. That would require that this fake package be maintained to always capture where lxml is viable. It would still require the test suite to additionally detect when lxml is missing and skip relevant tests.

tox wrapper

A separate program could be installed that would first check if lxml is viable and only invoke tox if it might be expected not to fail. Such an operation would be expensive, so would need to be aggressively cached (ideally beyond a single project).

CI plugin

A CI plugin could do something similar to the tox wrapper, especially if a CI plugin could terminate the execution with a non-failure status.

hacked tox

Someone could hack the tox code to hook into the install step and inspect a failed pip failed output, allowing for specific failures.

custom installer

Someone could contribute to pip or hack the code or provide a custom installer to allow for failures.


None of these options are great.

@jaraco
Copy link
Owner Author

jaraco commented Dec 9, 2022

I've started work on the tox wrapper in jaraco/safety-tox that runs tox normally, but suppresses the failure if it's related to a known flaky package. So far, it's been rough going because it requires teeing output from a process, which is a difficult problem.

@jaraco
Copy link
Owner Author

jaraco commented Dec 9, 2022

I've spent all morning on this and it's still broken. I'm going back to just manually excluding environments that fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@jaraco and others