Equivalent of pip install --no-binary

[spapanik]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Question

How can I specify that I don't want the binary package to be installed, ie. the equivalent of pip install --no-binary?

[jambonrose]

I would appreciate some clarity on this front as well. The currently recommended installation of psycopg2 is:

python3 -m pip install psycopg2>=2.7,<2.8 --no-binary psycopg2

What is the right way to install psycopg2 with Poetry?

Thanks!

[rectalogic]

This seems to work:

PIP_NO_BINARY="psycopg2" poetry install

[RevolutionTech]

Is there a solution with Poetry that involves a change to the pyproject.toml or something similar?

The team I work on is trying to install psycopg2 with --no-binary, and although I appreciate that your solution works well @rectalogic, it would be ideal if the solution wasn't co-ordinating that all developers on the team add an environment variable whenever executing poetry install (or update their local ~./bashrc, etc.).

[janw]

I agree with @RevolutionTech that this should be an option for dependencies in pyproject.toml, a no-binary flag. If this is still something to be done, I'd be happy to come up with an implementation.

[RevolutionTech]

In the interest of full transparency I should mention that this is no longer an urgent need for the team I work on since modern versions of psycopg2 do not install the binary version by default anyways.

That said, I still think that this would be a really great feature, and I am sure there are other packages besides psycopg2 where this presents a problem.

[b-long]

Related / duplicate: #1316 ?

[b-long]

I ran into an issue with the neuralcoref library (github, pypi) which was solved by changing my workflow to PIP_NO_BINARY="neuralcoref" poetry install .

A few questions:

1. Is this currently the best practice?
2. Is there any method of configuring this via pyproject.toml, poetry.lock, or poetry.toml ?
3. Are there any discussions in the Poetry namespace or https://github.com/pypa namespace to follow which might provide an answer to # 1 or # 2 ?

[spapanik]

Related / duplicate: #1316 ?

Yes, indeed

[jo7ueb]

I'm watching this issue as opencv-python requires --no-binary option in my case.

[rendaw]

This is not a duplicate of #1316 since this issue, as an install override, doesn't drag in the question of config files/lock file specifications, platform string matching, etc. There will always be systems that have nonstandard system libraries, patches, feature flags turned on/off that cause wheel incompatibility, where it doesn't make sense to force everyone to use source packages.

Another example, gevent==1.4.0 wheel doesn't work with new versions of Python 3.7 (Debian) due to binary incompatibility. Installing from source works, but PIP_NO_BINARY or other environment variables don't work. Manual fix is to pip remove gevent then install it manually with --no-binary after setting up the venv.

[rendaw]

PIP_NO_BINARY environment variable no longer works since (1.10?) since file resolution is being resolved by Poetry itself no longer pip.

[adriantre]

To install e.g. rasterio without wheels for GDAL, we need to provide the --no-binary flag. Current workaround is:

poetry add rasterio
poetry run pip install --force-reinstall rasterio --no-binary rasterio

which feels like bad practice.

[simon-keith]

We have a similar issue when installing geopandas with the pygeos backend, where we need to do :

pip install pygeos geopandas --no-binary pygeos,shapely

[kylebarron]

Reinstalling with pip --force-reinstall after poetry install is an interesting workaround. It's probably ideal to grab the version from the Poetry lockfile so that you're installing the same package version:

poetry add rasterio
rasterio_version=$(poetry show rasterio | grep version | cut -d ':' -f 2)
poetry run pip install --force-reinstall rasterio=="$rasterio_version" --no-binary rasterio

[sallamander]

Has anybody figured out how to get this into a poetry.lock and / or pyproject.toml file? The --force-reinstall works, but it'd be nice to be able to just do a poetry install from the toml or lock file and have that work.

Are there any plans to support this in the future if it's not already supported in some way?

[flyte]

None of these workarounds work when your project is published to PyPI and the user installs it with pip.

Our project is intended to run on small systems like Raspberry Pi, one of which (Zero W) is still being produced with a 32bit CPU. PyYAML doesn't include 32 bit binaries, so we need to make sure the source package is installed instead of the 64bit wheel.

Is there currently a workaround that will suit this use case? Changing the instructions from

pip install <project>

to

pip install --no-binary pyyaml pyyaml
pip install <project>

sucks.

EDIT: PyYAML was only failing to install because we were using --pre to install the pre-release version of the project, even though pip was downloading and using the exact same PyYAML .tar.gz package with and without --pre 🤷

[dazza-codes]

One option to try is to take full advantage of the dependency analysis that poetry provides to manage compatible versions and then export the lock file to a requirements.txt file, e.g.

poetry export --without-hashes --format requirements.txt --output requirements.txt
# it can help to remove editable flags:
sed -i -e 's/^-e //g' requirements.txt
python -m pip install \
    --force-reinstall \
    --no-binary pygeos,shapely,rasterio,pyproj \
    -r requirements.txt

Or build the package and then pip install it:

rm -rf dist/*
poetry build
python -m pip install \
      --no-compile \
      --force-reinstall \
      --no-binary fiona,pygeos,pyproj,shapely,rasterio,rtree \
      ./dist/*.tar.gz

The first approach can leverage the poetry.lock content since poetry export uses it (AFAIK). The second approach could take longer for pip to resolve the dependency tree, and possibly not as well as poetry.

In this approach, poetry might not install anything, pip does it all. In this approach, the env-var options should work when a build environment already provides shared libs that should be used for the python bindings.

[byoungb]

Another solution that appears to work for me is to use the url option and provide the source tar url.

cryptography = { url = "https://files.pythonhosted.org/packages/f9/4b/1cf8e281f7ae4046a59e5e39dd7471d46db9f61bb564fddbff9084c4334f/cryptography-36.0.1.tar.gz" }

It is not ideal, and a bit more work to maintain, but in my case it works.

Background:
I am running on AWS's aarch64, which for some reason the wheel created for aarch64 is not working... but everything works fine if it is built from source.

[joshfriend]

This is how I would have liked to see an option to skip wheels:

grpcio = { version = "^1.43.0", format = "sdist" }

or maybe to more closely match the pip flags:

grpcio = { version = "^1.43.0", no_binary = true }

(grpc/grpc#28387)

[hhstore]

poetry config:

• https://python-poetry.org/blog/announcing-poetry-1.2.0/#opting-out-of-binary-distributions

# Skip all binaries
$ poetry config --local installer.no-binary :all:

# Skip specific package binaries
$ poetry config --local installer.no-binary httpx,uvicorn

# Do not skip any binaries (default)
$ poetry config --local installer.no-binary :none:

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.