Bug when using Shapely in Windows conda

[philippfischer-sd]

Reproducing the Bug

The bug is very easy to reproduce by following these steps:

# Create a conda environment in Windows
conda create -n shapelybugpip python=3.8
conda activate shapelybugpip

# Inside the environment, use pip to install Shapely
pip install Shapely

# Then import some part of shapely that uses the geos library
python -c "import shapely.geometry"

Resulting Error

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\...\Miniconda3\envs\shapelybugpip\lib\site-packages\shapely\geometry\__init__.py", line 4, in <module>
    from .base import CAP_STYLE, JOIN_STYLE
  File "C:\...\Miniconda3\envs\shapelybugpip\lib\site-packages\shapely\geometry\base.py", line 19, in <module>
    from shapely.coords import CoordinateSequence
  File "C:\...\Miniconda3\envs\shapelybugpip\lib\site-packages\shapely\coords.py", line 8, in <module>
    from shapely.geos import lgeos
  File "C:\...\Miniconda3\envs\shapelybugpip\lib\site-packages\shapely\geos.py", line 154, in <module>
    _lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))
  File "C:\...\Miniconda3\envs\shapelybugpip\lib\ctypes\__init__.py", line 381, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'C:\...\Miniconda3\envs\shapelybugpip\Library\bin\geos_c.dll' (or one of its dependencies). Try using the full path with constructor syntax.

How to Fix

The reason is an error in this function:
https://github.com/Toblerity/Shapely/blob/8ef5b092750e3499893f990ab6498dca17dc2c80/shapely/geos.py#L59

The function doesn't actually do what is says. It simply checks if there is a conda environment, but not if shapely is installed as a conda package or as a pip package inside that environment.

There are multiple ways to fix it:

1. Don't use a hard if-else in this line https://github.com/Toblerity/Shapely/blob/8ef5b092750e3499893f990ab6498dca17dc2c80/shapely/geos.py#L159 but instead use the conda path as an additional search path (should be first), or if loading the conda lib fails, fallback to the pip loader below.
2. (cleaner) Fix the exists_conda_env() function to check if there actually is a shapely*.json file in the conda-meta folder and not just if the folder exists.

[sgillies]

@philippfischer-sd the good news is that the whole category of issues connected to loading GEOS at runtime using ctypes and libffi is going away in version 2.0. The bad news is that we're not going to do anything about this problem in 1.7 or 1.8. Meanwhile, there's a very good workaround: install shapely from conda-forge, not PyPI.

[philippfischer-sd]

Thanks @sgillies, unfortunately the conda workaround won't work for me, since I am releasing a PyPI package myself, which in its requirements (setup.py) has Shapely listed.
But good to know, that version 2.0 will not have the problem.

[sgillies]

And I've just added a long overdue question to the FAQ about this: https://github.com/Toblerity/Shapely/blob/master/FAQ.rst#i-installed-shapely-in-a-conda-environment-using-pip-why-doesnt-it-work.

[jorisvandenbossche]

unfortunately the conda workaround won't work for me, since I am releasing a PyPI package myself, which in its requirements (setup.py) has Shapely listed.

It's relatively easy to add your own package to conda-forge as well, so you your users have the option to use conda to install your package and automatically get Shapely from there as well (https://conda-forge.org/docs/maintainer/adding_pkgs.html)

Further, I just made a PR with a variation of what you suggested: #1108. Testing whether that change works for your situation is very welcome.

[philippfischer-sd]

unfortunately the conda workaround won't work for me, since I am releasing a PyPI package myself, which in its requirements (setup.py) has Shapely listed.

It's relatively easy to add your own package to conda-forge as well, so you your users have the option to use conda to install your package and automatically get Shapely from there as well (https://conda-forge.org/docs/maintainer/adding_pkgs.html)

Further, I just made a PR with a variation of what you suggested: #1108. Testing whether that change works for your situation is very welcome.

Awesome! Works for me. @sgillies: Is there a chance it can be included in the next version 1.8? Currently there is only an alpha version 1.8a1

[jorisvandenbossche]

Thanks for testing!
Yes, it will be included in Shapely 1.8