pot installation fails with numpy < 1.20

[hichamjanati]

Describe the bug

import ot fails with after pip install pot with numpy<=1.19.5

---> 22 from .emd_wrap import emd_c, check_result, emd_1d_sorted
     23 from .solver_1d import emd_1d, emd2_1d, wasserstein_1d
     24 

ot/lp/emd_wrap.pyx in init ot.lp.emd_wrap()

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

Fixes

Either we re-do the wheels with earlier versions of numpy or we require numpy >= 1.20. Since 1.20 however, numpy has dropped support of Python 3.6.

[rflamary]

This should be corrected by executing pip install with the following option --no-binary ":all:" that will not use the pre-compiled wheels. It is maybe better and future proof to compile as we do the wheels o the new version of numpy,n I don't see how we could do both in practice.

Python 3.6 support will be dropped in December so i'm all for forcing numpy >= 1.20 in a new bug correction release in january.

[ncourty]

The issue is also showing on google colab. The workarounnd suggested by rémi does not work here. For me, installing pot with
!pip install pot 'numpy>=1.20'
did the job

[rflamary]

I'm closing this bug, release 0.8.1 should have solved it (it now works on google collab with pip install POT)

[davidghiurco]

I think this exact same issue will surface again when built against numpy>=1.22, since numpy 1.20 and numpy 1.22 both contain backwards-breaking ABI changes.

Due to the lack of an upper bound in pyproject.toml on the requires=["numpy>=1.20"], I think the only reason the current latest POT 8.x wheels work is because they were built BEFORE numpy 1.22.0 was released on December 31st, 2021, see: https://pypi.org/project/numpy/1.22.0/

Might be a good idea to update pyproject.toml with requires = [... "numpy>=1.20, <1.22"], or otherwise bump the minimum required numpy version to >=1.22.

What do you think @rflamary? Should I make a separate issue for this?

EDIT: I confirmed my suspicion with a numpy engineer, see numpy/numpy#19479 (comment):

You have to compile against the oldest NumPy version you support. There is https://pypi.org/project/oldest-supported-numpy/ to help with that.

[rflamary]

yes please open a new issue with this discussion. I'm getting tired of those wheels ABI problems but I agree that using oldest-supported-numpy is a solution.On the other hand we need to use it only for building the wheels because I don't want to add another dependency.