BUG: installing celer with pip from github first fails, then succeeds

[mathurinm]

To reproduce in a newly created environement test where pip in installed:

(test) ➜  benchOpt git:(pip2conda) ✗ which pip
/home/mathurin/miniconda3/envs/test/bin/pip
(test) ➜  benchOpt git:(pip2conda) ✗ pip install git+https://github.com/mathurinm/celer.git
Collecting git+https://github.com/mathurinm/celer.git
  Cloning https://github.com/mathurinm/celer.git to /tmp/pip-req-build-wmet9675
  Running command git clone -q https://github.com/mathurinm/celer.git /tmp/pip-req-build-wmet9675
Collecting numpy>=1.12
  Using cached numpy-1.18.4-cp37-cp37m-manylinux1_x86_64.whl (20.2 MB)
Collecting seaborn>=0.7
  Using cached seaborn-0.10.1-py3-none-any.whl (215 kB)
Collecting scipy>=0.18.0
  Using cached scipy-1.4.1-cp37-cp37m-manylinux1_x86_64.whl (26.1 MB)
Collecting matplotlib>=2.0.0
  Using cached matplotlib-3.2.1-cp37-cp37m-manylinux1_x86_64.whl (12.4 MB)
Requirement already satisfied: Cython>=0.26 in /home/mathurin/miniconda3/envs/test/lib/python3.7/site-packages (from celer==0.5.dev0) (0.29.17)
Collecting scikit-learn>=0.23
  Using cached scikit_learn-0.23.0-cp37-cp37m-manylinux1_x86_64.whl (7.3 MB)
Collecting xarray
  Using cached xarray-0.15.1-py3-none-any.whl (668 kB)
Collecting download
  Using cached download-0.3.5-py3-none-any.whl (8.8 kB)
Collecting tqdm
  Using cached tqdm-4.46.0-py2.py3-none-any.whl (63 kB)
Collecting pandas>=0.22.0
  Using cached pandas-1.0.3-cp37-cp37m-manylinux1_x86_64.whl (10.0 MB)
Collecting kiwisolver>=1.0.1
  Using cached kiwisolver-1.2.0-cp37-cp37m-manylinux1_x86_64.whl (88 kB)
Collecting cycler>=0.10
  Using cached cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting python-dateutil>=2.1
  Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting threadpoolctl>=2.0.0
  Using cached threadpoolctl-2.0.0-py3-none-any.whl (34 kB)
Collecting joblib>=0.11
  Using cached joblib-0.14.1-py2.py3-none-any.whl (294 kB)
Requirement already satisfied: setuptools>=41.2 in /home/mathurin/miniconda3/envs/test/lib/python3.7/site-packages (from xarray->celer==0.5.dev0) (46.2.0.post20200511)
Collecting six
  Using cached six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting requests
  Using cached requests-2.23.0-py2.py3-none-any.whl (58 kB)
Collecting pytz>=2017.2
  Using cached pytz-2020.1-py2.py3-none-any.whl (510 kB)
Requirement already satisfied: certifi>=2017.4.17 in /home/mathurin/miniconda3/envs/test/lib/python3.7/site-packages (from requests->download->celer==0.5.dev0) (2020.4.5.1)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Using cached urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
Collecting idna<3,>=2.5
  Using cached idna-2.9-py2.py3-none-any.whl (58 kB)
Collecting chardet<4,>=3.0.2
  Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Building wheels for collected packages: celer
  Building wheel for celer (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/mathurin/miniconda3/envs/test/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-wmet9675/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-wmet9675/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-oygzm2ad
       cwd: /tmp/pip-req-build-wmet9675/
  Complete output (3178 lines):
  /home/mathurin/miniconda3/envs/test/lib/python3.7/site-packages/setuptools/dist.py:454: UserWarning: Normalizing '0.5dev' to '0.5.dev0'
    warnings.warn(tmpl.format(**locals()))
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/celer
  copying celer/homotopy.py -> build/lib.linux-x86_64-3.7/celer
  copying celer/__init__.py -> build/lib.linux-x86_64-3.7/celer
  copying celer/plot_utils.py -> build/lib.linux-x86_64-3.7/celer
  copying celer/dropin_sklearn.py -> build/lib.linux-x86_64-3.7/celer
  running build_ext
  cythoning celer/lasso_fast.pyx to celer/lasso_fast.cpp
  warning: celer/lasso_fast.pyx:159:37: Index should be typed for more efficient access
  cythoning celer/cython_utils.pyx to celer/cython_utils.cpp
  /home/mathurin/miniconda3/envs/test/lib/python3.7/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-req-build-wmet9675/celer/cython_utils.pxd
    tree = Parsing.p_module(s, pxd, full_module_name)
  
  Error compiling Cython file:
  ------------------------------------------------------------
  ...
  # License: BSD 3 clause
  
  cimport cython
  cimport numpy as np
  
  from scipy.linalg.cython_blas cimport ddot, dasum, daxpy, dnrm2, dcopy, dscal
  ^
  ------------------------------------------------------------
  
  celer/cython_utils.pyx:9:0: 'scipy/linalg/cython_blas.pxd' not found
## many cython errors

  celer/cython_utils.cpp:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
   #error Do not use this file, it is the result of a failed Cython compilation.
    ^~~~~
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for celer
  Running setup.py clean for celer
Failed to build celer
Installing collected packages: numpy, scipy, six, python-dateutil, pytz, pandas, kiwisolver, cycler, pyparsing, matplotlib, seaborn, threadpoolctl, joblib, scikit-learn, xarray, tqdm, urllib3, idna, chardet, requests, download, celer
    Running setup.py install for celer ... done
Successfully installed celer-0.5.dev0 chardet-3.0.4 cycler-0.10.0 download-0.3.5 idna-2.9 joblib-0.14.1 kiwisolver-1.2.0 matplotlib-3.2.1 numpy-1.18.4 pandas-1.0.3 pyparsing-2.4.7 python-dateutil-2.8.1 pytz-2020.1 requests-2.23.0 scikit-learn-0.23.0 scipy-1.4.1 seaborn-0.10.1 six-1.14.0 threadpoolctl-2.0.0 tqdm-4.46.0 urllib3-1.25.9 xarray-0.15.1

[mathurinm]

This was due to scipy not being installed in the env. In 0.5 we install requirements in the setup.py so this problems disappears.

[mathurinm]

This continued to happen in fresh envs, because cython was not installed when running setup.py, hence the .pyx files were not translated into .cpp and gcc complained that it could not find them