Skip to content

Cannot install scipy or anything that depends on it, like scikit-learn via pip #558

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rocketraman opened this issue Nov 4, 2020 · 4 comments
Labels
question Usability question, not directly related to an error with the image

Comments

@rocketraman
Copy link

rocketraman commented Nov 4, 2020

This image makes it very difficult (impossible?) to install numpy, scipy, and scikit-learn via pip, which as per #43 should be the recommended method. As I noted in a comment to that issue which was marked as spam (I admit the comment may have been unhelpful, but it certainly wasn't spam), this does not seem to work well.

Example Dockerfile:

FROM python:3.9.0-buster
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip3 install  --user -r requirements.txt

and requirements.txt as recommended by the scikit docs (https://scikit-learn.org/stable/install.html#install-official-release):

scikit-learn[alldeps]==0.21.1
numpy==1.17.2

results in output like this:

STEP 1: FROM python:3.9.0-buster
STEP 2: WORKDIR /usr/src/app
--> Using cache e7b40e146c5353be5a6ccbf3e9665279dd726eabb1c205a4ac48a1978889eefa
--> e7b40e146c5
STEP 3: COPY requirements.txt ./
--> 71c60313f5a
STEP 4: RUN pip3 install  --user -r requirements.txt
Collecting scikit-learn[alldeps]==0.21.1
  Downloading scikit-learn-0.21.1.tar.gz (12.2 MB)
     |████████████████████████████████| 12.2 MB 1.4 MB/s 
Collecting numpy>=1.11.0
  Downloading numpy-1.19.4-cp39-cp39-manylinux2010_x86_64.whl (14.5 MB)
     |████████████████████████████████| 14.5 MB 48.3 MB/s 
Collecting scipy>=0.17.0
  Downloading scipy-1.5.3.tar.gz (25.2 MB)
     |████████████████████████████████| 25.2 MB 31 kB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmprph12anc                                                                     
         cwd: /tmp/pip-install-1gimcwm9/scipy                                                                                                                                                                                      
    Complete output (139 lines):                                                                                                                                                                                                   
    setup.py:460: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-modern-metadata-ylec3blf'), proceeding with generating Cython sources and expanding templates                                       
      warnings.warn("Unrecognized setuptools command ('{}'), proceeding with "                                                                                                                                                     
    Running from SciPy source directory.                                                                                                                                                                                           
    lapack_opt_info:                                                                                                                                                                                                               
    lapack_mkl_info:                                                                                                                                                                                                               
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']                                                                                                                                    
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    openblas_lapack_info:                                                                                                                                                                                                          
    customize UnixCCompiler                                                                                                                                                                                                        
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries openblas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']                                                                                                                                  
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    openblas_clapack_info:                                                                                                                                                                                                         
    customize UnixCCompiler                                                                                                                                                                                                        
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries openblas,lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']                                                                                                                           
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    flame_info:                                                                                                                                                                                                                    
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries flame not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']                                                                                                                                     
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    atlas_3_10_threads_info:                                                                                                                                                                                                       
    Setting PTATLAS=ATLAS                                                                                                                                                                                                          
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/local/lib                                                                                                                                                                           
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries tatlas,tatlas not found in /usr/local/lib                                                                                                                                                                          
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib                                                                                                                                                                                 
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries tatlas,tatlas not found in /usr/lib                                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries tatlas,tatlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                               
    <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>                                                                                                                                                                  
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    atlas_3_10_info:                                                                                                                                                                                                               
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/local/lib                                                                                                                                                                           
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries satlas,satlas not found in /usr/local/lib                                                                                                                                                                          
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib                                                                                                                                                                                 
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries satlas,satlas not found in /usr/lib                                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries satlas,satlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                               
    <class 'numpy.distutils.system_info.atlas_3_10_info'>                                                                                                                                                                          
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    atlas_threads_info:                                                                                                                                                                                                            
    Setting PTATLAS=ATLAS                                                                                                                                                                                                          
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/local/lib                                                                                                                                                                           
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib                                                                                                                                                                                 
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib                                                                                                                                                                      
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                     
    <class 'numpy.distutils.system_info.atlas_threads_info'>                                                                                                                                                                       
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    atlas_info:                                                                                                                                                                                                                    
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/local/lib                                                                                                                                                                           
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries f77blas,cblas,atlas not found in /usr/local/lib                                                                                                                                                                    
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib                                                                                                                                                                                 
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu
    <class 'numpy.distutils.system_info.atlas_info'>
      NOT AVAILABLE
    
    accelerate_info:
      NOT AVAILABLE
    
    lapack_info:
    customize UnixCCompiler
      libraries lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE
    
    /tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      if getattr(self, '_calc_info_{}'.format(lapack))():
    lapack_src_info:
      NOT AVAILABLE
    
    /tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      if getattr(self, '_calc_info_{}'.format(lapack))():
      NOT AVAILABLE
    
    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
        main()
      File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 161, in prepare_metadata_for_build_wheel
        self.run_setup()
      File "/tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 253, in run_setup
        super(_BuildMetaLegacyBackend,
      File "/tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 145, in run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 583, in <module>
        setup_package()
      File "setup.py", line 579, in setup_package
        setup(**metadata)
      File "/tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/numpy/distutils/core.py", line 137, in setup
        config = configuration()
      File "setup.py", line 477, in configuration
        raise NotFoundError(msg)
    numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmprph12anc Check the logs for full command output.
Error: error building at STEP "RUN pip3 install  --user -r requirements.txt": error while running runtime: exit status 1

Noticing the message "No lapack/blas resources found." and updating the Dockerfile to include those system libraries:

FROM python:3.9.0-buster
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN apt-get update \
  && apt-get install -y --no-install-recommends gfortran libopenblas-dev liblapack-dev \
  && pip3 install  --user -r requirements.txt

still results in a failure (shortened for brevity):

...
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-t_f5ekvw/scikit-learn/setup.py", line 290, in <module>
        setup_package()
      File "/tmp/pip-install-t_f5ekvw/scikit-learn/setup.py", line 286, in setup_package
        setup(**metadata)
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/core.py", line 135, in setup
        config = configuration()
      File "/tmp/pip-install-t_f5ekvw/scikit-learn/setup.py", line 174, in configuration
        config.add_subpackage('sklearn')
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 1018, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 984, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 926, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/setup.py", line 66, in configuration
        config.add_subpackage('utils')
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 1018, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 984, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 926, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/utils/setup.py", line 8, in configuration
        from Cython import Tempita
    ModuleNotFoundError: No module named 'Cython'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-t_f5ekvw/scikit-learn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-t_f5ekvw/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-fiqixj7v/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /root/.local/include/python3.9/scikit-learn Check the logs for full command output.

and now installing cython explicitly by adding a pip install --user cython before installing requirements.txt:

...
Building wheels for collected packages: scikit-learn, numpy, scipy
  Building wheel for scikit-learn (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-1fi_vbt0/scikit-learn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-1fi_vbt0/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-hyn83c99
       cwd: /tmp/pip-install-1fi_vbt0/scikit-learn/
  Complete output (15 lines):
  Partial import of sklearn during the build process.
  Traceback (most recent call last):
    File "/tmp/pip-install-1fi_vbt0/scikit-learn/setup.py", line 187, in get_numpy_status
      import numpy
  ModuleNotFoundError: No module named 'numpy'
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-1fi_vbt0/scikit-learn/setup.py", line 290, in <module>
      setup_package()
    File "/tmp/pip-install-1fi_vbt0/scikit-learn/setup.py", line 278, in setup_package
      raise ImportError("Numerical Python (NumPy) is not "
  ImportError: Numerical Python (NumPy) is not installed.
  scikit-learn requires NumPy >= 1.11.0.
  Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html
  
  ----------------------------------------
  ERROR: Failed building wheel for scikit-learn

I'm building with podman 2.1.1.

When using FROM debian:buster, the same build works without any issues. I'm fine with using debian:buster but can either the image or the documentation be updated to explain why this usage of pip in this image does not work, and/or document how to make it work?

@wglambert wglambert added the question Usability question, not directly related to an error with the image label Nov 4, 2020
@wglambert
Copy link

See #540

I also came across this neat support matrix for what packages support Python 3.9
https://pyreadiness.org/3.9/

debian:buster uses Python 3.7.3
https://packages.debian.org/buster/python3

Using python 3.8 works fine

$ docker build -t python:test - << EOF
FROM python:3.8-slim
RUN apt-get update && apt-get install -y --no-install-recommends gfortran libopenblas-dev liblapack-dev && rm -rf /var/lib/apt/lists/*
RUN pip3 install cython
RUN pip3 install numpy==1.17.2
RUN apt update && apt install -y --no-install-recommends g++ && pip3 install scikit-learn[alldeps]==0.21.1 && apt remove -y g++ && rm -rf /var/lib/apt/lists/*
EOF
Sending build context to Docker daemon  2.048kB
Step 1/5 : FROM python:3.8-slim
 ---> 0f59d947500d
Step 2/5 : RUN apt-get update && apt-get install -y --no-install-recommends gfortran libopenblas-dev liblapack-dev && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 006a4eb6efa7
Step 3/5 : RUN pip3 install cython
 ---> Using cache
 ---> 46b0efff21a1
Step 4/5 : RUN pip3 install numpy==1.17.2
 ---> Using cache
 ---> 0b930b8f4d7f
Step 5/5 : RUN apt update && apt install -y --no-install-recommends g++ && pip3 install scikit-learn[alldeps]==0.21.1 && apt remove -y g++ && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 39ee4a2fc474
Successfully built 39ee4a2fc474
Successfully tagged python:test

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
python              test                39ee4a2fc474        About a minute ago   637MB
python              3.8-slim            0f59d947500d        21 hours ago         113MB

@rocketraman
Copy link
Author

Thanks for the explanation about incompatibilities with Python 3.9. I'm not normally a Python developer so I didn't realize 3.9 wasn't compatible with 3.7.x (it's too bad that they don't follow semver guidelines).

Closing.

@varsha1288
Copy link

This image makes it very difficult (impossible?) to install numpy, scipy, and scikit-learn via pip, which as per #43 should be the recommended method. As I noted in a comment to that issue which was marked as spam (I admit the comment may have been unhelpful, but it certainly wasn't spam), this does not seem to work well.

Example Dockerfile:

FROM python:3.9.0-buster
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip3 install  --user -r requirements.txt

and requirements.txt as recommended by the scikit docs (https://scikit-learn.org/stable/install.html#install-official-release):

scikit-learn[alldeps]==0.21.1
numpy==1.17.2

results in output like this:

STEP 1: FROM python:3.9.0-buster
STEP 2: WORKDIR /usr/src/app
--> Using cache e7b40e146c5353be5a6ccbf3e9665279dd726eabb1c205a4ac48a1978889eefa
--> e7b40e146c5
STEP 3: COPY requirements.txt ./
--> 71c60313f5a
STEP 4: RUN pip3 install  --user -r requirements.txt
Collecting scikit-learn[alldeps]==0.21.1
  Downloading scikit-learn-0.21.1.tar.gz (12.2 MB)
     |████████████████████████████████| 12.2 MB 1.4 MB/s 
Collecting numpy>=1.11.0
  Downloading numpy-1.19.4-cp39-cp39-manylinux2010_x86_64.whl (14.5 MB)
     |████████████████████████████████| 14.5 MB 48.3 MB/s 
Collecting scipy>=0.17.0
  Downloading scipy-1.5.3.tar.gz (25.2 MB)
     |████████████████████████████████| 25.2 MB 31 kB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmprph12anc                                                                     
         cwd: /tmp/pip-install-1gimcwm9/scipy                                                                                                                                                                                      
    Complete output (139 lines):                                                                                                                                                                                                   
    setup.py:460: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-modern-metadata-ylec3blf'), proceeding with generating Cython sources and expanding templates                                       
      warnings.warn("Unrecognized setuptools command ('{}'), proceeding with "                                                                                                                                                     
    Running from SciPy source directory.                                                                                                                                                                                           
    lapack_opt_info:                                                                                                                                                                                                               
    lapack_mkl_info:                                                                                                                                                                                                               
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']                                                                                                                                    
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    openblas_lapack_info:                                                                                                                                                                                                          
    customize UnixCCompiler                                                                                                                                                                                                        
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries openblas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']                                                                                                                                  
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    openblas_clapack_info:                                                                                                                                                                                                         
    customize UnixCCompiler                                                                                                                                                                                                        
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries openblas,lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']                                                                                                                           
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    flame_info:                                                                                                                                                                                                                    
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries flame not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']                                                                                                                                     
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    atlas_3_10_threads_info:                                                                                                                                                                                                       
    Setting PTATLAS=ATLAS                                                                                                                                                                                                          
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/local/lib                                                                                                                                                                           
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries tatlas,tatlas not found in /usr/local/lib                                                                                                                                                                          
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib                                                                                                                                                                                 
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries tatlas,tatlas not found in /usr/lib                                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries tatlas,tatlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                               
    <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>                                                                                                                                                                  
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    atlas_3_10_info:                                                                                                                                                                                                               
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/local/lib                                                                                                                                                                           
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries satlas,satlas not found in /usr/local/lib                                                                                                                                                                          
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib                                                                                                                                                                                 
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries satlas,satlas not found in /usr/lib                                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries satlas,satlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                               
    <class 'numpy.distutils.system_info.atlas_3_10_info'>                                                                                                                                                                          
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    atlas_threads_info:                                                                                                                                                                                                            
    Setting PTATLAS=ATLAS                                                                                                                                                                                                          
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/local/lib                                                                                                                                                                           
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib                                                                                                                                                                                 
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib                                                                                                                                                                      
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                                
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu                                                                                                                                                     
    <class 'numpy.distutils.system_info.atlas_threads_info'>                                                                                                                                                                       
      NOT AVAILABLE                                                                                                                                                                                                                
                                                                                                                                                                                                                                   
    atlas_info:                                                                                                                                                                                                                    
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/local/lib                                                                                                                                                                           
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries f77blas,cblas,atlas not found in /usr/local/lib                                                                                                                                                                    
    customize UnixCCompiler                                                                                                                                                                                                        
      libraries lapack_atlas not found in /usr/lib                                                                                                                                                                                 
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu
    <class 'numpy.distutils.system_info.atlas_info'>
      NOT AVAILABLE
    
    accelerate_info:
      NOT AVAILABLE
    
    lapack_info:
    customize UnixCCompiler
      libraries lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE
    
    /tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      if getattr(self, '_calc_info_{}'.format(lapack))():
    lapack_src_info:
      NOT AVAILABLE
    
    /tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      if getattr(self, '_calc_info_{}'.format(lapack))():
      NOT AVAILABLE
    
    Traceback (most recent call last):
      File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
        main()
      File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 161, in prepare_metadata_for_build_wheel
        self.run_setup()
      File "/tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 253, in run_setup
        super(_BuildMetaLegacyBackend,
      File "/tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 145, in run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 583, in <module>
        setup_package()
      File "setup.py", line 579, in setup_package
        setup(**metadata)
      File "/tmp/pip-build-env-nff2v6pv/overlay/lib/python3.9/site-packages/numpy/distutils/core.py", line 137, in setup
        config = configuration()
      File "setup.py", line 477, in configuration
        raise NotFoundError(msg)
    numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmprph12anc Check the logs for full command output.
Error: error building at STEP "RUN pip3 install  --user -r requirements.txt": error while running runtime: exit status 1

Noticing the message "No lapack/blas resources found." and updating the Dockerfile to include those system libraries:

FROM python:3.9.0-buster
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN apt-get update \
  && apt-get install -y --no-install-recommends gfortran libopenblas-dev liblapack-dev \
  && pip3 install  --user -r requirements.txt

still results in a failure (shortened for brevity):

...
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-t_f5ekvw/scikit-learn/setup.py", line 290, in <module>
        setup_package()
      File "/tmp/pip-install-t_f5ekvw/scikit-learn/setup.py", line 286, in setup_package
        setup(**metadata)
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/core.py", line 135, in setup
        config = configuration()
      File "/tmp/pip-install-t_f5ekvw/scikit-learn/setup.py", line 174, in configuration
        config.add_subpackage('sklearn')
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 1018, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 984, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 926, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/setup.py", line 66, in configuration
        config.add_subpackage('utils')
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 1018, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 984, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/root/.local/lib/python3.9/site-packages/numpy/distutils/misc_util.py", line 926, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/utils/setup.py", line 8, in configuration
        from Cython import Tempita
    ModuleNotFoundError: No module named 'Cython'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-t_f5ekvw/scikit-learn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-t_f5ekvw/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-fiqixj7v/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /root/.local/include/python3.9/scikit-learn Check the logs for full command output.

and now installing cython explicitly by adding a pip install --user cython before installing requirements.txt:

...
Building wheels for collected packages: scikit-learn, numpy, scipy
  Building wheel for scikit-learn (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-1fi_vbt0/scikit-learn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-1fi_vbt0/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-hyn83c99
       cwd: /tmp/pip-install-1fi_vbt0/scikit-learn/
  Complete output (15 lines):
  Partial import of sklearn during the build process.
  Traceback (most recent call last):
    File "/tmp/pip-install-1fi_vbt0/scikit-learn/setup.py", line 187, in get_numpy_status
      import numpy
  ModuleNotFoundError: No module named 'numpy'
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-1fi_vbt0/scikit-learn/setup.py", line 290, in <module>
      setup_package()
    File "/tmp/pip-install-1fi_vbt0/scikit-learn/setup.py", line 278, in setup_package
      raise ImportError("Numerical Python (NumPy) is not "
  ImportError: Numerical Python (NumPy) is not installed.
  scikit-learn requires NumPy >= 1.11.0.
  Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html
  
  ----------------------------------------
  ERROR: Failed building wheel for scikit-learn

I'm building with podman 2.1.1.

When using FROM debian:buster, the same build works without any issues. I'm fine with using debian:buster but can either the image or the documentation be updated to explain why this usage of pip in this image does not work, and/or document how to make it work?

python3 -m pip install --upgrade pip
pip3 install scipy

@santokalayil
Copy link

RUN python3.10 -m pip install --upgrade pip
RUN python3.10 -m pip install scipy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Usability question, not directly related to an error with the image
Projects
None yet
Development

No branches or pull requests

4 participants