MacOS: Numpy RuntimeError: Polyfit may be poorly conditioned

[kylebarron]

Hello!

I'm new to wheels and not entirely sure if this is cibuildwheel's fault. I'm using Github Actions to build wheels for my project, and while wheels originally built successfully for Mac, Windows, and Linux, they're now failing for Mac.

The project is written in Cython and also depends on Numpy; therefore both need to exist at the time wheels are built. I have python -m pip install numpy Cython in the CIBW_BEFORE_BUILD environment variable. Full build configuration here.

The main error of interest in the build log is:

RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. If you compiled yourself, see site.cfg.example for information. Otherwise report this to the vendor that provided NumPy.

Which leads me to wonder if Numpy is being built incorrectly.

I didn't see any existing issues here that looked to be related. Is this something you've seen before?

Full build log (not sure if the full logs are visible)

Longer traceback:

  ...
  Building wheel for numpy (PEP 517): started
  Building wheel for numpy (PEP 517): still running...
  Building wheel for numpy (PEP 517): finished with status 'done'
  Created wheel for numpy: filename=numpy-1.19.0-pp36-pypy36_pp73-macosx_10_9_x86_64.whl size=4070475 sha256=84fd08b871eec5b4bdbc25472592f768298c18eb02a581071e51d28b935c4f5a
  Stored in directory: /Users/runner/Library/Caches/pip/wheels/ec/3e/c8/4e514ba0d2a793b3a05a4a8e67abd80753ebbd32f335ed38cb
Successfully built numpy
Installing collected packages: numpy, Cython
Successfully installed Cython-0.29.21 numpy-1.19.0
+ pip wheel . -w /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibuildwheels_5z_9g_/built_wheel --no-deps
Processing /Users/runner/work/quantized-mesh-encoder/quantized-mesh-encoder
    ERROR: Command errored out with exit status 1:
     command: /tmp/cibw_bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-req-build-0huwpwwt/setup.py'"'"'; __file__='"'"'/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-req-build-0huwpwwt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-pip-egg-info-bo0_wgya
         cwd: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-req-build-0huwpwwt/
    Complete output (10 lines):
    init_dgelsd failed init
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-req-build-0huwpwwt/setup.py", line 4, in <module>
        import numpy as np
      File "/tmp/pypy3.6-v7.3.1-osx64/site-packages/numpy/__init__.py", line 286, in <module>
        raise RuntimeError(msg)
    RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. If you compiled yourself, see site.cfg.example for information. Otherwise report this to the vendor that provided NumPy.
    RankWarning: Polyfit may be poorly conditioned
    
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Traceback (most recent call last):
  File "/Users/runner/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/runner/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/runner/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/cibuildwheel/__main__.py", line 313, in <module>
    main()
  File "/Users/runner/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/cibuildwheel/__main__.py", line 232, in main
    cibuildwheel.macos.build(build_options)
  File "/Users/runner/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/cibuildwheel/macos.py", line 197, in build
    call(['pip', 'wheel', options.package_dir, '-w', built_wheel_dir, '--no-deps'] + get_build_verbosity_extra_flags(options.build_verbosity), env=env)
  File "/Users/runner/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/cibuildwheel/macos.py", line 29, in call
    return subprocess.check_call(args, env=env, cwd=cwd, shell=shell)
  File "/Users/runner/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['pip', 'wheel', '.', '-w', '/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibuildwheels_5z_9g_/built_wheel', '--no-deps']' returned non-zero exit status 1.
##[error]Process completed with exit code 1.

[Czaki]

I'm new to wheels and not entirely sure if this is cibuildwheel's fault. I'm using Github Actions to build wheels for my project, and while wheels originally built successfully for Mac, Windows, and Linux, they're now failing for Mac.

In previous pipeline did you try to build wheel for pypy? Because wheels for cpython are build correctly.
It looks like there is problem with pypy and numpy 1.19 maybe You can check if limit numpy to 1.18.5 will fix it.

[joerick]

I'd argue you don't really want to be building a numpy as part of your build - a wheel should be sufficient. Looks like numpy don't publish many pypy wheels yet. If pypy support isn't important to you, you can disable it using CIBW_SKIP=pp* or similar.

[YannickJadoul]

I think you might need to report this to the PyPy project: https://foss.heptapod.net/pypy/pypy

[kylebarron]

Wow, thanks all for the quick responses! I didn't realize it might be PyPy-related; PyPy support isn't important to me so I'll try turning that off.

In previous pipeline did you try to build wheel for pypy?

Yes, in 0.1.2 I successfully built for PyPy on MacOS, see quantized_mesh_encoder-0.1.2-pp36-pypy36_pp73-macosx_10_9_x86_64.whl. I didn't change the build config at all between 0.1.2 and 0.2.0, hence my confusion.

I think you might need to report this to the PyPy project

I made an issue here.

[kylebarron]

After removing PyPy from the build list, MacOS wheels build fine. (Unsurprisingly building the wheels is considerably faster because I'm no longer building Numpy).

I'll close this because it sounds like a PyPy issue.

[mattip]

For future reference: by default NumPy may be using Accelerate to provide BLAS fun tions. This is broken and Apple won’t fix it. The solution is to use OpenBLAS instead when building NumPy. It would be better if NumPy would just supply a binary wheel, this may happen soon.

[mattip]

The actual error mentions this:

RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. If you compiled yourself, see site.cfg.example for information. Otherwise report this to the vendor that provided NumPy.

[kylebarron]

Thanks. I saw that line but I didn't know what it meant.

[Czaki]

Two things to be done:

brew install openblas
OPENBLAS="$(brew --prefix openblas)" cibuildwheel ...

numpy/numpy#17784 (comment)

[dofuuz]

related issue:
numpy/numpy#15947