Install from source fails

[dlakaplan]

Following the instructions at:
https://github.com/numpy/numpy-user-dtypes/tree/main/quaddtype

Install from scratch:

conda create -n nptest2 python=3.13                                                                                            
conda activate nptest2	                                                                                                       
pip install "numpy @ git+https://github.com/numpy/numpy.git"                                                                   
python -c 'import numpy; print(numpy.__version__)'                                                                             
# returns                                                                                                                      
# 2.5.0.dev0+git20260109.f254f12                                                                                               
pip install pytest meson meson-python                                                                                          
git clone git@github.com:numpy/numpy-user-dtypes.git                                                                           
cd numpy-user-dtypes/quaddtype                                                                                                 
python -m pip install . -v --no-build-isolation                                                                                
# says:                                                                                                                        
# Successfully built numpy_quaddtype                                                                                           
# Installing collected packages: numpy_quaddtype               

But when I try to run the tests:

python -m pytest tests 
==================================================== test session starts ====================================================
platform darwin -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/kaplan/pythonpackages/numpy-user-dtypes/quaddtype
configfile: pyproject.toml
collected 0 items / 4 errors                                                                                                

========================================================== ERRORS ===========================================================
____________________________________________ ERROR collecting tests/test_dot.py _____________________________________________
ImportError while importing test module '/Users/kaplan/pythonpackages/numpy-user-dtypes/quaddtype/tests/test_dot.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../miniforge3/envs/nptest2/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_dot.py:3: in <module>
    from utils import create_quad_array, assert_quad_equal, assert_quad_array_equal, arrays_equal_with_nan
tests/utils.py:1: in <module>
    from numpy_quaddtype import QuadPrecision, QuadPrecDType
numpy_quaddtype/__init__.py:3: in <module>
    from ._quaddtype_main import (
E   ModuleNotFoundError: No module named 'numpy_quaddtype._quaddtype_main'
___________________________________________ ERROR collecting tests/test_finfo.py ____________________________________________
ImportError while importing test module '/Users/kaplan/pythonpackages/numpy-user-dtypes/quaddtype/tests/test_finfo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../miniforge3/envs/nptest2/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_finfo.py:3: in <module>
    from utils import assert_quad_equal as assert_quad_close
tests/utils.py:1: in <module>
    from numpy_quaddtype import QuadPrecision, QuadPrecDType
numpy_quaddtype/__init__.py:3: in <module>
    from ._quaddtype_main import (
E   ModuleNotFoundError: No module named 'numpy_quaddtype._quaddtype_main'
_______________________________________ ERROR collecting tests/test_multithreading.py _______________________________________
ImportError while importing test module '/Users/kaplan/pythonpackages/numpy-user-dtypes/quaddtype/tests/test_multithreading.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../miniforge3/envs/nptest2/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_multithreading.py:15: in <module>
    from numpy_quaddtype import *
numpy_quaddtype/__init__.py:3: in <module>
    from ._quaddtype_main import (
E   ModuleNotFoundError: No module named 'numpy_quaddtype._quaddtype_main'
_________________________________________ ERROR collecting tests/test_quaddtype.py __________________________________________
ImportError while importing test module '/Users/kaplan/pythonpackages/numpy-user-dtypes/quaddtype/tests/test_quaddtype.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../miniforge3/envs/nptest2/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_quaddtype.py:6: in <module>
    from mpmath import mp
E   ModuleNotFoundError: No module named 'mpmath'
================================================== short test summary info ==================================================
ERROR tests/test_dot.py
ERROR tests/test_finfo.py
ERROR tests/test_multithreading.py
ERROR tests/test_quaddtype.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
===================================================== 4 errors in 0.77s =====================================================

I installed mpmath, but that did not solve it:

python -m pytest tests
==================================================== test session starts ====================================================
platform darwin -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/kaplan/pythonpackages/numpy-user-dtypes/quaddtype
configfile: pyproject.toml
collected 0 items / 4 errors                                                                                                

========================================================== ERRORS ===========================================================
____________________________________________ ERROR collecting tests/test_dot.py _____________________________________________
ImportError while importing test module '/Users/kaplan/pythonpackages/numpy-user-dtypes/quaddtype/tests/test_dot.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../miniforge3/envs/nptest2/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_dot.py:3: in <module>
    from utils import create_quad_array, assert_quad_equal, assert_quad_array_equal, arrays_equal_with_nan
tests/utils.py:1: in <module>
    from numpy_quaddtype import QuadPrecision, QuadPrecDType
numpy_quaddtype/__init__.py:3: in <module>
    from ._quaddtype_main import (
E   ModuleNotFoundError: No module named 'numpy_quaddtype._quaddtype_main'
___________________________________________ ERROR collecting tests/test_finfo.py ____________________________________________
ImportError while importing test module '/Users/kaplan/pythonpackages/numpy-user-dtypes/quaddtype/tests/test_finfo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../miniforge3/envs/nptest2/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_finfo.py:3: in <module>
    from utils import assert_quad_equal as assert_quad_close
tests/utils.py:1: in <module>
    from numpy_quaddtype import QuadPrecision, QuadPrecDType
numpy_quaddtype/__init__.py:3: in <module>
    from ._quaddtype_main import (
E   ModuleNotFoundError: No module named 'numpy_quaddtype._quaddtype_main'
_______________________________________ ERROR collecting tests/test_multithreading.py _______________________________________
ImportError while importing test module '/Users/kaplan/pythonpackages/numpy-user-dtypes/quaddtype/tests/test_multithreading.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../miniforge3/envs/nptest2/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_multithreading.py:15: in <module>
    from numpy_quaddtype import *
numpy_quaddtype/__init__.py:3: in <module>
    from ._quaddtype_main import (
E   ModuleNotFoundError: No module named 'numpy_quaddtype._quaddtype_main'
_________________________________________ ERROR collecting tests/test_quaddtype.py __________________________________________
ImportError while importing test module '/Users/kaplan/pythonpackages/numpy-user-dtypes/quaddtype/tests/test_quaddtype.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../miniforge3/envs/nptest2/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_quaddtype.py:8: in <module>
    import numpy_quaddtype
numpy_quaddtype/__init__.py:3: in <module>
    from ._quaddtype_main import (
E   ModuleNotFoundError: No module named 'numpy_quaddtype._quaddtype_main'
================================================== short test summary info ==================================================
ERROR tests/test_dot.py
ERROR tests/test_finfo.py
ERROR tests/test_multithreading.py
ERROR tests/test_quaddtype.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
===================================================== 4 errors in 0.39s =====================================================

Apologies if I'm being dense. I had gotten this to work previously but was trying to update it for the latest improvements. However, I didn't keep very detailed notes so I may have missed a step.

[SwayamInSync]

Hi @dlakaplan you're running

python -m pytest tests

it means you are inside the quaddtype folder and usually python interpreter gets confused here on what to use, the local code or the installed package.

Please run the following

cd ..
pytest quaddtype/tests/ --disable-warnings

This issue ususally happens on macos

[ngoldbaum]

FWIW you can avoid this issue by moving the source code into an src folder. https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/

[SwayamInSync]

FWIW you can avoid this issue by moving the source code into an src folder. packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout

Good point, will drop a PR for this, needs to update meson

[ngoldbaum]

I'm having trouble building from source for a different reason:

± python -m pip install ".[test]" --no-build-isolation
Processing /Users/goldbaum/Documents/numpy-user-dtypes/quaddtype
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [48 lines of output]
      + meson setup /Users/goldbaum/Documents/numpy-user-dtypes/quaddtype /Users/goldbaum/Documents/numpy-user-dtypes/quaddtype/.mesonpy-n9sq561o -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/Users/goldbaum/Documents/numpy-user-dtypes/quaddtype/.mesonpy-n9sq561o/meson-python-native-file.ini
      The Meson build system
      Version: 1.10.0
      Source dir: /Users/goldbaum/Documents/numpy-user-dtypes/quaddtype
      Build dir: /Users/goldbaum/Documents/numpy-user-dtypes/quaddtype/.mesonpy-n9sq561o
      Build type: native build
      Project name: numpy_quaddtype
      Project version: undefined
      C compiler for the host machine: cc (clang 17.0.0 "Apple clang version 17.0.0 (clang-1700.6.3.2)")
      C linker for the host machine: cc ld64 1230.1
      C++ compiler for the host machine: c++ (clang 17.0.0 "Apple clang version 17.0.0 (clang-1700.6.3.2)")
      C++ linker for the host machine: c++ ld64 1230.1
      Host machine cpu family: aarch64
      Host machine cpu: aarch64
      Program python found: YES (/Users/goldbaum/.pyenv/versions/3.14.2/bin/python)
      Found pkg-config: YES (/opt/homebrew/bin/pkg-config) 2.5.1
      Run-time dependency python found: YES 3.14
      Found pkg-config: YES (/opt/homebrew/bin/pkg-config) 2.5.1
      Found CMake: /Users/goldbaum/.pyenv/versions/3.14.2/bin/cmake (4.2.0)
      Run-time dependency qblas found: NO (tried pkgconfig, framework and cmake)
      Looking for a fallback subproject for the dependency qblas

      Executing subproject qblas

      qblas| Project name: qblas
      qblas| Project version: 1.0.0
      qblas| Build targets in project: 0
      qblas| Subproject qblas finished.

      Dependency qblas from subproject subprojects/qblas found: YES 1.0.0
      Run-time dependency sleef found: YES 3.9.0
      Library sleefquad found: NO
      Library tlfloat found: NO
      WARNING: Subproject sleef's revision may be out of date; its wrap file has changed since it was first configured

      Executing subproject sleef

      sleef| Project name: sleef
      sleef| Project version: 3.8
      sleef| Program cmake found: YES (/Users/goldbaum/.pyenv/versions/3.14.2/bin/cmake)
      sleef| Program ninja found: YES (/Users/goldbaum/.pyenv/versions/3.14.2/bin/ninja)
      sleef| Build targets in project: 1
      sleef| Subproject sleef finished.


      ../meson.build:72:30: ERROR: Requested variable "tlfloat_dep" not found.

      A full log can be found at /Users/goldbaum/Documents/numpy-user-dtypes/quaddtype/.mesonpy-n9sq561o/meson-logs/meson-log.txt
      [end of output]

Let me know if you need additional debug info.

[SwayamInSync]

WARNING: Subproject sleef's revision may be out of date; its wrap file has changed since it was first configured

You seem to have old installation present in subprojects ideally that folder should be clean when installing from source. You can use the reinstall.sh or

rm -rf build/
rm -rf dist/
rm -rf subprojects/qblas
rm -rf subprojects/sleef
rm -rf subprojects/pythoncapi-compat

then build again

[SwayamInSync]

I should also write this down in README right

[ngoldbaum]

I was getting tripped up by git clean -fxd not deleting the sleef or qblas folder. Apparently, git will ignore the -f/--force flag if a gitignored subfolder is also a git repo, so you need to say git clean -ffxd. This is totally new git trivia to me and I kind of hate it.

Probably worth documenting this somewhere in some kind of development guide.

[dlakaplan]

Yes, I ran in to this same issue, so definitely good to document.

[dlakaplan]

Hi @dlakaplan you're running

python -m pytest tests

it means you are inside the quaddtype folder and usually python interpreter gets confused here on what to use, the local code or the installed package.

Please run the following

cd ..
pytest quaddtype/tests/ --disable-warnings

This issue ususally happens on macos

OK, thanks. Also maybe document if you can, since the way to run the tests had a typo which is why I got things wrong:

Current README:

cd ..
python -m pytest/quaddtype/tests/

but presumable there should be a space after pytest, not a slash. And I think it needs cd ../../ or similar?

[dlakaplan]

Also, what about mpmath not being installed automatically?

[ngoldbaum]

Also, what about mpmath not being installed automatically?

I think the instructions should suggest doing pip install “.[test]” to get the test dependencies:

https://github.com/numpy/numpy-user-dtypes/blob/ace8f05e35e32ecf30ea55a4c64463ea4cfd6d27/quaddtype/pyproject.toml#L36

[SwayamInSync]

I'll add these README updates in the attached PR