Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set minimum numpy version to avoid ABI mismatch error #126

Merged
merged 2 commits into from
Mar 1, 2024

Conversation

ngoldbaum
Copy link
Collaborator

@ngoldbaum ngoldbaum commented Mar 1, 2024

This fixes an error reported by @AdamGlustein:

RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xe . Check the section C-API incompatibility at the Troubleshooting ImportError section at https://numpy.org/devdocs/user/troubleshooting-importerror.html#c-api-incompatibility for indications on how to solve this problem .
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data01/home/user/hfalgo_ext/python/Python-3.8.12/lib/python3.8/site-packages/csp/__init__.py", line 3, in <module>
    from csp.baselib import *
  File "/data01/home/user/hfalgo_ext/python/Python-3.8.12/lib/python3.8/site-packages/csp/baselib.py", line 18, in <module>
    from csp.impl.wiring import DelayedEdge, Edge, OutputsContainer, graph, input_adapter_def, node
  File "/data01/home/user/hfalgo_ext/python/Python-3.8.12/lib/python3.8/site-packages/csp/impl/wiring/__init__.py", line 3, in <module>
    from csp.impl.wiring.delayed_edge import DelayedEdge
  File "/data01/home/user/hfalgo_ext/python/Python-3.8.12/lib/python3.8/site-packages/csp/impl/wiring/delayed_edge.py", line 8, in <module>
    from .graph import graph
  File "/data01/home/user/hfalgo_ext/python/Python-3.8.12/lib/python3.8/site-packages/csp/impl/wiring/graph.py", line 15, in <module>
    from csp.impl.wiring.cache_support.dataset_partition_cached_data import DataSetCachedData, DatasetPartitionCachedData
  File "/data01/home/user/hfalgo_ext/python/Python-3.8.12/lib/python3.8/site-packages/csp/impl/wiring/cache_support/dataset_partition_cached_data.py", line 12, in <module>
    from csp.adapters.output_adapters.parquet import resolve_array_shape_column_name
  File "/data01/home/user/hfalgo_ext/python/Python-3.8.12/lib/python3.8/site-packages/csp/adapters/output_adapters/parquet.py", line 13, in <module>
    from csp.lib import _parquetadapterimpl
ImportError: numpy.core.multiarray failed to import

According to https://numpy.org/doc/stable/dev/depending_on_numpy.html#build-time-dependency, we can build against numpy 1.25 or newer and numpy 1.19 or newer should be able to import the build. However, numpy 1.25 isn't supported on python 3.8, so I've decided to set a lower bound on an older version of numpy on the build host, which should also avoid ABI mismatch errors.

timkpaine
timkpaine previously approved these changes Mar 1, 2024
@ngoldbaum
Copy link
Collaborator Author

Trying one more time using oldest-supported-numpy, which should automagically pick the oldest numpy supported for each python version.

Once we drop support for python3.8, we can switch to using numpy 1.26 (or numpy 2 if we're ready for numpy 2 at that point).

Signed-off-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>
@timkpaine
Copy link
Member

do we want to have explicit runtime tests?
e.g.

Signed-off-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>
@ngoldbaum ngoldbaum merged commit 493240a into Point72:main Mar 1, 2024
15 checks passed
@timkpaine timkpaine added type: bug Concrete, reproducible bugs part: dependencies PRs that update a dependency file labels Jun 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
part: dependencies PRs that update a dependency file type: bug Concrete, reproducible bugs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants