Bump scipy version to <1.11.0 #90

je-cook · 2022-10-17T06:05:37Z

This PR bumps scipy version to allow installation of numpy >=1.23 alongside numba and numba-scipy

As discussed in #88 the scipy upgrade was waiting for anaconda to have a later release in its default repo which it now does (https://anaconda.org/anaconda/scipy).

I have pinned to less than the next minor version of scipy as it is unlikely behaviour will change on patch version changes. If you would like this stricter let me know.

Closes #88

EDIT

Scipy have bumped the deprecation to after 1.11.0 (see scipy/scipy#15596) and 1.10.0 is now on the anaconda repo.

david-zwicker · 2022-11-28T15:55:59Z

It would be great to merge this PR and have a new release, so we can use numba-scipy with newer scipy versions. Is there any timeline on merging this PR?

stuartarchibald · 2022-11-28T16:02:06Z

@david-zwicker I'll raise this at the public Numba meeting tomorrow. Thanks for the PR @je-cook.

je-cook · 2023-01-12T11:48:43Z

Hi, is there any feeling on when this could be merged?
Its limiting our access to some f2py improvements that were added in numpy 1.23 so I'm hoping soonish :)

Bump scipy version to allow installation of numpy >=1.23 alongside numba and numba-scipy

je-cook · 2023-04-24T06:40:47Z

gentle ping @stuartarchibald @esc is there any update on this or any chance it can be merged?

esc · 2023-04-24T08:36:14Z

gentle ping @stuartarchibald @esc is there any update on this or any chance it can be merged?

Thank you for asking about this. Perhaps @brandonwillard has some input on this too?

brandonwillard · 2023-04-26T00:57:06Z

gentle ping @stuartarchibald @esc is there any update on this or any chance it can be merged?

Thank you for asking about this. Perhaps @brandonwillard has some input on this too?

I'm all for this update, as long as the tests pass, of course (and it looks like CI is awaiting approval).

je-cook · 2023-05-03T09:28:37Z

Thanks @brandonwillard, @esc could you approve the CI run so we can move forward with this?

esc · 2023-05-04T06:42:19Z

/azp run

azure-pipelines · 2023-05-04T06:42:28Z

Azure Pipelines successfully started running 1 pipeline(s).

esc · 2023-05-04T07:57:45Z

It looks like CI is bust somehow:

==================================== ERRORS ====================================
_______________________ ERROR collecting test_sparse.py ________________________
/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/numba_scipy/tests/test_sparse.py:4: in <module>
    import scipy.sparse
/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/scipy/sparse/__init__.py:229: in <module>
    from .base import *
/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/scipy/sparse/base.py:8: in <module>
    from .sputils import (isdense, isscalarlike, isintlike,
/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/scipy/sparse/sputils.py:[16](https://github.com/numba/numba-scipy/actions/runs/4729770315/jobs/8706873616?pr=90#step:5:17): in <module>
    supported_dtypes = [np.typeDict[x] for x in supported_dtypes]
/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/scipy/sparse/sputils.py:16: in <listcomp>
    supported_dtypes = [np.typeDict[x] for x in supported_dtypes]
/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/numpy/__init__.py:320: in __getattr__
    raise AttributeError("module {!r} has no attribute "
E   AttributeError: module 'numpy' has no attribute 'typeDict'
_______________________ ERROR collecting test_special.py _______________________
ImportError while importing test module '/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/numba_scipy/tests/test_special.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/miniconda/envs/test_env/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/numba_scipy/tests/test_special.py:11: in <module>
    import scipy.special as sc
/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/scipy/special/__init__.py:643: in <module>
    from .basic import *
/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/scipy/special/basic.py:[19](https://github.com/numba/numba-scipy/actions/runs/4729770315/jobs/8706873616?pr=90#step:5:20): in <module>
    from . import orthogonal
/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/scipy/special/orthogonal.py:81: in <module>
    from numpy import (exp, inf, pi, sqrt, floor, sin, cos, around, int,
E   ImportError: cannot import name 'int' from 'numpy' (/usr/local/miniconda/envs/test_env/lib/python3.8/site-packages/numpy/__init__.py)
=========================== short test summary info ============================
ERROR test_sparse.py - AttributeError: module 'numpy' has no attribute 'typeD...
ERROR test_special.py
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
============================== 2 errors in 10.[31](https://github.com/numba/numba-scipy/actions/runs/4729770315/jobs/8706873616?pr=90#step:5:32)s ==============================
Error: Process completed with exit code 2.

And on azure, we are seeing:

je-cook · 2023-05-04T08:07:35Z

Hmm odd, from what I can tell those numpy imports were removed in numpy>v1.21.

It also looks weird because I had missed the setup.py version change so I'd have thought it would actually have been the old scipy that was tested...

i've pushed the scipy change, but not quite sure how to do interdependent versioning if we need numpy<=1.21 & scipy<=1.7.3 or numpy>1.21 & scipy>1.7.3

esc · 2023-05-04T09:30:22Z

Does anyone know why I need to approve the workflow everytime?

esc · 2023-05-04T09:30:47Z

Also, does anyone know what is up with the Azure images?

je-cook · 2023-05-04T10:46:46Z

Does anyone know why I need to approve the workflow everytime?

I think its because this is my first PR to the project, on another project I've got it set to 'require approval for first time contributors' and it seems to be the case that you have to approve for all updates (for us at least)

esc · 2023-05-04T12:12:51Z

Does anyone know why I need to approve the workflow everytime?

I think its because this is my first PR to the project, on another project I've got it set to 'require approval for first time contributors' and it seems to be the case that you have to approve for all updates (for us at least)

yeah, that makes sense, I'll keep approving them until this PR is merged then.

esc · 2023-05-04T12:13:28Z

Also, does anyone know what is up with the Azure images?

could it be, that the image is outdated and now longer available?

As title

esc · 2023-05-04T12:19:06Z

Also, does anyone know what is up with the Azure images?

could it be, that the image is outdated and now longer available?

Tryin' to bump the Azure Linux Image here: #95

je-cook · 2023-05-04T12:20:39Z

I forgot to quote 3.10 so it will probably fail without that

esc · 2023-05-09T10:24:52Z

Hi @esc I've just updated everything and I think I need approval to change the macOS VM image for azure as macOS-10.15 doesnt exist like you fixed in #95.

Hopefully this time I've caught everything 🙂

Splendid, thank you!

esc · 2023-05-09T10:33:57Z

/azp run

azure-pipelines · 2023-05-09T10:34:07Z

Azure Pipelines successfully started running 1 pipeline(s).

esc · 2023-05-09T10:34:50Z

@brandonwillard can I get a second set of eys 👀 on this? it LGTM from my end.

esc · 2023-05-09T11:20:09Z

@je-cook good news, this one tested as green.

je-cook · 2023-05-09T12:29:37Z

Looks like github was not as clever as I hoped and if you leave out a variable in an include outside of the matrix it defaults to an empty string and doesnt cycle through the matrix for that value!

esc · 2023-05-10T06:25:28Z

OK, looks like the test matrix is all green, However, I see that there are tests for Python 3.6 and 3.7 -- now I am confused as to why we want these? The latest Numba release only supports 3.8, 3.9, 3.10 and 3.11..

See also: the version support table for Numba.

https://numba.readthedocs.io/en/stable/user/installing.html#version-support-information

je-cook · 2023-05-10T06:55:10Z

Happy to drop them if we're keeping in lockstep with numba itself. I just didn't want to deprecate anything without wider consent

je-cook · 2023-05-10T07:07:23Z

I'm not sure whether the 32bit version will work with the newer stuff but we'll see!

esc · 2023-05-10T07:40:27Z

I'm not sure whether the 32bit version will work with the newer stuff but we'll see!

I don't think we support 32-bit anymore. But I am not certain as of when that's the case. @stuartarchibald do you know if we have a hardware-support table too?

esc · 2023-05-10T09:36:28Z

@je-cook what's up with the Numpy 1.24 on conda issue?

je-cook · 2023-05-10T10:00:41Z

I think its because numba 0.57 (which added numpy 1.24 support) is not on the conda repo yet. Thats my best guess looking around the azure logs for a few minutes, https://dev.azure.com/numba/numba-scipy/_build/results?buildId=15042&view=results

esc · 2023-05-10T11:01:49Z

I think its because numba 0.57 (which added numpy 1.24 support) is not on the conda repo yet. Thats my best guess looking around the azure logs for a few minutes, https://dev.azure.com/numba/numba-scipy/_build/results?buildId=15042&view=results

Yes, looking at:

https://anaconda.org/conda-forge/numba/files

I concur with this assessment. I would like to find out when 0.57 is likely to land in CF and then decide on how to proceed.

esc · 2023-05-10T11:06:16Z

I think its because numba 0.57 (which added numpy 1.24 support) is not on the conda repo yet. Thats my best guess looking around the azure logs for a few minutes, https://dev.azure.com/numba/numba-scipy/_build/results?buildId=15042&view=results

Yes, looking at:

https://anaconda.org/conda-forge/numba/files

I concur with this assessment. I would like to find out when 0.57 is likely to land in CF and then decide on how to proceed.

Looking at:

conda-forge/llvmlite-feedstock#69

and

conda-forge/numba-feedstock#115

it may been some time until 0.57.0 lands in conda-forge -- thus I would suggest to merge this PR as it get's the job done and then add in 1.24 when 0.57.0 lands on CF. Are well good with that?

esc

Question:

esc · 2023-05-15T09:05:49Z

.github/workflows/test_and_deploy.yml

+        exclude:
+          - pyver: 3.11
+            numpy-version: "<1.22.0"
+        include:


Why do we need to include these? I thought these would be covered by the martrix?

There isn't a deprecation doc for scipy in numba-scipy as far as I'm aware.
I added them as extra tests to the matrix because only 3.8 is supported by the older scipy versions (which in turn requires an older numpy version) and previously that configuration was tested.

OK, I see, well, that's fine, I guess.

esc

.

esc · 2023-05-16T13:16:53Z

@je-cook thank you for the patch! @brandonwillard thank you for the review!

t-brito · 2023-08-29T11:41:38Z

Apologies for posting in a Merged PR, but I was wondering if there were any plans to package this change into a new patch version on PyPI (i.e. version 0.3.2?)

Directory structure for python package. numba-scipy is specified as a github dependency because the version on pypi isn't yet compatible with numpy >=1.23. This has been addressed in a pull request(numba/numba-scipy#90) and I expect to use the pypi version in future.

esc · 2023-09-13T08:57:38Z

Apologies for posting in a Merged PR, but I was wondering if there were any plans to package this change into a new patch version on PyPI (i.e. version 0.3.2?)

Feel free to open a new release request as an issue on github and I'll see what I can do about a release. 👍

je-cook changed the title ~~Bump scipy version to <1.10.0~~ Bump scipy version to <1.11.0 Feb 20, 2023

je-cook added 2 commits April 15, 2023 09:30

Bump scipy version to <1.10.0

a17eb9d

Bump scipy version to allow installation of numpy >=1.23 alongside numba and numba-scipy

Bump scipy version to <1.11.0

38718b4

je-cook force-pushed the je-cook/scipy-version-bump branch from d8bac0d to 38718b4 Compare April 18, 2023 07:30

esc previously approved these changes May 4, 2023

View reviewed changes

missed scipy version in setup.py

040a85e

je-cook dismissed esc’s stale review via 040a85e May 4, 2023 08:04

pin numpy version for certain scipy versions

48d9133

numpy versions >1.22 dropped support python 3.6 and 3.7

5e2e25b

esc mentioned this pull request May 4, 2023

test https://github.com/numba/numba-scipy/pull/90 with new azure images #95

Merged

esc and others added 2 commits May 4, 2023 14:18

update Azure Linux image

d73c2c7

As title

3.10 -> '3.10'

6d5b33f

github not as clever as hoped

0e4fdc4

exclude numpy<1.22 for python>=3.11

2225631

je-cook added 2 commits May 10, 2023 07:58

deprecate python 3.6 and 3.7

6445dba

move azure to more up to date python numpy and scipy

2b5e7be

numpy 1.24 incompatible on conda for some reason...

1d88419

brandonwillard approved these changes May 10, 2023

View reviewed changes

esc reviewed May 15, 2023

View reviewed changes

esc approved these changes May 16, 2023

View reviewed changes

esc merged commit 1e2f244 into numba:main May 16, 2023

je-cook deleted the je-cook/scipy-version-bump branch August 30, 2023 15:24

oliverfunk mentioned this pull request Sep 28, 2023

Package new release to include latest patch change (version 0.3.2) #97

Closed

Bump scipy version to <1.11.0 #90

Bump scipy version to <1.11.0 #90

Conversation

je-cook commented Oct 17, 2022 • edited Loading

david-zwicker commented Nov 28, 2022

stuartarchibald commented Nov 28, 2022

je-cook commented Jan 12, 2023

je-cook commented Apr 24, 2023

esc commented Apr 24, 2023

brandonwillard commented Apr 26, 2023

je-cook commented May 3, 2023

esc commented May 4, 2023

azure-pipelines bot commented May 4, 2023

esc commented May 4, 2023

je-cook commented May 4, 2023

esc commented May 4, 2023

esc commented May 4, 2023

je-cook commented May 4, 2023

esc commented May 4, 2023

esc commented May 4, 2023

esc commented May 4, 2023 • edited Loading

je-cook commented May 4, 2023

esc commented May 9, 2023

esc commented May 9, 2023

azure-pipelines bot commented May 9, 2023

esc commented May 9, 2023

esc commented May 9, 2023

je-cook commented May 9, 2023 • edited Loading

esc commented May 10, 2023

je-cook commented May 10, 2023

je-cook commented May 10, 2023

esc commented May 10, 2023

esc commented May 10, 2023

je-cook commented May 10, 2023 • edited Loading

esc commented May 10, 2023

esc commented May 10, 2023

esc left a comment

Choose a reason for hiding this comment

esc May 15, 2023

Choose a reason for hiding this comment

je-cook May 15, 2023 • edited Loading

Choose a reason for hiding this comment

esc May 16, 2023

Choose a reason for hiding this comment

esc left a comment • edited Loading

Choose a reason for hiding this comment

esc commented May 16, 2023

t-brito commented Aug 29, 2023

esc commented Sep 13, 2023

je-cook commented Oct 17, 2022 •

edited

Loading

esc commented May 4, 2023 •

edited

Loading

je-cook commented May 9, 2023 •

edited

Loading

je-cook commented May 10, 2023 •

edited

Loading

je-cook May 15, 2023 •

edited

Loading

esc left a comment •

edited

Loading