MAINT: remove support for Python 3.7 (hence NumPy 1.16)

[tupui]

We should be dropping Python 3.7 with the next release (1.8), hence NumPy 1.16 will not be supported and we will move to NumPy 1.17.3.

This PR touches the setup.py, pyproject.toml, CI, some other smaller things and some documentation.

Related to recent discussions and linked to #14649. Also in accordance with NEP29 (NumPy already dropped support for 3.7 on main). And see here for official Python support. There will be just 1 year of support of 3.7 for the next release if we don't migrate now.

I think I touched everything with these 2 PRs. Just not sure about the Dockerfile.

[tupui]

Note:

In the process, I tried to update to ubuntu:20.04 instead of 18.04 in a few workflows, but it causes a warning FAILED scipy/stats/tests/test_continuous_basic.py::test_cont_basic[500-200-ncf-arg74]

scipy/integrate/quadpack.py:400: in quad
    warnings.warn(msg, IntegrationWarning, stacklevel=2)
E   scipy.integrate.quadpack.IntegrationWarning: The occurrence of roundoff error is detected, which prevents 
E     the requested tolerance from being achieved.  The error may be 
E     underestimated.

and also a different value in a doctest

File "build/testenv/lib/python3.8/site-packages/scipy/optimize/optimize.py", line 994, in check_grad
Failed example:
    check_grad(func, grad, [1.5, -1.5])
Expected:
    2.9802322387695312e-08
Got:
    4.2146848510894035e-08

So I reverted these and now all is passing.

[mdhaber]

I always get the ncf quadpack error on my computer.

[tupui]

I always get the ncf quadpack error on my computer.

Interesting, maybe something to look at after that. Or if you know a simple fix we can also do this now. I could not reproduce locally.

[mdhaber]

I haven't thought about a fix. Previously I had assumed it must be a very specific thing, but it sounds it's at least a little more widespread. Note that this is my Windows machine, not Ubuntu.

[rgommers]

Platform-specific failures are always good to fix; if you see them locally while CI is green, it usually means others see them too.

[tupui]

Platform-specific failures are always good to fix; if you see them locally while CI is green, it usually means others see them too.

Ok, then I can re-update the CI to use 20.04 to make the CI fail. It will just not work for wheel_optimized_gcc48 as GCC version is not correct then. And I guess we don't want to change this, or do we?

[rgommers]

This LGTM. We should add 3.10, but that's best done in a separate PR.

[rgommers]

Ok, then I can re-update the CI to use 20.04 to make the CI fail.

This is green now, so I'd leave that for a separate follow-up PR.

It will just not work for wheel_optimized_gcc48 as GCC version is not correct then. And I guess we don't want to change this, or do we?

We should bump the lowest GCC version, but last time we tried there was an issue on the scipy-wheels repo, so best not to do it in this PR.

[rgommers]

Also once this is merged, scipy-wheels should be updated to match. Do you want to do that @tupui?

[tupui]

Ok great then! I will look at bumping GCC as soon as this merges. For the other bug, @mdhaber do you have time to take a look since you can reproduce?

[rgommers]

It's all green, so let's give it a go - merged.

[tupui]

Also once this is merged, scipy-wheels should be updated to match. Do you want to do that @tupui?

Sure, glad to learn how and help with that.

[mdhaber]

@tupui Before I can think about the ncf issue I need to be able to work with SciPy master again. This PR forced me to upgrade to Python 3.8 and now I get a DLL load error when I try to import any scipy subpackage; please gee gh-11062.