bump minimum versions, drop py38

[jhamman]

This updates our minimum versions based on our 24/18/12 month policy.

Details are shown below.

• User visible changes (including notable bug fixes) are documented in whats-new.rst

❯ ./ci/min_deps_check.py ./ci/requirements/min-all-deps.yml
...

Package           Required             Policy               Status
----------------- -------------------- -------------------- ------
python            3.9     (2020-10-07) 3.9     (2020-10-07) =
boto3             1.20    (2021-11-08) 1.20    (2021-11-08) =
bottleneck        1.3     (2021-01-20) 1.3     (2021-01-20) =
cartopy           0.20    (2021-09-17) 0.20    (2021-09-17) =
cdms2             3.1     (-         ) -       (-         ) (!)
cfgrib            0.9     (2019-02-25) 0.9     (2019-02-25) =
cftime            1.5     (2021-05-20) 1.5     (2021-05-20) =
dask-core         2022.1  (2022-01-14) 2022.1  (2022-01-14) =
distributed       2022.1  (2022-01-14) 2022.1  (2022-01-14) =
flox              0.5     (2022-05-02) 0.3     (2021-12-28) > (!)
h5netcdf          0.13    (2022-01-12) 0.13    (2022-01-12) =
h5py              3.6     (2021-11-17) 3.6     (2021-11-17) =
hdf5              1.12    (2021-01-01) 1.12    (2021-01-01) =
iris              3.1     (2021-11-23) 3.1     (2021-11-23) =
lxml              4.7     (2021-12-14) 4.7     (2021-12-14) =
matplotlib-base   3.5     (2021-11-17) 3.5     (2021-11-17) =
nc-time-axis      1.4     (2021-10-23) 1.4     (2021-10-23) =
netcdf4           1.5.7   (2021-04-19) 1.5     (2021-04-19) = (w)
numba             0.55    (2022-01-14) 0.55    (2022-01-14) =
numpy             1.21    (2021-06-22) 1.21    (2021-06-22) =
packaging         21.3    (2021-11-18) 21.3    (2021-11-18) =
pandas            1.3     (2021-07-02) 1.3     (2021-07-02) =
pint              0.18    (2021-10-26) 0.18    (2021-10-26) =
pseudonetcdf      3.2     (2021-10-16) 3.2     (2021-10-16) =
pydap             3.2     (2020-10-13) 3.2     (2020-10-13) =
rasterio          1.2     (2021-09-02) 1.2     (2021-09-02) =
scipy             1.7     (2021-06-27) 1.7     (2021-06-27) =
seaborn           0.11    (2020-09-19) 0.11    (2020-09-19) =
sparse            0.13    (2021-08-28) 0.13    (2021-08-28) =
toolz             0.11    (2020-09-23) 0.11    (2020-09-23) =
typing_extensions 4.0     (2021-11-17) 4.0     (2021-11-17) =
zarr              2.10    (2021-09-19) 2.10    (2021-09-19) =

Errors:
-------
1. not found in conda: cdms2

[keewis]

this seems fine to me, but I didn't check the typing / h5netcdf parts too closely

[headtr1ck]

I have very few time lately but I should be able to have a look in the next few days of you can wait a bit longer.
But also fine if you merge now and I check this dtype thing later.

[dcherian]

Thanks @jhamman

[st-bender]

Hi,
Just to let you know that this change breaks python 3.8 setups with automatic updates becuase the pandas version is not restricted, so it will be happily updated to version 2 or higher. Which in turn is not compatible with xarray < 2023.2, which cannot be installed on python 3.8 because of this change.
Don't know why the min python version was changed, this PR doesn't say why it was necessary.
Cheers.

[dcherian]

Here is our support policy for versions: https://docs.xarray.dev/en/stable/getting-started-guide/installing.html#minimum-dependency-versions though I think we dropped py38 too early.

For your current issue, I'm surprised this patch didn't fix it: conda-forge/conda-forge-repodata-patches-feedstock#429

cc @hmaarrfk @ocefpaf

[st-bender]

Hi @dcherian

Here is our support policy for versions: https://docs.xarray.dev/en/stable/getting-started-guide/installing.html#minimum-dependency-versions though I think we dropped py38 too early.

I assume you have given this a lot of thought, but imho the minimum dependency versions should be decided according to features needed, not timing.

For your current issue, I'm surprised this patch didn't fix it: conda-forge/conda-forge-repodata-patches-feedstock#429

Thanks for the pointer.
I am not sure why, maybe I was updating too eagerly before the feedstock was fixed, but mamba update --all on py38 pulled pandas 2.0 without updating xarray.

python3.8 -m pip install xarray will result in incompatible versions.

cc @hmaarrfk @ocefpaf

[crusaderky]

I assume you have given this a lot of thought, but imho the minimum dependency versions should be decided according to features needed, not timing.

It's not based on timing.
The policy is there so that, when a developer finds that they have to do extra labour to support an old version of a dependency, they can instead drop the support for the old version without needing to seek approval from the maintainers.

[st-bender]

Hi,

I assume you have given this a lot of thought, but imho the minimum dependency versions should be decided according to features needed, not timing.

It's not based on timing. The policy is there so that, when a developer finds that they have to do extra labour to support an old version of a dependency, they can instead drop the support for the old version without needing to seek approval from the maintainers.

That's not how I interpret the link given by @dcherian, which states "rolling" minimum versions based on age.

[MuellerSeb]

This also breaks xarray on ubuntu 20.04 which ships with Python 3.8 and is supported until April 2025. Python 3.8 is also supported at least until October 2024. Together with the Pandas 2 issue (that I am also running into), this is very unfortunate.

[crusaderky]

This also breaks xarray on ubuntu 20.04 which ships with Python 3.8 and is supported until April 2025. Python 3.8 is also supported at least until October 2024.

Not investing effort to support 5 years old dependencies was a very conscious decision. This is not something unique we do; we simply adhere to NEP29: https://numpy.org/neps/nep-0029-deprecation_policy.html

If for whatever reason you want to use python shipped by ubuntu 20.04, as opposed to conda/venv/poetry/whatever, you should also be prepared to stick to older versions of the python packages. Note that 5 years is the duration of security support. I'm not personally aware of security issues in xarray since Python 3.8 was abandoned (I've been a bit out of the loop and I could be proven wrong), but in the unlikely event that one should arise in xarray, we would consider a backport to Python 3.8.

[crusaderky]

It just occurred to me that xarray dropped Python 3.8 3 months before what NEP-29 recommends. I think this is a problem.
Let's continue this discussion on #7777.