Unable to install aiida-core in python 3.9 conda environment

[ltalirz]

From aiidateam/aiida-core#5037 by @csadorf

It is currently not possible to install aiida-core with Python 3.9, see minimal example:

$ conda create -n aiida-py39 python=3.9 aiida-core
Collecting package metadata (current_repodata.json): \done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: /
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package python conflicts for:
aiida-core -> aio-pika~=6.6 -> python[version='2.7.*|3.5.*|3.6.*|>=3.6|>=3.9,<3.10.0a0|>=3|>=3.5,<3.6.0a0|=2.7|>=3.5|>=2.7|3.4.*|3.8.*|3.7.*']
python=3.9
aiida-core -> python[version='>=2.7,<2.8.0a0|>=3.5|~=3.7|>=3.6,<3.7.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0']

This is odd, because the package is no-arch as of 6.7.1 which would be compatible with our constraint. Installing aio-pika in isolation works flawlessly:

conda create -n aio-pika python=3.9 aio-pika~=6.6

and will install version 6.8.

[ltalirz]

@astamminger any ideas?

[csadorf]

@ltalirz Please feel free to assign this issue to me for the time being.

[chrisjsewell]

It seems to be suggesting that aiida-core cannot be python 3.9.
What if you try to install when specifying aiida-core=1.6.4

[chrisjsewell]

It’s of note that before v1.4 aiida-core indeed has not python3.9 dist: https://anaconda.org/conda-forge/aiida-core/files

I believe it is trying to install an older version of aiida-core to meet the aio-pika requirement

[csadorf]

It seems to be suggesting that aiida-core cannot be python 3.9.
What if you try to install when specifying aiida-core=1.6.4

It fails with:

$ conda create -n test python=3.9 aiida-core==1.6.4
# [...]
Package python conflicts for:
python=3.9
aiida-core==1.6.4 -> aio-pika~=6.6 -> python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.6|>=3.9,<3.10.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0|>=3|>=3.5,<3.6.0a0|=2.7|>=3.5|>=2.7']
aiida-core==1.6.4 -> python~=3.7

This is a bit cryptic to me since aio-pika is available for Python 3.9 in version 6.8 which should be compatible with ~=6.6.

It’s of note that before v1.4 aiida-core indeed has not python3.9 dist: https://anaconda.org/conda-forge/aiida-core/files

I believe it is trying to install an older version of aiida-core to meet the aio-pika requirement

Maybe, but even when I explicitly request the latest version, it still fails with a weird error (see above).

As another data point, I ran a few tests and was able to create an environment based on the aiida-core/environments.yml file (largely identical to the requirements specified in the conda recipe) with Python 3.9 by disabling a few requirements (shown below):

diff environment.yml environment.py39.yml
8c8
< - python~=3.7
---
> - python~=3.9
19c19
< - django~=2.2
---
>   #- django~=2.2
28,30c28,30
< - plumpy~=0.19.0
< - pgsu~=0.2.0
< - psutil~=5.6
---
>   #- plumpy~=0.19.0
>   #- pgsu~=0.2.0
>   #- psutil~=5.6

[ltalirz]

hm.... https://github.com/conda-forge/pgsu-feedstock/blob/f1bc80c1ddf54232e178d1814582c3ef3027f25e/recipe/meta.yaml#L17

[ltalirz]

I'll take care of that conda-forge/pgsu-feedstock#8

[ltalirz]

PR is merged conda-forge/pgsu-feedstock#8
Perhaps re-try in ~1h whether the issue still persists

[csadorf]

PR is merged conda-forge/pgsu-feedstock#8
Perhaps re-try in ~1h whether the issue still persists

@ltalirz Thank you very much. That has improved the situation. The two remaining issues are:

• plumpy, which has no Python 3.9 distributions
• django, which has no Python 3.9 distributions, but is noarch as of version 3.0.9.

I understand we can fix plumpy ourselves, but django might be an issue, since we are currently constrained to ~=2.2. Would it be possible to make django an optional dependency?

[ltalirz]

For plumpy there's a PR open conda-forge/plumpy-feedstock#47
The build failed but the logs are no longer available; I'm restarting this to see what the problem was

For django, I think the easiest might be to get them to do a python3.9 release for the 2.2.x line
https://github.com/conda-forge/django-feedstock

I'm not sure whether there is now an established route for adding a build for previous versions - @chrisjsewell @astamminger do you know?

[ltalirz]

For plumpy, I think the issue is the outdated pyyaml 5.1.2 dependency conda-forge/plumpy-feedstock#47 (comment): https://github.com/aiidateam/plumpy/blob/77e38202a1763fcf61fb67664542715c83351bc6/setup.py#L33

Opened aiidateam/plumpy#220 (also has links to previous discussion on this)

[astamminger]

I'm not sure whether there is now an established route for adding a build for previous versions

There are separate branches to also support updating of older versions. Maybe re-rendering on that branch would do the trick?
Not sure, never tried :D

[ltalirz]

Thanks @astamminger , that makes sense - I'll give it a go
conda-forge/django-feedstock#124

[ltalirz]

django should be taken care of now as well conda-forge/django-feedstock#124 - just plumpy remains

[csadorf]

Can we make a new aiida-core release to resolve this issue?

[csadorf]

@ltalirz With version 1.6.5, aiida-core can now be installed with conda and Python 3.9. The issue is therefore resolved.

[ltalirz]

fantastic, thanks to everyone involved!