⬆️ UPDATE: Siesta MaX-1.1

[albgar]

The updated role builds and deploys version 'MaX-1.1' of Siesta (with support for pseudopotentials in PSML format, and for Lua scripting, among other improvements). It does not build support for the ELSI library as this is more of a performance feature.

• It installs the siesta executable, as well as several post-processing utilities, in /usr/local/bin.
• Requisite domain-specific libraries (libpsml, xmlf90, libgridxc, flook) are in /usr/local/lib, and their module files in /usr/local/include.
• It uses ldconf to add to the shared-library search path.
• It installs files for several pseudopotential families in /usr/local/share/siesta/psml-files.
• For the interface with the AiiDA role, it defines the siesta_executables variable.

[ltalirz]

Thanks a lot @albgar, I can see this was quite a lot of work!

Just out of curiosity, since we were discussing this previously - have there been any developments on the spack/easybuild/conda front for siesta builds?

For now it's perfectly fine to install packages from source here in the ansible role, but of course this all needs to be maintained, i.e. when the next version of siesta is released, it likely won't be as easy as just increasing the siesta version in the default.yaml ;-)

[ltalirz]

@chrisjsewell Feel free to merge this or to review as well

[chrisjsewell]

thanks @albgar, just some nitpicks

[chrisjsewell]

@albgar on a related matter that you may be able to help with, I've actually this week created a role for building abinit.
In this I had an issue building for Ubuntu 20.04, because libblacs-mpi-dev was not available on apt (see apt installs here https://github.com/marvel-nccr/ansible-role-abinit/blob/master/tasks/main.yml, also link to docs https://www.netlib.org/blacs/).
But in https://github.com/siesta-project/ansible-role-siesta/blob/master/tasks/main.yml I notice that you just skip installing it for ubuntu > 16.
So is it just not required for these distros, or is there another package that takes its place?

[albgar]

@albgar on a related matter that you may be able to help with, I've actually this week created a role for building abinit.
In this I had an issue building for Ubuntu 20.04, because libblacs-mpi-dev was not available on apt (see apt installs here https://github.com/marvel-nccr/ansible-role-abinit/blob/master/tasks/main.yml, also link to docs https://www.netlib.org/blacs/).
But in https://github.com/siesta-project/ansible-role-siesta/blob/master/tasks/main.yml I notice that you just skip installing it for ubuntu > 16.
So is it just not required for these distros, or is there another package that takes its place?

Newer versions of scalapack have the blacs "built-in". So you just need to install the scalapack-dev (I forgot the exact name) package.

[chrisjsewell]

thanks! ok I'll try that

[chrisjsewell]

The updated role builds and deploys version 'MaX-1.1' of Siesta

Also one last thing, similar to what @ltalirz asked, with the changes to this role, what versions does it now support; only MaX-1.1 or is it e.g. back-compatible to v4.1, and what will potentially be involved to "upgrade" to the next version of siesta.

Also in terms of aiida-siesta, I assume the current version of that supports this version of siesta (and since what aiida-siesta version).

Perhaps some information like this could be added to the README.md, and also your comment about not supporting the ELSI library etc

[albgar]

We have currently a number of features in development waiting to be merged, and at the same time we are in the middle of a re-design of the building system for Siesta, so things are in flux. Hopefully, with the new building system in place and a more sedate version evolution, the role can inherit a bit more stability.

I have added a few more comments to README.md.

[chrisjsewell]

Thanks again @albgar but one last thing 😬

So when adding the new Abinit role to Quantum Mobile, I found that all the libxc tests were failing when using the apt installed libxc-dev, which installs v4.0.0 (see marvel-nccr/ansible-role-abinit#9). But when I directly compiled v4.3.4 and linked to that it all worked.
So I've created a seperate https://github.com/marvel-nccr/ansible-role-libxc role and in marvel-nccr/ansible-role-abinit#10 allow for the direct specification of the path to libxc, e.g. pointing to /usr/local/libxc-4.3.4 rather than the apt installed /usr/local.

Given that Siesta also uses libxc;
(a) will one of the default tests identify any issues with libxc, e.g. use a PAW xc-functional, and
(b) if there are any issues, how easy/hard would it be to allow for a similar direct specification of the libxc path

edit: we could even consider moving tasks/libpsml.yml to this shared role, as I believe abinit can also link to that

[albgar]

This is strange... in the virtual machine I build on focal with my current role the version of libxc-dev is 4.3.4-1 (also stated in https://packages.ubuntu.com/search?keywords=libxc-dev&searchon=names&suite=focal&section=all), and not 4.0.0 as you seem to imply.

Nevertheless, the idea to install libxc explicitly is good, as we would have full control over it.
However, I would like to re-use the Siesta role for things other than the vanilla Quantum Mobile, and then it would be better if the installation of libxc is local to the role, instead of assuming that somehow we are installing libxc earlier in the sequence of tasks. On the other hand, libxc installation can be time consuming, and the tests take long, so i can see the benefits of a "canonical" installation for everybody.

[chrisjsewell]

Yep that was the idea; to keep it modular.
The abinit role does not intrinsically depend on the libxc one, it simply defaults to apt installing unless you set a predefined path.
So this would not preclude the use of abinit or siesta outside of QM.

I note the link you provide only shows Ubuntu 20.04, perhaps for Ubuntu 18.04 (i.e. as currently used by QM) it is not as up-to-date?

[chrisjsewell]

Yep indeed it's a different version for 18.04: https://packages.ubuntu.com/search?keywords=libxc-dev&searchon=names&section=all

I do want to move QM to 20.04 eventually, but not quite there with all codes just yet!

[albgar]

I have implemented a mechanism to use the libxc role if the distribution's libxc-dev package is not up to standard. So in this sense I am using the libxc role as a "subroutine". However, a few things would need to be improved to make this more robust (some probably reflect my limited ansible skills):

• Since compilation of libxc is expensive, before "calling" the libxc role I check for existence of the /usr/local/libxc-x.y.z directory, where x.y.z is the version I want. This is "hardwired" to the current installation prefix in the libxc role. If it changes, it will break. It would be great if the role could "return" quickly when called with a version that is already installed, and provided the installation path to the client.

• The libxc role does not currently install shared libraries. A client then needs to disable shared library support (e.g. in libgridxc) to use it. A role variable could be used to request shared libraries. However, this introduces more complications: what would happen if another role further down on the QM installation list actually installs a libxc-dev package, which has precedence in the shared libs search path?

Regarding the installation path, I see it as a welcome development that you are using sub-directories in /usr/local. Could everybody else do so? It would be good to have things in /usr/local/siesta-x.y.z, for example, in case we want to install different versions. Actually, support for modules (of the lmod kind) might be good too.

[albgar]

The CI checks fail because the libxc role cannot be found. How do I fix this?

[chrisjsewell]

If you have a look on the abinit role's molecule folder, you can see ho I do it there: look at the molecule.yml and requirements.yml

[chrisjsewell]

The libxc role does not currently install shared libraries. A client then needs to disable shared library support (e.g. in libgridxc) to use it. A role variable could be used to request shared libraries. However, this introduces more complications: what would happen if another role further down on the QM installation list actually installs a libxc-dev package, which has precedence in the shared libs search path?

Yes I must admit, I'm not an expert in the use of shared libraries.
I did have a brief look before, if there was anything similar to python/conda virtual environments in the "apt world", but I didn't see anything obvious (although now that you mention lmod I guess that is somewhat similar).
Any good reading/resources you could point me to would be welcome.

Regarding the installation path, I see it as a welcome development that you are using sub-directories in /usr/local. Could everybody else do so? It would be good to have things in /usr/local/siesta-x.y.z, for example, in case we want to install different versions. Actually, support for modules (of the lmod kind) might be good too.

Obviously the drawback is that the executables will not then be in the $PATH by default, and its just not something we have needed for now in Quantum Mobile, where we only have a single version of each code, and don't necessarily require the extra complication.

But yes I could see lmod as useful for the more general use of these roles.

Perhaps using something like https://github.com/idiv-biodiversity/ansible-role-lmod (this only supports redhat though)
Then each role could write a ModuleFile and also, in some way, contribute to a meta module that just loads the "default" versions of all codes (a bit like how we have the release_notes role for generating a single file of all code versions).

[chrisjsewell]

Thanks for all the work!