Making GroupPath more powerful 🚀📁

[zhubonan]

I found that the use of Group being an essential part of organising data in AiiDA.
The existing GroupPath is very useful, for example, I can group my works into groups of structure and workflows using the structures of various properties.
On the other hand, I think we can perhaps enhancement the system a bit further. On file system, a path can be either a directory or a file. Perhaps we can have similar thing in AiiDA as well? A the moment a GroupPath always points to a (potential) Group (like a directory), but not any actual data (Node)

For example, I am performing a study of material X, and have the following groups

X/structures/exp   #initial experimental structure
X/structures/optimsed  # optimise strucrture for further analysis
X/relaxation  # relaxation workflows
X/bandstructure  # band structure workflows
X/phonons   # phonon workflows
X/......

If I run these calculations "by hand", I can place folders of calculations under relaxation, a structure into exp and the final optimised structure into optimised an so on. This allow a nature way of organising the data.

The issue of them being groups in AiiDA is that to access the nodes, I have to use the nodes attribute, iterate and select the right one.
(Of cource, I can also just note down the UUID of these workflows somewhere, but this can be a bit scary for people not used to UUIDs).
It would be nice if, say, I can do X/structure/exp/icsd-128948 to access the node of the structure, and similar X/relaxation/final-pbe-converged to access the relaxation providing the converged structure, similar to working on a file system folder structure.

A possible way to achieve this is to assign some kind of alias to nodes (independent of label), this alias does not have to be unique, and can probably be stored in the extras section. If there are duplicated alias inside a Group one can just refuse to resolve the given GroupPath.

For example, group.add_nodes(node, alias='icsd-12992') woud allow GroupPath('X/structures/exp/icsd-12992') to be solved to the node.
The alias can be stored as:

{
    '_group_alias': 'icsd-12992'
}

in the extras of the node. It may also be possible to make the alias specific to each group, e.g. making the _group_alias a dictionary. Although perhaps this is a bit of overkill.

Another added benefit is that it would make people that already used to a file based manual working routine easier to work with AiiDA.

Do you think this can be a good idea?

Pinging GroupPath expert @chrisjsewell

[chrisjsewell]

Thanks for the ping @zhubonan, yeh sounds interesting.
I guess the "ideal" way to do this, would be to add another column to the db_dbgroup_dbnodes join table, with said alias,
but that could be a lot of work to write in to aiida-core 😬

[zhubonan]

Do you think there would be a performance issue with the extras approach?
I think adding another column would be too much work.

A hacky temporary solution would be subclass GroupPath , and add an add_node methods that add the node to the group and then set the alias in its extras.
This way no update of the working code in aiida-core is needed to begin with.

[chrisjsewell]

Do you think there would be a performance issue with the extras approach?

Potentially, but mainly it just doesn't quite make logical sense, that the alias should be the same for every group,
e.g. what happens if you do

group1.add_nodes(node, alias='an-alias')
group2.add_nodes(node, alias='another-alias')

Does it change the "global" alias, or does it except, disallowing changes to a previously set alias?

I think adding another column would be too much work.

Perhaps, but then there needs to be some thought, if there is a "middle ground"

[zhubonan]

Potentially, but mainly it just doesn't quite make logical sense, that the alias should be the same for every group,
e.g. what happens if you do

Yeah good point, my implementation at the moment would raise an error in the second call, enforcing the same name to be used.
I was previously thinking that the nodes are like files on the disk, so they should have a single name.

However, they are in fact the alias are more like hardlinks, so it makes sense for the same node to have different names in different "folders" (Groups).

[zhubonan]

I managed to get a prototype code working

For example:

base_path.browse.conv.conv_encut_520().get_node()

would get a node stored in conv path relative to the base path with alias conv_encut_520.

An interesting usage is to allow making tree digram of the group/nodes as folders, and decorate the leafs by attributes:

Perhaps I should make a plugin for it? No change of the code in aiida-core is needed.

[ltalirz]

Perhaps I should make a plugin for it? No change of the code in aiida-core is needed.

That sounds like a good way forward to me. We can always still include it (or a version of it) in aiida-core at a later point in time

[chrisjsewell]

Yeh cheers, looks good 😄
Indeed though, if we start with a plugin, it would be easier to mess around with initially

[zhubonan]

Sure I will put it into a plugin, easy to do it this way.

[zhubonan]

Code as a plugin is now located at:
https://github.com/zhubonan/aiida-grouppathx