Platform proposal #1
Conversation
| (including `localhost`). | ||
|
|
||
| * If a platform has a list of hosts then the host used for each operation on that | ||
| platform (job submission, polling, log retrieval, etc) will be randomised. This |
There was a problem hiding this comment.
Since the host select functionality was migrated to Cylc we have selection criterion and thresholds we could also use.
(There is considerable scope to improve this for little extra work if interested, see cylc/cylc-flow#3391)
It would make sense to unify this with the existing host select functionality.
docs/proposal-platforms.md
Outdated
| * Note that the platform will be selected each time a job is run. If you | ||
| specify a list of platforms, tasks must not assume that retries of that task | ||
| will use the same platform. | ||
| * For convenience it will also be possible to specify a platform alias which |
There was a problem hiding this comment.
This gives us a pretty much complete rose host-select replacement.
On the rose host-selectie side of things, how does suite servers fit into this model?
suite servers is effectively a logical grouping i.e:
platforms = cylc01, cylc02, cylc03, ...Which also has some extra configuration for the manner of host selection.
| will seldom be needed in the future (the main use at present is to select from | ||
| a list of login nodes). However, there are still cases where this may be | ||
| needed. For example running a command which: | ||
| * Returns the platform which has the shortest queue time. |
There was a problem hiding this comment.
Note: Functionality we may want to build into Cylc.
|
|
||
| ```ini | ||
| [platforms] | ||
| [[desktop\d\d|laptop\d\d]] |
There was a problem hiding this comment.
Using regexes in this way presents problems for inheritance / overriding due to the multitude of ways in which a regex could be specified.
Will need to think on whether there is a nice way to do this better.
|
|
||
| * The default for `hosts` is the platform name and platform section headings can | ||
| be regular expressions to match multiple platform names. This allows us to | ||
| define many platforms in a single section and handles the case where you have |
There was a problem hiding this comment.
This is effectively the old rose host-select functionality of grouping together "like" hosts. Perhaps there is a better way to specify these "groupings"?
| * The default platform is `localhost`. At cylc 7, any settings for `localhost` | ||
| act as defaults for all other hosts but we do **not** propose doing this for | ||
| platforms. The downside is more duplication if you need to override a | ||
| particular setting for every platform but we think this is clearer and safer. |
There was a problem hiding this comment.
Perhaps we might want to think about the "inheritance" model. Would it be a good idea if we were able to do something along the lines of:
[*]
# settings which apply to *all* platforms
[abc\d\d\d]
# settings which apply to abc123
[abc*]
# settings which also apply to abc123
docs/proposal-platforms.md
Outdated
| ```ini | ||
| [platforms] | ||
| [[hpcl[12]-bg]] | ||
| hosts = ("-bg$","") |
| * The simplest example. The job will be run on the host `desktop01`. | ||
|
|
||
| `platform = special` | ||
| * There is no platform named `special` defined so this will fail validation. |
There was a problem hiding this comment.
This is an interesting point. It means that suites could fail validation depending on the global configuration.
We should make this optional to allow for CI testing etc.
docs/proposal-platforms.md
Outdated
| * The host used will be chosen at random. | ||
|
|
||
| `platform = hpcl1-bg, hpcl2-bg` | ||
| * The platform used will be chosen at random. |
There was a problem hiding this comment.
I think it might be better to specify such "groupings" explicitly and more centrally, this would give us the ability to configure the behaviour of the group e.g. host selection method. Random is good, but for various reasons we have built extra functionality into Cylc already.
|
|
||
| ## Deprecation method | ||
|
|
||
| * You can't mix old with new. If `platform` is defined for **any** task then any |
There was a problem hiding this comment.
So this would mean we can't override settings at the task level e.g:
[foo]
platform = bar
[[platform]]
host = bar01 # hardcode a login nodeThis is probably a good thing the user should just go away and create a new platform with the modification.
Under this proposal, in particular the system of effectively naming platforms after their regexes, how could I duplicate a platform to apply a small diff?
dpmatthews
force-pushed
the
platform-proposal
branch
from
0dbab58
to
12f6e72
Compare
dpmatthews
force-pushed
the
platform-proposal
branch
from
12f6e72
to
517ba5c
Compare
No description provided.