family hold beyond n=0

[hjoliver]

Currently cylc hold ARG works to hold:

• future tasks, if ARG is a specific task ID
• waiting tasks in the scheduler task pool, if ARG is a glob or a family name

@ColemanTom wants to hold a whole family in advance - i.e. not just the family members, if any, currently in the pool.

https://cylc.discourse.group/t/how-to-rewind-workflow-not-to-a-checkpoint/719

It's not necessarily easy to figure out all the individual task IDs and hold them separately.

[hjoliver]

From #4246

Me:

Presumably we could figure out what the matched tasks or family members will be, and put those in the future hold list.

@oliver-sanders:

The suggestion of expanding families is orthogonal and was previously rejected due to issues with multiple flows and graph branching.

I've kinda forgotten that conversation, sorry Oliver. But maybe we made the wrong call (or you can correct me again).

It is possible of course, but would require more thought, for consistency it should apply to all of the "task glob" commands, but we would need to think about how that would interact with cylc reset, there may be a housekeeping problem with graph branching (if an action is run against a task which isn't run, then we will accumulate messages causing a slow memory leak unless housekept), and if expanding families, why not cycles?

We could, initially at least, only implement this for families. A family name is simply than a glob, and less risky. There's no wildcard involved, the members are well-defined.

Ideally we could do it with globs too, but there is as clear use case (as raised by Tom) for family hold, and we could implement that first.

The other issue you raise there also applies to individual tasks, right? I can hold a future task, by ID, that might not ever appear in the pool because of future graph branching. If we don't make housekeeping possible somehow, that would cause a memory leak (the list of future tasks to hold), but any such leak should be small and inconsequential - it would only grow as a result of a small number of manual intervention commands.

[oliver-sanders]

I've kinda forgotten that conversation, sorry Oliver. But maybe we made the wrong call

Possibly, I'm not particularly happy with the status quo which is definitely a backwards step from Cylc 7 so happy to re-open this one. For reference the list of commands which currently "glob" for tasks are:

• hold
• kill
• pause
• ping
• poll
• release
• remove
• set_outputs
• show
• trigger

We would need to make sure any proposed functionality could be adapted for all of these. Note, currently only the "trigger" command will auto-spawn a task.

Off the top of my head, the issues with doing this are:

• Should probably have some form of housekeeping for operations.
  • Could couple to the runahead limit?
• Operations apply to all selected tasks irrespective of whether they would be run or not (according to graph logic).
  • Perhaps irrelevant providing we have housekeeping?
• Operations applied to one flow could linger and affect a subsequent flows.
  • Housekeeping might help to narrow down the window of opportunity for this.
  • Alternatively we could add the --flow argument to these commands and default as --flow=all. If we expand that all into the corresponding flow numbers at the time the command is processed then we could limit the scope of the operation so that subsequent flows are not affected.
• Require some form of visibility for requested operations which are waiting for tasks to be spawned before taking effect.
  • No idea on this one.

[oliver-sanders]

IMO, we should come up with a general policy for all these commands rather than working around the limitations in each command individually e.g. #5677.

[oliver-sanders]

The following issues all concern the limitation of multi-task commands where globs/families are only expanded in the context of the task pool.

• family hold beyond n=0 #5695
• Held running tasks: delay spawning or hold children? #5696
• cylc show for future tasks #5677
• Clearer message for task not found in n=0 #5678

If we are going to address this we should come up with a consistent approach which works for the relevant commands so we can implement and document it consistently rather than hashing out the details for each command separately.

This is also somewhat related to:

• tui: expand cycles/families in mutations to the visible selection #4246
• aoft: expand cycles/families in mutations to the visible selection cylc-ui#686

Although this issue will not close those by itself as this is a different problem related to display and selection.

[hjoliver]

IMO, we should come up with a general policy for all these commands rather than working around the limitations in each command individually e.g. #5677.

If we are going to address this we should come up with a consistent approach which works for the relevant commands so we can implement and document it consistently rather than hashing out the details for each command separately.

If possible, but issues typically first come to light with individual commands - particular when reported by users as is the case with all of these - and need to be documented (Issues posted, I mean) as they arise. We can certainly consolidate and triage later, if appropriate.

And in the context of this particular issue, "hold" is kind of a special case - we can already hold future tasks individually, which means maintaining a list of future tasks that will be held if/when they get spawned. The problem is simply that we need to be able to hold future tasks collectively - by family name or pattern match.

I can't see much commonality with the other commands you've listed above, in terms of teeing up future behaviour.

Note, currently only the "trigger" command will auto-spawn a task.

And set-outputs (soon to be set).

[hjoliver]

Superseded by #5827