Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow trigger-now in a paused workflow? #5963

Open
hjoliver opened this issue Feb 9, 2024 · 4 comments · May be fixed by #6192
Open

Allow trigger-now in a paused workflow? #5963

hjoliver opened this issue Feb 9, 2024 · 4 comments · May be fixed by #6192
Labels
question Flag this as a question for the next Cylc project meeting.
Milestone

Comments

@hjoliver
Copy link
Member

hjoliver commented Feb 9, 2024

There is a valid use case (especially since the demise of the old cylc submit command for running individual tasks sans scheduler) early on in workflow development and debugging, for starting a workflow paused and then manually triggering individual tasks to run.

Currently "paused" means "don't run anything", so if you trigger a task in a paused workflow, the task won't run until the workflow gets un-paused.

I think we probably intended cylc hold for this sort of thing, but that doesn't do it without additional seemingly unnecessary effort: holding an already-active task (even via a "hold all" pattern) does not prevent downstream future tasks from running unless every one of those has also been specifically held.

So ... is there any good reason not to allow manual triggering (in the "run right now" sense) of tasks in a paused workflow?

See cylc/cylc-ui#1651

@hjoliver hjoliver added the question Flag this as a question for the next Cylc project meeting. label Feb 9, 2024
@hjoliver hjoliver transferred this issue from cylc/cylc-ui Feb 9, 2024
@oliver-sanders oliver-sanders added this to the some-day milestone Feb 9, 2024
@ColemanTom
Copy link
Contributor

I would personally like this (which is probably obvious based on the previous Issues I've raised).

@dwsutherland
Copy link
Member

So would I, even to be able to release/resume a specific branch while the rest stays paused/held

@oliver-sanders
Copy link
Member

oliver-sanders commented Jun 14, 2024

Allow trigger-now in a paused workflow?

This use case has always been possible at Cylc 8, use the "hold after cycle point". Documenting this in cylc/cylc-doc#739

So would I, even to be able to release/resume a specific branch while the rest stays paused/held

This will not be possible via play/pause because this is a global setting (i.e. you can pause a workflow, but you can't pause a subset of it). However, this is possible via hold/release (which are local settings).

I.E. this is a hold/release use case, not a play/pause use case.

I think we probably intended cylc hold for this sort of thing

Yes.

Task hold and workflow pause are orthogonal features, there are advantages to being able to slam the brakes on a workflow, manual triggers and all. There are also interventions beyond trigger that can cause tasks to submit (e.g. set).

Holding all tasks in a workflow is a little clumsy at the moment due to future matching requirements (#5827), however, this is a priority issue to solve so should go away before too long. Holding all tasks via this interface achieves the desired result, IMO this is a better solution to changing the meaning of pause to satisfy this use case.

What remains (after #5827) is to improve access to task hold to make it easier to reach and more intuitive to use, ideas:

  • Consider reinstating the cylc run --hold argument to start a workflow with all tasks held.
  • Improve access to the Hold/Release commands in the GUI menu.
  • Consider changing the default task list in the GraphQL mutations for Hold/Release from null to */*. This would mean that if you clicked the Hold command from the workflow menu, it would hold all tasks by default without the user having to type */* into the box.
  • Fix this small data store issue data store: n>0 tasks past the hold point are not marked as held #6140 which makes the effect of "hold after cycle point" less obvious.

With this using hold/release would be as convenient as using play/pause.

@hjoliver
Copy link
Member Author

hjoliver commented Jun 18, 2024

there are advantages to being able to slam the brakes on a workflow, manual triggers and all.

Not sure I buy this. What's the advantage of being able to slam the brakes on manual "trigger now" exactly? By definition that means run this task now, so I wouldn't do it in the first place if there was any reason I didn't actually want it to trigger.

There's nothing fundamentally wrong with cylc pause meaning "the scheduler will not automatically submit any tasks" but the user can still do it manually.

It's conceptually (and implementation-ally) simpler than cylc hold "*" where users may have to think about exactly what the glob is going to match.

If we do make cylc hold "*" completely task-pool agnostic it would not matter that a similar thing might be achievable both ways in the future. After all, there's almost complete overlap between the effect of cylc pause <workflow> and cylc hold <all tasks>.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Flag this as a question for the next Cylc project meeting.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants