Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

background job runner and multiple hosts? #4922

Closed
hjoliver opened this issue Jun 21, 2022 · 3 comments · Fixed by #4938
Closed

background job runner and multiple hosts? #4922

hjoliver opened this issue Jun 21, 2022 · 3 comments · Fixed by #4938
Assignees
Labels
could be better Not exactly a bug, but not ideal.
Milestone

Comments

@hjoliver
Copy link
Member

hjoliver commented Jun 21, 2022

This is currently legal:

[platforms]
   [[bg-jobs]]
        hosts = foo, bar, baz  # OOPS
        job runner = background

However it doesn't make sense for the background job runner to allow multiple interchangeable hosts, because background jobs have to be polled and killed by PID on the exact host they were submitted to. (Similarly at, presumably).

So I guess we should either:

  1. make it an error to list multiple hosts for the background job runner
  2. or handle background jobs differently: allow multiple job submission hosts but always interact with jobs via the specific host they were submitted to

(And either way, document that background platforms are different in this respect).

What do you think @wxtim ? (I don't think we're already doing 2. above? ... if we are, close as invalid!)

@hjoliver hjoliver added bug? Not sure if this is a bug or not question Flag this as a question for the next Cylc project meeting. labels Jun 21, 2022
@hjoliver hjoliver added this to the cylc-8.0.0 milestone Jun 21, 2022
@dpmatthews
Copy link
Contributor

make it an error to list multiple hosts for the background job runner

Seems reasonable (and for at).
Not really a bug though - it's really just a configuration issue that we can catch (there are plenty we can't).

or handle background jobs differently: allow multiple job submission hosts but always interact with jobs via the specific host they were submitted to

No - this is what platform groups are for.

@hjoliver
Copy link
Member Author

make it an error to list multiple hosts for the background job runner

Seems reasonable (and for at). Not really a bug though - it's really just a configuration issue that we can catch (there are plenty we can't).

Fair enough. It is a likely configuration mistake though - it would be easy to think a background platform should list of "all the hosts
we can run background jobs on".

or handle background jobs differently: allow multiple job submission hosts but always interact with jobs via the specific host they were submitted to

No - this is what platform groups are for.

👍

@hjoliver hjoliver added could be better Not exactly a bug, but not ideal. and removed bug? Not sure if this is a bug or not labels Jun 22, 2022
@oliver-sanders
Copy link
Member

oliver-sanders commented Jun 22, 2022

make it an error to list multiple hosts for the background job runner

Makes sense to me, same for at. Could be a simple validation check in the global config.

@dpmatthews dpmatthews removed the question Flag this as a question for the next Cylc project meeting. label Jun 22, 2022
@hjoliver hjoliver self-assigned this Jun 28, 2022
@hjoliver hjoliver modified the milestones: cylc-8.0rc4, cylc-8.0.0 Jul 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
could be better Not exactly a bug, but not ideal.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants