Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for GPU parameters at StepChain spec level #10401

Closed
amaltaro opened this issue Mar 31, 2021 · 2 comments · Fixed by #11588
Closed

Add support for GPU parameters at StepChain spec level #10401

amaltaro opened this issue Mar 31, 2021 · 2 comments · Fixed by #11588

Comments

@amaltaro
Copy link
Contributor

amaltaro commented Mar 31, 2021

Impact of the new feature
ReqMgr2

Is your feature request related to a problem? Please describe.
It's a new project to support GPU processing in central production workflows, where WM system will be the bridge between Offline/CMSSW and the grid resources made available to CMS (through the glideinWMS layer).

Describe the solution you'd like
This GH issue depends on #10388, which contains all the specific details to be implemented in this ticket as well.

Then, the solution expected from this issue is to support the same RequiresGPU and GPUParams parameters at the StepChain spec. Those 2 parameters should also be supported at the Step dictionary level. In case the Step dictionary is missing these 2 parameters, they should be inherited from the top level specification (just as it's done for Multicore).

Note that the same validation should also apply to the Step level parameters.

Describe alternatives you've considered
Not to validate the optional parameters.

Additional context
Major discussion happened here: cms-sw/cmssw#33057
and description of these parameters: cms-sw/cmssw#33057 (comment)

Related to #11410

@amaltaro
Copy link
Contributor Author

amaltaro commented Apr 3, 2023

@khurtado we might just update this GH issue (from the action item created in today's WMCore meeting).

@amaltaro
Copy link
Contributor Author

amaltaro commented May 9, 2023

As discussed in different venues, StepChain requests are not suitable for different resource requirements among the different steps.
Having said that, the only option we have at the moment is to enforce the whole StepChain workflow to require GPU, whenever any of its steps requires GPU.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant