Accommodate Nextflow CPU/Memory in Nomad Job taskResources #27
Conversation
| void "submit a task with docker volume"(){ | ||
| given: | ||
| def config = new NomadConfig( | ||
| client:[ | ||
| address : "http://${mockWebServer.hostName}:${mockWebServer.port}" |
There was a problem hiding this comment.
I've removed this test as we decided not to continue with docker volume.
abhi18av
changed the title
|
|
||
| JobRegisterRequest jobRegisterRequest = new JobRegisterRequest(); | ||
| jobRegisterRequest.setJob(job); | ||
| JobRegisterResponse jobRegisterResponse = jobsApi.registerJob(jobRegisterRequest, config.jobOpts.region, config.jobOpts.namespace, null, null) | ||
| jobRegisterResponse.evalID | ||
| } | ||
|
|
||
| TaskGroup createTaskGroup(String id, String name, String image, List<String> args, String workingDir, Map<String, String>env){ | ||
| def task = createTask(id, image, args, workingDir, env) | ||
| TaskGroup createTaskGroup(String id, String name, String image, List<String> args, String workingDir, Map<String, String>env, Resources resources){ |
There was a problem hiding this comment.
don't like to expose specific nomad objects outside of the service
I mean, Resources is something strong related with the Service but not with the Task. Maybe we can use some TaskResource class as parameter and let the Service do the conversion to a nomad Resource object
it sounds a little more complicated/elaborated but in this way we have separate both concepts (nextflow vs nomad)
There was a problem hiding this comment.
Hi @jagedn , okay so I've tried to refactor the code keeping different layers separate. Let me know what you think about the current structure?
I do think that we can explore the idea regarding a specific "translation" class which covers all aspects of NomadJobDefinition from a NextflowTask, however I suggest that we still continue the current iterations and revisit this once the plugin has stabilised in its design - what do you think?
I have drafted an initial support for cores and memory.
NOTE: As Nomad interprets
cpusinMegaHertzdifferent from the integer values inNextflow, therefore, I have mappedNextflow CPU -> Nomad Cores.Tested this with the
nf-core/fetchngspipelineResources