Skip to content
This repository has been archived by the owner on Sep 20, 2024. It is now read-only.

Deadline: Be able to pass Mongo url to job #2616

Merged

Conversation

iLLiCiTiT
Copy link
Member

Brief description

Add option to pass mongo url to deadline job.

Changes

  • added attributes pass_mongo_url to CollectDefaultDeadlineServer plugin which is used in submitters to define if OPENPYPE_MONGO environment is passed to job environments

Testing notes:

  1. Trigger deadline publishing
  2. The job (worker) should not need to have set mongo url in keyring or in global environments

@mkolar
Copy link
Member

mkolar commented Jan 31, 2022

@iLLiCiTiT iLLiCiTiT self-assigned this Jan 31, 2022
@iLLiCiTiT iLLiCiTiT added backend module: Deadline AWS Deadline related features labels Jan 31, 2022
@iLLiCiTiT iLLiCiTiT requested review from antirotor and kalisp and removed request for antirotor January 31, 2022 18:10
@BigRoy
Copy link
Collaborator

BigRoy commented Jan 31, 2022

The job (worker) should not need to have set mongo url in keyring or in global environments

What's the use case for this? As in, why do we want to pass the mongo URL? Is it just so it doesn't need to be explicitly set on each machine?

We could also do it the other way around and have the default OPENPYPE_MONGO value when not defined in a job set in GlobalJobPreLoad and use it from a setting on the OpenPype Deadline Plug-in. Similar to how OpenPype Executables is collected? (Which can be done without needing the ingest that into jobs using an Event Plug-in as shown in #2605). Not saying that is better - but just throwing it out there. Actually, that might be less useful. On the off chance that a single studio runs against multiple OpenPype databases (e.g. OP 2 vs 3 or maybe some other means of splitting things up in completely different setups) but use a single Deadline database then moving it to Deadline Plug-in settings would actually be less useful. I guess allowing to pass it with the job is our best bet then.


Also, these code changes really want me to offload the actual deadline submission into its own library function for the Deadline module, just so we have more code re-use across all plug-ins instead of needing to keep them in sync like this, copying and pasting the code to all of them.

@mkolar
Copy link
Member

mkolar commented Feb 2, 2022

What's the use case for this? As in, why do we want to pass the mongo URL? Is it just so it doesn't need to be explicitly set on each machine?

Exactly, there is a solid pushback against setting them on machines on a few client sites, so this should make it more flexible.

@mkolar
Copy link
Member

mkolar commented Feb 2, 2022

Also, these code changes really want me to offload the actual deadline submission into its own library function for the Deadline

very true. @antirotor didn't you start working on that some time ago?

@mkolar mkolar requested review from mkolar and removed request for mkolar February 2, 2022 09:55
@antirotor
Copy link
Member

very true. @antirotor didn't you start working on that some time ago?

yes but just bits and pieces. Nothing definitive and I agree that it must be done

Submission to DL checks context for flag for pushing through OPENPYPE_MONGO
Copy link
Member

@kalisp kalisp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there should be context.data.get and not context.get , at least according to my testing.
If not, please revert commit, if this fix is ok, just dismiss review and re-request.

@iLLiCiTiT iLLiCiTiT requested a review from kalisp February 7, 2022 12:23
@antirotor antirotor merged commit a1d84a5 into develop Feb 7, 2022
@antirotor antirotor deleted the feature/OP-2539_Add-openpype-mongo-environments-to-deadline branch February 7, 2022 14:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
module: Deadline AWS Deadline related features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants