Houdini: Add support to split Deadline render tasks in export + render #5420
Conversation
ynbot
added
host: Houdini
module: Deadline
There was a problem hiding this comment.
Looking at the amount of renderers this touches and how this is build this looks like a massive (and massively welcome) contribution - so great job.
There's definitely still some major caveats, but the concept is solid. I think having the foundation laid out in AbstractSubmitDeadline to e.g. support the same in a trivial manner in other hosts would be solid.
We could also do it differently where get_job_info or processs_submission or whatever could just receive multiple jobs like a list (or just the one for backwards compatibility). Then the Abstract Submit Deadline class would just submit each. Potentially the job infos themselves could've also prepped the dependencies in advance without the actual ids but just the actual jobs being submitted. Instead of adding extra arguments to the methods this would just allow the inherited submission plugins (like the one from houdini) to just submit any amount of jobs in order for the render. As long as they'd mark one the render job for the SubmitPublishJob logic I suppose?
def get_submission(self):
job_info_export = ...
job_info_render = ...
job_info_render.dependencies = [job_info_export]
return [job_info_export, job_info_render]
openpype/settings/entities/schemas/projects_schema/schema_project_deadline.json
Outdated
Show resolved
Hide resolved
This reverts commit aab913a.
… a bit more generic
I really like what you suggest of abstracting the dependencies on a separate function, I can try it out tomorrow |
|
sorry for my noob question
do you mean something like exporting mantra ifd, and use it to render in mantra ? I tested it as a user, and these are my results: I found 3 jobs
|
|
Great, I like the idea behind this. |
Yeah, all farm submission rendering pipelines that I have seen on all studios I have worked always do this, it's a big waste of license usage and resources to use the DCC license and its environment to render in the farm. Instead what you want is to export your render scene from your DCC into the scene format of the render engine (i.e., .rib for renderman, .ifd for Mantra, .ass for Arnold... in the future .usd for all that fully support Hydra) and then render directly from that scene file format so you can distribute it across multiple machines, that way you can chunk your translate task (usually 10 frames per task) to benefit from only having to open the scene once (using a single DCC license) and export to the render scene format (it's usually pretty fast if using procedurals) and then spawn all the render tasks with a different chunk size (usually 1 frame per task as renders are much slower) and using only render licenses which are usually much cheaper than the DCC one. You can find more info here for Mantra: https://www.sidefx.com/docs/houdini/render/ifd_workflows.html
It's a bit related but I don't think that's solving the same issue. Also while it might be useful in very edge cases I haven't personally seen the use of needing to cache and publish .ifd, redshift proxy or vray scenes before. Arnold .ass files are useful for caching as you can reference them as procedurals in any of the Arnold plugins but I'm not sure if that workflow is also common for other renderers. Maybe the redshift proxy and vray scenes are but .ifds? Given that those are unique to Houdini I'm not sure if there's much value to that. I will have a look at that PR and comment on it
That's correct yeah, and for an Arnold job you should see something like: I think it would be nice to add better labels so it's clear what's export vs what's render (even though it's kind of obvious in some cases due to the Plugin used) |
Thanks for testing @moonyuet ! And yeah as I added as |
Thank you @moonyuet for checking, I have now updated the code with "2" for both the create and collect of VRay ROP |
|
I got the error below when the vray-standalone run in the farm I believe it is mainly because inputFilename is {project_name}.0000.vrscene. If it is {project_name}.0001.vrscene, it should be rendering sucessfully. See the simple test below. |
Right... I had to use "0" padding on the input file path as that's what Arnold and Mantra Deadline plugins used as pattern to replace with the input frame of the task, I would need to look into the VRay plugin to check what it uses instead |
Looking into the Houdini deadline submitter plugin, I see that for VRay it does this other logic: elif exportType == "Vray":
exportFileName = ifdFile
if export_will_overwrite( node, jobProperties ):
exportFileName = hou.expandString(".$F4").join(os.path.splitext(ifdFile))
fileHandle.write( "InputFilename=%s\n" % exportFileName )
fileHandle.write( "CommandLineOptions=%s\n" % jobProperties.get( "vrayarguments", "" ) )
fileHandle.write( "Threads=%s\n" % jobProperties.get( "vraythreads", 0 ) )
# Check whether the .vrscene file names are different
SeparateFilesPerFrame = ( not single_export_file( node ) ) or export_will_overwrite( node, jobProperties )
fileHandle.write( "SeparateFilesPerFrame=%s\n" % SeparateFilesPerFrame )I think I might need to add |
@moonyuet could you try again please? |
|
it renders out the vrscene as expected.
but the submit publish job doesn't publish the vrscene. |
Awesome! Yeah I wouldn't expect any of the "translate" render scene files to be published, only the final renders, that's the usual workflow on the pipelines that I have worked on. The .vrscene, .ass, .rib... files are only used as an intermediate data to create the render but they are kind of useless after. Only on the cases where the render is faulty you'd want to check those scene files again for debugging and try re-render that same file... but our default behavior was to actually delete the intermediate render scene files as soon as the render is done to save storage so maybe we would want to add those to the explicit delete plugin but I don't see any benefit on publishing the render scenes, do you? |
make sense though, it is kinda useless after rendering. If we want to just publish the vrscene, we can just make the publisher to do this. |
👍 It's the same here. Being able to disable cleanup of them could be nice for debugging. But it's also useful for e.g. re-rendering if a certain frame did something weird on a particular machine! So we'd probably clean up later rather than directly. |
|
i have tried mantra ifd files but it seems that it only supports Houdini 18.5 or lower. so when I used 19.0 or 19.5, it fails to find the render plugin and errors out in deadline. The log below: I tried to submit manually to deadline too. I find out that the "SceneFile" probably need to be similar to the way set in "InputFilename" in vray job submission |
Yeah that is because the vanilla installation of Deadline doesn't have the executables for Houdini/Mantra 19.0 and 19.5, you have to go to the Mantra.param file of the ./plugins/Mantra folder of your repo and create the entries for those.
I think that separation of frames isn't a thing in Mantra as it is in VRay, I tested it on my side and the split of export + render for Mantra renders are working as expected |
|
I'm curious to check this out when its merged with develop branch and stable. ### Thank You @fabiaserra @moonyuet @BigRoy @MustafaJafar |
|
Hi @antirotor , you reviewed this PR #4903 last week, thank you! Could you look at this one when you have some time. |
|
I'm sorry for being late.. but this PR needs some follow up PRs to fix some issues.. This PR only works when I have these two options enabled If
Also, it broke many product types as
|
Changelog Description
This adds initial support in Houdini so when submitting render jobs to Deadline it's not running as a single Houdini task but rather it gets split in two different tasks: Export + Render. This way it's more efficient as we only need a Houdini license during the export step and the render tasks can run exclusively with a render license. Moreover, we aren't wasting all the overhead time of opening the render scene in Houdini for every frame.
I have also added the corresponding settings json files so we can set some of the default values for the Houdini deadline submitter.
Testing notes:
Split export and render jobs(set to default for now) andSubmitting to farmare enabled