Publishing: Low-level dynamic runtime instance creator

[BigRoy]

Changelog Description

This PR implements an approach to "lower level publishing" (tested in Houdini).
This focuses on PDG/TOPs or any other type of batch creating runtime instances.

• TOPs/PDG example on accessing AYON api for folders, tasks, workfiles and publish the workfiles in background houdini processes (similar to legacy/old style publishing of workfiles)
• TOPs/PDG example on low level ingesting of publishes using an in-memory dynamic Creator.

Note that this is Houdini-tested but the "lower level dynamic" publishing is logic that is not Houdini only.

This PR adds a Batch/Dynamic Creator:

Currently, it creates runtime instances (they will disappear once you refresh the publisher)
It doesn't inject any metadata. but it expects you to pass them manually.
Here is an example
https://gist.github.com/MustafaJafar/bd2a388e4a6aa3613d64a186ebb6660c

Additional info

As part of the Ynput Houdini Workgroup sessions I developed this quick prototype to expose a way to batch publish and ingest files. Consider it more of an exploration of what's possible then a "drop it in production now" ready-to-go solution.

TODO

• Keep in mind this set of goals
• Move core of logic out of Houdini
• Expose a simple API with examples
• Expose examples
• This PR should solve the issues here

Explainer

Since this PR is separated from #542 some of the logic in the demo provided there is now actually here - in particular to the logic the Houdini TOPs / PDG demos show.

2024-05-22.20-48-11.mov

Demo scene file

The demo scene file:

ynts_char_hero_pdg_v012.zip

Testing notes:

1. Check out the branch
2. Check out the explainer video
3. Test the demo scene file
4. Comment with great ideas on how to improve

[moonyuet]

I tried to create instance with ingest
And it errors out during creating instance where the product type can't be found.

Traceback (most recent call last):
  File "D:\ayon-core\client\ayon_core\pipeline\create\context.py", line 2124, in _create_with_unified_error
    result = creator.create(*args, **kwargs)
  File "D:\ayon-core\server_addon\houdini\client\ayon_houdini\plugins\create\create_dynamic.py", line 53, in create
    product_type = pre_create_data.get("product_type") or instance_data["product_type"]
KeyError: 'product_type'