wg-usd - Improvements to Publisher

[BigRoy]

Issue

The configuration needed for an instance that contributes to a USD Asset or Shot can have quite a few influencing settings that may be quite confusing to the artist without a better visual guide in the UI as to the attributes state and what they actually end up doing.

As an example, when publish a USD file you can contribute it to another product (usually the asset or the shot) as a sublayer or a reference within a variant. If a USD reference then all your contents must exist in one group and it's recommended to define the default prim of the layer. Then, these sublayers or references are 'ordered' so that one contribution can be stronger than another. Usually a task would come with a predefined order, e.g. layout before animation, animation before lighting, etc. But if there are multiple animation tasks, what would be their ordering? Usually a Shot contribution would be even done TO a layer of the shot. Like contributing the character_hero animation to the animation layer of the shot. It can all easily become tricky to grasp.

Furthermore, by its design of USD, it's fundamental that things are placed in the correct position in the hierarchy since all authored opinions are path based - making it even more crucial that it's trivial to understand for an artist where something will end up what they are publishing.

Here's an example of a current state of a USD instance in Houdini:

And here's the same for a Maya USD instance:

And just as extra reference - here's our state for the render instances in Maya:

Note that I believe in develop currently the Deadline Pools and Machine Lists are text fields instead of dropdown menu and enums - that is a local customization that we're using

Here I'd like to raise some pointers on things that could improve the User Experience in the Publisher UI with the intent of streamlining the implementation needs for the USD tooling.

Notes on Publisher UI or workflow

Unable to disable attribute based on another attribute (e.g. enabled only if another boolean checkbox is on)

This is a major one - you currently cannot visually disable or hide attributes based on the state of others. I'm not sure what the best design is to allow this to be implemented without actually coding attribute access to one another. But Houdini for example has some custom 'expressions' you can set for "Disable when" and "Hide when" where you can basically check the value of another parm, if it's true, then it hides or disables the attribute. It's pretty clean and makes sense as well.

Potentially custom 'callback' scripts with access to the current attributes could make this much more powerful for very specific cases.

Buttons (or custom widgets)

This might be the easiest way to 'for now' step outside of the Publisher UI and implement whatever custom thing you might need for a specific case, e.g. "picking the position in the USD asset" could have a button, when clicked pops up a tree view of the latest asset USD file so that the user could pick the new state. Then on accept it could update values on the other attributes in the UI.

Basically on clicking the button a callback script would trigger that has access to the selected instances (for which the attribute is valid), access to all its current values and attribute definitions, and can basically update some values so at the end of the callback it can return e.g. what changed, or what should be refreshed, etc.

This could be used for USD - but also for other things that might be hard to 'pre-load' in the UI, like e.g. "Pick pools" or "Pick Machine Allow/Deny List" for Deadline where the pools available might be dependent on another attribute like the chosen deadline server. Or like "Pick colorspace from OCIO config" to only compute the colorspaces if the user clicked that instead of always on reset (it can be slow?)

Having this can help write e.g. a pop-up tool where you can easily re-order multiple instances from your scene and _maybe even! manipulate other layers in the next asset USD file you'll be publishing. It'd at least allow you to visualize where things end up, the order of contributions, etc.

This can easily become a very powerful feature.

Hard to distinguish what checkbox belongs to ‘what’

When an instance ends up having quite a few optional things to set or custom attributes it easily becomes hard to really track what attribute belongs to what output (e.g. abc, usd) of the instance or what plug-in it's really related to.

You can add headers and separators to kind of help with that - but that can be quite tricky across separate plug-ins since they are ordered by the "order" of the plug-ins and are not grouped by anything else. It could maybe be nice to say, group these with "USD" so that any attribute with that name would be grouped together - no matter the source plug-in.

More useful 'defaults' (see which are non-default, and allow to revert to default)

Currently in the UI you cannot easily see whether an attribute is on their default value or not. I'd want to recommend a similar feature as Houdini offers where any attribute not on its default value would appear bold in the UI.

Similarly having a right-click “revert to defaults” feature for an attribute would be perfect. Like, Houdini, we could also set a CTRL + Middle Mouse Button shortcut that reverts it to the default as well so you can quickly click a few attributes to revert to defaults.

We could even have a toggle in the UI to e.g. "show only non-default attributes" or other search/filtering like that. That'd only really be useful if the amount of attributes would really grow more substantially (preferably we can simplify instead hehe) so can be overkill to design with currently.

Create tab > on create reverts back to defaults after creating an instance

This is more of an annoyance. But if on the Create tab you're looking to create a few instances and you have toggled a few of the Create options as 'settings' you want to use for creating the instances you're in bad luck. Because currently, on "Create" the pre create attributes reset back to their defaults.

Preferably these would remain as they were, unless you click "Reset" in the UI.

Tooltips

Visually I think we can improve tooltips by design - a part of that is for debuggability for developers.

I think, similar to Houdini it would be great if it could incorporate just more technical details like the key of the attribute. And maybe even which "plug-in" defined it so it may be, by hovering, a bit more clearer what it relates to.

Having a tooltip possible be expendable to a more thorough "Help" would be very nice as well.

Houdini also allows defining a "Help" per attribute, which acts like a the tooltip. But based on that it can automatically generate a "F1" help page for the node/instance you're operating on. I think this could be very nice actually.

For example, when clicking the top right question mark in the UI we could show a browser or markdown rendered webpage that could e.g. list the Instance's main help from the Creator, but also list a section with all the configurable parameters with their tooltips. So that instead of needing to go and hover over each attribute you can read through all the bells and whistles that you can configure. This can be extra nice, because you can then CTRL+F over the text to search for a particular thing among the attributes / docs that you'd expect for that particular instance.

Actions on 'warning' or 'success'

Pyblish itself allows defining plug-in actions on warnings or success or 'always' if the plug-in has run. This can be tremendously useful to implement something after publishing to e.g. "Play the review" file that was published directly. Or "Show in Ftrack" the new publish so you can continue there with it. Or "Show job in Deadline" after a farm submission. Or even something like implementing a "Copy to clipboard" could be an interesting global action on successful publish.

This is also noted on Ynput Discord here.

The warning one can also be nice so that if a warning has occurred that you can still look into it easily after the publish finished.

[MustafaJafar]

So, USD API allows us to modify USD files without applying these modifications in our scenes.
For example I can have my tea pot in a maya scene and I want to publish it as a new variant for tea pot asset.
for a classic workflow, I'll need to import the tea pot asset and add my variant then publish the asset.
unlike what you have mentioned above. I can just have my new tea pot and ask the publisher to add as a variant to my tea pot asset.

but, in the latter situation we fall in a problem where artists lack proper visualization tools where they can be 100% of the results of the publisher.

Also, we can embed the publish settings in that visualization tool so artists can interactively check the results for different configurations.

I'm just trying to analyze the requirements so we can have better planning/designing.

---

I can imagine something very basic like this showing before and after with quick actions.

Or even it can be a full USD editor like usd qtpy or this one (it looks interesting).

I wish if I could mention fabia, he used to work for MPC/Mill and I think he can share us some helpful thoughts.
Millefiori: A USD-Based Sequence Editor

[BigRoy]

So, USD API allows us to modify USD files without applying these modifications in our scenes.
For example I can have my tea pot in a maya scene and I want to publish it as a new variant for tea pot asset.
for a classic workflow, I'll need to import the tea pot asset and add my variant then publish the asset.
unlike what you have mentioned above. I can just have my new tea pot and ask the publisher to add as a variant to my tea pot asset.

Yes, exactly. We're basically using the USD Python API to create automatically what otherwise a user would have to do manually by hand (and likely every now and then will make an error with as well). So we basically automate the logic.

This ensures constant stability - no human error in that area.
This ensures it's faster - less steps for the user to go through.
This ensures simplicity - user has to know less about how it works.

This last point can be considered a huge downside - the fact that now suddenly we expose these tooling from the publisher they might not as quickly get familiar with toolings in DCCs to do this manually or be aware of how USD itself works internally. At the same time, I personally think that's totally fine. The same goes for Alembic, rarely does any artist really understand how the data is being stored in the file - but they do get somewhat familiar over time with how it behaves in production. They gain knowledge and learn to "expect" certain things of it - they don't have that yet for USD so it can be quite scary for newcomers; best to simplify as much as possible and where we can adhere to familiarity - making it behave 'similar' to concepts they do know.

but, in the latter situation we fall in a problem where artists lack proper visualization tools where they can be 100% of the results of the publisher.

That, but also aside of visualizing - being able to understand what it actually means to have a variant, what it means to have either a variant or sublayer - of which they might lack understanding. The USD terminology and workflows being unfamiliar makes it extra important that they can rapidly visualize since they are currently unable to rely on existing knowledge to expect things to work a certain way.

Also, we can embed the publish settings in that visualization tool so artists can interactively check the results for different configurations.

Hm, well - yes. Sort of. It does get a bit trickier - because you want the user to be able to customize the output BEFORE publishing, that is - before the output data actually exists. As such, the visualization of that output will be limited by the fact that the actual USD file does not exist. So the visualization would be mostly 'hierarchy' based like: "Your model will appear in the hierarchy here." instead of just seeing your 3D scene.

It's mostly:

• You are publishing a model of a vase, a tall and regular one. You want each to be the main "model" variant of the asset with one called "tall" and the other "regular". This we can visualize in the hierarchy, that after the publishing finishes - the asset will then have 2 variants - and it can also display which would be the default variant, etc.
• You are publishing a model of a hat, which is just a part of the character. Where do you want it to end up? Where should the 'variant' toggle be for this? You likely don't want to toggle the GLOBAL variant of the model since it's just a headgear toggle.

[BigRoy]

@iLLiCiTiT any chance you could comment on this based on the Publisher change requests?

I'm looking to maybe get the ButtonDef working with access to the other attribute definitions in the callback, but unfortunately there's no easy access from the attribute defs or the widgets towards all the attribute definition values to e.g. tweak other values from the callback. My prototype stranded a bit there unfortunately.

I'd love to e.g be able to either access the other attribute definitions, their attribute widgets or just get a dict of attribute definiton keys and their current values - and from the callback be able to return a dict with keys and new values and the new values would then get applied to the other attribute definitions.

I had e.g. this:

class ButtonAttrWidget(_BaseAttrDefWidget):
    def _ui_init(self):
        button = QtWidgets.QPushButton(self.attr_def.label, parent=self)
        button.clicked.connect(self._emit_callback)

        if self.attr_def.tooltip:
            button.setToolTip(self.attr_def.tooltip)

        self._button_widget = button

        self.main_layout.addWidget(button, 0)
        self.main_layout.addStretch(1)

    def _emit_callback(self):
        parent_widget = self.parent()
        assert isinstance(parent_widget, AttributeDefinitionsWidget)

        current = parent_widget.current_value()

        result = self._callback(current)
        if result:
            assert isinstance(result, dict), \
                "Callback result must be a dictionary"
            parent_widget.set_value(result)

But unfortunately the parent in the Publisher UI wasn't a AttributeDefinitionsWidget and thus I couldn't access the values.

Anyway, let me know what options there might be to improving the UX short-term.

[iLLiCiTiT]

Anyway, let me know what options there might be to improving the UX short-term.

The main idea of current publisher implementation is that UI does not do anything backend related. Communicates with controller, and all arguments and return values in that communication can be converted to json, so we can separate UI and backend to different processes. In other words UI should never trigger anything host related directly.

So I don't think there is any short-term solution at this moment.

[BigRoy]

The main idea of current publisher implementation is that UI does not do anything backend related. Communicates with controller, and all arguments and return values in that communication can be converted to json, so we can separate UI and backend to different processes. In other words UI should never trigger anything host related directly.

I'm not looking to perform anything host-related. I just want to tweak the values of the attribute definitions based on the one attribute definition's callback.

If there's a huge reason this is never possible then there's a huge issue with the "main idea" that's present if that blocks the issues described in this issue. It should really be possible to enable/disable attributes based on other states and ALSO to allow some sort of tools to manipulate the instances nicely for more complex cases, e.g. so that a Deadline Pool doesn't have to be a string value but can actually be selected from the available pools on the selected deadline server - or more necessary for my use case, can simplify and help visualize what the settings to do the publish USD contribution.

I understand that "it's currently not possible." My question really is, in what way do you think we can solve the issues I've decribed in this issue. How can we reach the goal? I'm totally fine if it's not exactly the way I said we should fix it, as long as we're able to solve the issue itself.

If I know what SHOULD be done to solve this - then I can look into it. With "it's not possible" this just means it remains a you problem and I'll have to wait for you to have time to look into it and solve it yourself which I'm hoping doesn't necessarily have to be you coding it.

[iLLiCiTiT]

If I know what SHOULD be done to solve this - then I can look into it. With "it's not possible" this just means it remains a you problem and I'll have to wait for you to have time to look into it and solve it yourself which I'm hoping doesn't necessarily have to be you coding it.

I didn't say it's not possible, I said there is no short-term solution. I think this would require to change how we're approaching attribute definitions in CreateContext, which is not short-term solution...

And at the same time it's not something what a single person should handle based on single request. It requires to collect feedbacks from multiple people. Gather what we need to change, what is "must have", what is "nice to have", then discuss how/if it can be achieved, what are downsides and benefits. Based on the discussion come to a conclusion, which may be that we completelly change the implementation of attribute definitions, or we won't try to separate UI from backend, and keep logic of attributes definition only for UI, or won't allow multiselection of instances in UI, or something else. I can't answer because there is so much variables I don't know about.

[BigRoy]

Thanks - as part of the USD workgroup calls it has come up that there's definitely need for change with regards to the Publisher UX. It's not just my sole opinion and steering being involved.

But totally agree we should 'align' where to go next with the Publisher considering my inputs above but hopefully even more. The question becomes - how do we now collect the required input and then act on it? What's needed to get the "discussion" going aside of creating this issue - which is in essence, a bit of that already?

[iLLiCiTiT]

You asked for solution. But I think at this moment we need a set of use-cases, without any solutions or suggestions. Solutions and suggestions are this moment blocking, especially because solution for single use-case usually does not count on full picture.

For example:

• Attribude definitions per instance, and collect them per create plugin and publish plugin would have custom own filtering mechanism for instances.
  • How to handle multiselection?
• Dynamic change of visible attributes definitions, or completelly change attribute definitions based on attribute definition value.
  • How to handle then in headless mode?
  • Should each attribute value change trigger re-collection of all attribute definitions and re-create all widgets?
  • How to define them?

Both are possible, but require sacrifices, and are against other features we have now.

I can continue:

• Have a button that would trigger action in host.
  • Should that action happen for selected instance/s?
• Capture unsaved changes during UI session.
• Allow instance multiselection.
• ...