[MAYA-127758] Setting visibility does not consistently route edits

[gracekumagai]

Describe the bug
When working on #2546, @jallex and I noticed that after we had registered an edit router to route visibility edits to the session layer, setting visibility was not consistently set in the session layer.

Expected behavior
We would expect to have visibility routed to the session layer (our routing layer) after we've registered the edit router.

From chatting with our artists, their main workflow for setting visibility is to use Ctrl+h and Shift+h hotkeys. On our side, the Display menu actions and corresponding hotkeys would be the top priority to fix.

Fwiw - they also heavily rely on using Display Layers to toggle visibility.

Steps to reproduce
We tested with this PR #2546. Then, we created a stage and using the context menu in the outliner, added prims to test with.

Ctrl + h hotkey | Display -> Hide -> Hide Selection menu item | cmds.hide()

If we select a new prim + use action above, it successfully routes visibility edits to the session layer!

If we multi-select an invisible prim (where visibility is authored on the session layer) + a visible prim (where the prim does not have visibility explicitly authored yet), it will hide both and route invisibility to the session layer. However, it also prints the following error:

// Error: Cannot edit [visibility] attribute because there is a stronger opinion in [anonymousLayer1-session.usda].

Shift + H hotkey | Ctrl + Shift + H hotkey | Display -> Show -> Show Selection or Show Last Hidden menu items | cmds.showHidden()

If we select an invis prim (where visibility is set on the session layer) + use action above, it does not make the prim invisible and prints the same "stronger opinion" error as above. We would expect visibility to be updated in the session layer and for the prims to be visible.

If we select an invis prim (where visibility is set on the target layer (incorrectly)), it sets visibility=inherited on the session layer and visibility=invisible remains on the target layer.

H hotkey | set visibility using pxr.Usd.Attribute

To test setting the visibility using an pxr.Usd.Attribute, we used the following code:

prim = mayaUsd.ufe.ufePathToPrim(ufe.PathString.string(sceneItem.path()))
visibilityAttr = prim.GetAttribute('visibility')
visibilityAttr.Set(UsdGeom.Tokens.visible) # or UsdGeom.Tokens.invisible

If we select a prim + use action above, it always routes visibility (invis and vis) to the target layer. We would expect visibility to be routed to the session layer.
If try to vis a prim (where the visibility edit is authored to the session layer) by using the action above, it does not change the visibility in the viewport and does not return the "stronger opinion" error in the script editor.

cmds.setAttr() | ufe.PyUfe.AttributeEnnumString

To test setting the visibility using an ufe.PyUfe.AttributeEnnumString, we used the following code:

attrs = ufe.Attributes.attributes(sceneItem)
visibilityAttr = attrs.attribute(UsdGeom.Tokens.visibility)
visibilityAttr.set(UsdGeom.Tokens.visible) # or UsdGeom.Tokens.invisible

To test setting the visibility using cmds.setAttr(), we used the following code

cmds.setAttr(pathString, UsdGeom.Tokens.visible) # or UsdGeom.Tokens.invisible

It always routes visibility (invis and vis) to the target layer. We would expect visibility to be routed to the session layer.

Except, if trying to vis a prim (where the visibility edit is authored to the session layer), prints the same "stronger opinion" error as above and does not make prim visible.

ufe.Object3d.object3d

To test setting the visibility using an ufe.Object3d.object3d, we used the following code

object3d = ufe.Object3d.object3d(sceneItem)
visibleCmd = object3d.setVisibleCmd(True) # or False
visibleCmd.execute()

If we try to invis a prim, it routes to the session layer as expected!

If we try to vis a prim (where the visibility edit is authored on the session layer), it prints the same "stronger opinion" error as above and does not make prim visible.
If we try to vis a prim (where there are not any visibility edits), it does not author any information.

Specs (if applicable):

• Maya version 2023.1
• Maya USD commit SHA [dev at 023a7d8] + PR Route visibility edits and run clang-format #2546
• Pixar USD commit SHA [dev at 8dcb0c2]
  Pixar Jira: MAYA-3451

[ppt-adsk]

The session layer can be set as the target for visibility edit routing using the following Python code:

import mayaUsd

def getSessionLayer(context, routingData):
    prim = context.get('prim')
    if prim is None:
        print('Prim not in context')
        return
        
    routingData['layer'] = prim.GetStage().GetSessionLayer().identifier
    
mayaUsd.lib.registerEditRouter('visibility', getSessionLayer)

[santosd]

@gracekumagai, Wanted to check back on this and see if the above comment from ppt-adsk was helpful in resolving the issue on your end?

[santosd]

Hi @gracekumagai, happy new year! Just wanted to check back and see if this can be closed out or if you had any further questions.

[santosg87]

closing this one for now, based on the convo above.

@gracekumagai feel free to re-open if the offered solution isn't helpful :)

[gracekumagai]

Hey @santosd & @santosg87, apologies for the delay in testing!

When reporting the initial issues, I used a very similar script mentioned in @ppt-adsk comment.

I've retested everything and it looks like Ctrl+h and Shift+h (and setting visibility using object3d.setVisibleCmd) work as expected! However, I still ran into issues with the other methods for setting visibility. In terms of priority, artist facing actions are higher priority.

For steps to reproduce, I used the Kitchen_set and ran the script in the comment above before performing the following actions.

---

Ctrl + h hotkey | Display -> Hide -> Hide Selection menu item | cmds.hide()

Works as expected with 1 or multiple objects selected. No longer prints an error!

Shift + H hotkey | Ctrl + Shift + H hotkey | Display -> Show -> Show Selection or Show Last Hidden menu items | cmds.showHidden()

Works as expected with 1 or multiple objects selected!

H hotkey

If we select a prim and use the H hotkey, it always routes visibility (invis and vis) to the target layer. We would expect visibility to be routed to the session layer.

This leads to a strange behaviour when used in a combination with Ctrl+h and Shfit+h hotkeys (and other actions mentioned above.

For example,

1. If we select a prim,
2. press Ctrl+h to hide the prim
3. press Ctrl+Shift+h to unhide the prim. It writes an over to the session layer where the visibility is set to "inherited"
4. re-select the prim
5. press H. It does not hide the prim and this edit is written to the target layer

We would expect pressing H to route edits to the session layer and to be used before/after Ctrl+h and Shift+h hiding actions.

Set visibility using pxr.Usd.Attribute

If we select a prim + use code below, it always routes visibility (invis and vis) to the target layer. We would expect visibility to be routed to the session layer.

To test setting the visibility using an pxr.Usd.Attribute, we selected a prim and ran the following code:

for sceneItem in ufe.GlobalSelection.get():
  prim = mayaUsd.ufe.ufePathToPrim(ufe.PathString.string(sceneItem.path()))
  visibilityAttr = prim.GetAttribute('visibility')
  visibilityAttr.Set(UsdGeom.Tokens.visible) # or UsdGeom.Tokens.invisible

Similar to the H hotkey, we run into the same issue when used in combination with Ctrl+h and Shift+h

ufe.PyUfe.AttributeEnnumString

If we select a prim + use code below, it always routes visibility (invis and vis) to the target layer. We would expect visibility to be routed to the session layer.

To test setting the visibility using an ufe.PyUfe.AttributeEnnumString, we used the following code:

for sceneItem in ufe.GlobalSelection.get():
  attrs = ufe.Attributes.attributes(sceneItem)
  visibilityAttr = attrs.attribute(UsdGeom.Tokens.visibility)
  visibilityAttr.set(UsdGeom.Tokens.visible) # or UsdGeom.Tokens.invisible

This leads to a strange behaviour when used in a combination with Ctrl+h and Shfit+h hotkeys

For example,

1. If we select a prim,
2. press Ctrl+h to hide the prim
3. press Ctrl+Shift+h to unhide the prim. It writes an over to the session layer where the visibility is set to "inherited"
4. re-select the prim
5. Run code above. It does not hide the prim and prints this error:

# Error: RuntimeError: file <maya console> line 10: Cannot edit [visibility] attribute because there is a stronger opinion in [Kitchen_set-session.usda]. 
# setAttr: Error reading data element number 1: invisible

We would expect cmds.setAttr() to set visibility to route edits to the session layer and to be used before/after Ctrl+h and Shift+h hiding actions.

cmds.setAttr()

If we select a prim + use code below, it always routes visibility (invis and vis) to the target layer. We would expect visibility to be routed to the session layer.

To test setting the visibility using cmds.setAttr(), we used the following code

for sceneItem in ufe.GlobalSelection.get():
    pathString = ufe.PathString.string(sceneItem.path())
    cmds.setAttr('%s.visibility' % pathString, UsdGeom.Tokens.invisible) # or UsdGeom.Tokens.visible

Similar to ufe.PyUfe.AttributeEnnumString, this leads to a strange behaviour when used in a combination with Ctrl+h and Shfit+h hotkeys

ufe.Object3d.object3d

Works as expected with 1 or multiple objects selected.

I ran the following code to test:

for sceneItem in ufe.GlobalSelection.get():
    object3d = ufe.Object3d.object3d(sceneItem)
    visibleCmd = object3d.setVisibleCmd(False) # or True
    visibleCmd.execute()

---

I'm testing with:

• maya2023.2
• maya-usd commit sha (dev at abf2928)

[santosd]

Hi @gracekumagai thank you for the response on this one, I will take a look and if need be log an issue for the remaining workflows.

[santosd]

Hi @gracekumagai, sorry for missing the full context of the initial post. I do see that certain commands for editing the visibility don't get routed correctly. I have created an internal ticket for us to take a look at MAYA-127758.

[gracekumagai]

Hey @santosd, no problem and thank you for looking into it! I'm happy to have had the chance to go back in since a couple of the issues have been resolved 🎉

[dj-mcg]

11/20/23 Update:

• Ctrl+H and Shift+H routed as expected
• H still routes to current layer. Session layer opinions will override and will not report issues re: stronger layers
• cmds.setAttr routes to current layer. Session layer opinions override and WILL report stronger layer warnings
• object3d.setVisibility routes to current layer. No errors about stronger opinions
• object3d.setVisibleCmd routes as expected