MaterialX NodeDefs Missing

[DanGroverUK]

Hi! I'm fairly new to this, so apologies if any of this is obvious, but I have a little problem...

Maya: 2022.4
maya_usd: 0.20.0 Installed from the released Windows binary
Houdini 19.0.720

I have a Houdini LOP that's written out a USD that contains MaterialX shaders and uses a few of the Houdini-specific nodes that ship with Houdini - namely huniformramp.mtlx and hinvlinear.mtlx. If you load this into Maya 2022.4 with maya_usd-0.20.0 as installed from the Windows binary, it opens fine but immediately crashes if you turn Textures on in the viewport. If you open it in Maya 2022.4 with the standard maya_usd-0.13.0, it loads OK and shows a simply usdPreviewSurface. These Houdini-sourced materialx definitions seem to just contain standard materialx nodes (including ifgreatereq which I understand is somewhat new, but was introduced before the MaterialX-1.38 that ships with maya_usd-0.20.0).

If I add these handful of Houdini-specific material definitions to Maya's MaterialX library (as set in the mayaUsd.mod file) in maya_usd-0.20.0, it no longer crashes but nor does it display anything all that useful - just a grey shader. There are also quite a few errors in the Output window that make me think not everything is quite hunky dory. When it's first loaded, I get...

// Error: Failed verification: ' nodeDef ' -- Could not find MaterialX NodeDef for Node 'NG_Maya/N4'. Please recheck library paths. // // Error: Failed verification: ' nodeDef ' -- Could not find MaterialX NodeDef for Node 'NG_Maya/N4'. Please recheck library paths. // // Error: Caught exception 'Could not find a nodedef for node 'N4'' while processing '/Proxy_vegGrasses_USD_layer_asset_v033Shape_000001DF3222A7F0/vegGrasses/mat/leafGreenCMAT' //

I'm not sure how to work out what NG_Maya/N4 is defined as - I assume it represents a NodeGraph, and I assume it's a name given during a translation process, but as far as I can tell the required node definitions for this material are all where they should be, but this can't be the case or I'd not be getting the error!

In addition to that, If I select the material from the USD Stage in the Outliner, I get...

`# Error: Failed verification: ' item ' #

Error: file: C:/Program Files/Autodesk/Maya2022/scripts/others/showEditor.mel line 3703:RuntimeError: file C:\local_pipe\rez\cache\maya_usd\0.20.0-a\24fa\a\mayausd\MayaUSD3\lib\python\ufe_ae\usd\nodes\usdschemabase\ae_template.py line 338: Invalid attributes object # `

Line 338 of ae_template.py appears to relate to attribute connections (and, indeed, these connections appear valid and visible if the USD is loaded in maya_usd-0.13.0).

If I drill down into the material's shaders, the aforementioned Houdini-specific nodes appear to show their inputs all correctly, but the NodeGraph that they're contained within doesn't show any attributes (and I get a similar error to above) - where as in maya_usd-0.13.0, this node displays two attributes with connections to the shaders within.

I suspect these issues all have a common cause, namely not being able to find a required node definition. This is causing it to not show correctly in the viewport, not create the attribute connections and ultimately not work properly.

Does anyone know if I'm doing anything obviously wrong, or if they've experienced this before? Or, perhaps more helpfully (in a 'teach a man to fish' way), does anyone know how I might find out exactly what NG_Maya/N4 is so that I can hopefully answer this for myself?!

(And as an aside, I was getting quite a few complaints in the Output window about not being able to find an RGB color with the name 'lead' or 'polymeshDormant' - simply adding these using Maya command displayRGBColor makes the error go away, but it could be a relevant diagnostic symptom, as my fairly uneducated glance at the code seems to suggest that prior to 2023, Maya should fall back to supplying a preset value defined in the C++ rather than just wigging out like this?)

Thanks and sorry for rambling!

Edit: I just saw I'm not alone in experiencing this, though the feature request here is intended as a way around the issue rather than an actual fix:

#2637

[DanGroverUK]

Having done a little more reading of Issues, as well as re-reading the 'Viewport' section of the current MaterialX readme page, I think that what I'm trying to do here isn't currently possible - the docs state that "We are using the hdMtlx translation framework, which is also in use for hdStorm and hdPrman, and using the same GLSL code generator as hdStorm, so valid USD MaterialX material graphs should display correctly" and I'm now interpreting the "valud USD MaterialX material graphs" as those using the standard material libraries only. Would this be correct, does anyone know?

[santosd]

@JGamache-autodesk, with your knowledge in MaterialX could you confirm this?

[JGamache-autodesk]

This should definitely be possible. Both USD and MayaUSD will search for MaterialX libraries by following the PXR_MTLX_STDLIB_SEARCH_PATHS environment variable.

Now there are two options:

1- Houdini kept its own definitions in a very separate path from the MaterialX libraries it ships with. You only need to add that path to the env var.
2- Houdini has merged its definitions with the ones shipped with MaterialX. Not nice since there could be conflicts if the MaterialX versions are not fully aligned between Houdini and MayaUSD. You will need to extract the Houdini files into a separate path and add that path to the env var. Sometimes the implementation NodeGraphs are kept in seperate files: be sure these are also found in the path you are adding.

Technically, you should be able to do the reverse and add the MayaUSD libraries to the Houdini env var for library paths and be able to use a LdkColorCorrect node from MayaUSD/libraries/adsk/maya/lookdevKit.mtlx from MayaUSD.

Enable the following environment variable to help debugging: TF_DEBUG=HDVP2_DEBUG_MATERIAL. This will dump the MaterialX document and the USD shader graphs into the output window and should allow you to find out what is the node type of NG_Maya/N4 and where it comes from.

In the Maya script editor, Python tab, you can very quickly find if the MaterialX nodes were read. In this example we look to see if the MayaUSD libraries were loaded by looking for the color correct node:

from pxr import Sdr
r = Sdr.Registry()
n = r.GetShaderNodeByIdentifier("LdkND_ColorCorrect_color4")
print(n.GetInputNames())

[DanGroverUK]

Thanks very much for the information!

As a troubleshooting step, I decided to wholesale replace the maya-usd 'USD3/library' folder with the contents from Houdini's (that lives on the same PXR_MTLX_STDLIB_SEARCH_PATHS env var). In fact, Houdini only has the one path, which possibly explains why it has a 'houdini' folder in there that's not part of the actual standard library.

Anyway, no dice - I essentially get the same error. Houdini actually has a different set of environment variables, though, and it's changed between v19 and v19.5 of Houdini - I did try emulating these just as a hail mary but, unsurprisingly, it broke basically everything so I reverted pretty quickly...

Anyway, the TF_DEBUG env var sounds very intruiging but I think I must be doing something wrong because it doesn't seem to actually print anything. If I run...

from pxr import Tf Tf.Debug.IsDebugSymbolNameEnabled("HDVP2_DEBUG_MATERIAL")

It prints True, and I can turn it (and any other Debug Symbols) on and off with Tf.Debug.SetDebugSymbolsByName("HDVP2*", 1) but it doesn't seem to actually print anywhere. I've tried setting the output file to both sys.__stderr__ and sys.__stdout__ (as well as leaving it unchanged) but no dice. I think this would be very, very useful, though, so I can see for sure what it's not able to read!

[JGamache-autodesk]

On Windows, you need to open the output window, which can probably be found under the menu bar entry Windows -> Output window.

[DanGroverUK]

D'oh - thanks for that! That's what I get for starting out as a Max user...

Anyway, that's very helpful, and weirdly I've found... well, it has nothing to do with the Houdini MaterialX definitions anyway. It seems to be very unusual, what gets thrown up as the problem nodes, including in materials entirely using standard nodedefs and nodegraphs.

This got me thinking that perhaps the errors listed when trying to view Extra Attributes on materials and their shaders weren't symptoms but actually the cause. It seems to happen just on shader's that have connected attributes, complaining on line 338 of a RuntimeError: Invalid attributes object. However, that's just in a bit of code about applying pretty names; If I set the optionVar attrEditorIsLongName to 0, it eschews this and successfully links the connections as expected.

Now, this may merely be a visual thing that only occurs when you select an option with the 'Extra Attributes' rollout extended but as part of an abortive attempt to troubleshoot I noticed that line 50 here was failing, arguing that the MaterialBindingAPI object doesn't have a 'GetMaterialPurposes' method. I haven't found a material yet that does, though whilst I must admit I'm a little out of my depth in the office USD API docs, it's certainly still present. Nonetheless, this has now got me wondering if there is something related to the attributes being read or connected that's throwing the NG_X NodeDef errors when the materials are first translated on load.

I'll update again when I find out anything else.

[daviddeacon]

@santosd @JGamache-autodesk Not to hijack the discussion, but these continuing issues do add some more weight behind my request to have an option to be able to disable MaterialX and just fall back to a UsdPreviewSurface. 🙏
Here is the link to the issue/enhancement request - #2637

[JGamache-autodesk]

I had a look at the scene itself. The USD viewer application has also issues with the N4 node, which is the normalmap node. usdview gives the following error:

Warning: in HdMtlxCreateMtlxDocumentFromHdMaterialNetworkInterface at line 558 of usd\pxr\imaging\hdMtlx\hdMtlx.cpp -- Validation warnings for generated MaterialX file.
Node interface error: Input 'in' doesn't match declaration: <normalmap name="mtlxnormalmap1" type="vector3" nodedef="ND_normalmap">

The issue is that this node is fed from:

        def Shader "mtlximage6"
        {
            uniform token info:id = "ND_image_color3"
            asset inputs:file = @./Textures/King Textures/King Black Textured/DefaultMaterial_Normal_OpenGL_1001.png@
            color3f outputs:out
        }

Which means a color3 output is fed into a vector3 input. Change it to:

        def Shader "mtlximage6"
        {
            uniform token info:id = "ND_image_vector3"
            asset inputs:file = @./Textures/King Textures/King Black Textured/DefaultMaterial_Normal_OpenGL_1001.png@
            vector3f outputs:out
        }

To make both USD and MayaUSD happy. As a bonus, Maya will not try to color correct a vector3 image, which might provide better results for your normal map.

[DanGroverUK]

To the extent that this could widely be an explanation for the error message when it's presented via a node that definitely does have a definition present, I wonder if there could be some under-the-hood wires plugged in the wrong way around somewhere?

I ask because I provided a very simple example in my Issue which is (I believe?) fairly unambiguously "correct" vis-a-vis input and output format - namely that if you plug a node's vector3f output into a vector3f input of a separate3 node (ND_separate3_vector3), it throws the above error.

In fact, if you switch the specific separate3 node to one that accepts a color3f (ie ND_separate3_color3) and plug in a node with a color3f output, the same issue arises. I haven't tried any of the other formats (ND_separate3_xxx), but if...

a) The error refers to missing node definitions but
b) The node definitions exist, and
c) the node definitions define the inputs and outputs as the same format and
d) In at least one instance (your findings above) both a and b have been true specifically because c wasn't true, then...

Could there be some issue in the translation process wherein the ASCII node definitions aren't being hooked up to the correct data formats?

[JGamache-autodesk]

I had a second look at the separate3 issue and found out it was fixed in USD 22.05. See #2717 for details.