Create material from textures

[Cinifreak]

This PR recreated after closing #1572 as the previous PR had some problems with force pushing and CLA.

[Michaelredaa]

Hey @kwokcb ,

I apologize for the delay, there were some issues with the old PR.

First and foremost, thank you for your suggestions. They mean a lot to me.

I've made several improvements to the code. Utilizing MaterialX modules like createValidChildName and FilePath has helped to enhance the code's quality. I've also extended the FilePath class to handle various aspects of texture files, such as UDIMs and patterns. I'd greatly appreciate your feedback on these changes.

Initially, this function was built for standard surface shading models, but it's a great idea to consider extending it to support other shading models in later stages of development.

I've also added a --tileimage flag that enables you to set the tile image

The same json configuration file can be further extended to support different DCCs and library conventions. It has been successfully tested in Substance Painter (feel free to test it :)) ). I'm planning to structure the json configuration file to incorporate the concept of inheritance and overrides, similar to houdini packages. This will enable us to override specific patterns by creating separate files as needed.

Thank you!

[kwokcb]

Hi @Cinifreak,

No need to apologize -- I only did a quick look with some possible suggestions :).

I will leave some thoughts (not work!) for other shader models since I'll probably forget about it later on :).

• OpenPBR is coming, but it's still early days so I assume a "moving target"
• Perhaps UsdPreviewSurface is of lesser concern though I can't say with any authority. As mentioned it's mostly due to it's "space" consideration for normal connections.
• glTF PBR peculiarities might be worth keeping in mind in terms of mapping logic: Here you can have a single texture which maps to N channels on the surface shader (in this case occlusion, roughness, metalness). I only raise this to check that this would not cause a lot of problems if desired in later development. A 1:N mapping with channel specific attributes on the shader inputs to allow unpacking the texture.

[kwokcb]

This looks great!

Some minor niggly items.

The only item I think would be nice to address is if the json file cannot be found then the script will halt prematurely.

[jstone-lucasfilm]

@Cinifreak @Michaelredaa @kwokcb I like the idea behind this proposal, where a Python script would attempt to map a set of textures to a known shading model, and then construct a MaterialX document with its best guess at the material that corresponds to the author's original intent.

I'm not sure that the approach in this PR is the one that I would recommend for achieving this goal, though, and I have a few thoughts on how it could be implemented in a more forward-looking way.

The most important change that I would recommend is to make this script fully driven by the open definitions of shading models in the MaterialX Data Libraries, rather than leveraging a custom JSON file that duplicates this information in a new form.

By directly using the MaterialX data libraries to determine which shading model is the closest match to the input textures, your script would work automatically with all shading models that MaterialX supports, including Autodesk Standard Surface, glTF PBR, UsdPreviewSurface, and even the upcoming OpenPBR shading model, without needing to extract aspects of each shading model and store it in a custom JSON file.

With the MaterialX data libraries as a guide, I believe it should be possible to write very general-purpose string heuristics that find the closest match between a texture set and all available shading models, without needing to hardcode any details about the naming conventions of any specific shading model.

If you have the bandwidth to take this on, I'd be happy to continue the conversation and provide further ideas on how to achieve this, but of course there's no pressure if you're too busy with other projects.

[Michaelredaa]

@jstone-lucasfilm I think the idea of this proposal is what you recommended on Slack. If MaterialX data libraries have a solution to map between shader modules and texture set names patterns, it will be better to use it instead of the JSON mapping. So if you have any extra information about those libraries, I will be happy to know about them and use them.

[jstone-lucasfilm]

@Michaelredaa Just to set the historical context for this pull request, here's the original thread where @Cinifreak mentioned the idea:

https://academysoftwarefdn.slack.com/archives/C0230LWBE2X/p1684944149082959

And here was my original recommendation in that thread:

This is a really interesting idea, and I could imagine a future Python script that would scan a folder of textures, make an educated guess about the shading model that was intended, and then write out a MaterialX file that binds the textures to this shading model in a clear, unambiguous way.

For guessing the shading model, we might consider comparing the texture filenames against all shading models in the MaterialX data libraries, which would make the process extensible to custom studio shading models as well as open models such as Standard Surface and glTF PBR.

I still believe this is the right approach, and I can provide a few thoughts on how this might be implemented in practice.

As a starting point, I would recommend the SequenceMatcher class that is built into Python, which allows you to efficiently compare the similarity of strings:
https://thecleverprogrammer.com/2022/03/03/sequencematcher-in-python/

By comparing the filenames in a texture set against the inputs of each shading model in the MaterialX data libraries, it ought to be possible to determine the likelihood that the texture set belongs to a specific shading model, and then to determine which texture ought to be assigned to each input that model.

There are some subtleties to think through, for example you may find that common synonyms such as "albedo" and "diffuse" should be accommodated in the script, but otherwise it ought to work for all shading models that MaterialX supports, and should extend automatically to both custom studio models and future models such as OpenPBR.

Let me know how this approach sounds to you!

[Michaelredaa]

@jstone-lucasfilm SequenceMatcher seems like a smart way to get the likeness between 2 strings. I see your approach to making it dynamically auto-guess the texture type based on the shader module input names. I like it, but I have some concerns about it.

• SequenceMatcher returns a ratio, in which ratio will we consider this matched or not? I will consider that we will take the highest match among all textures.
• There are some edge cases in the inputs that match in part of the name, like base_color, coat_color, sheen_color, specular_roughness, coat_roughness,.. 
• The full name of the texture may match with any input and give us the wrong results.
• I thought to use regex with SequenceMatcher, but this will take a lot of hardcoded work to match some patterns like base, base_color, BaseColor, baseColor, ...

I made a quick script to iterate on inputs of mtlx standard surface, then get the max matched texture to easy to understand, we can ignore the UDIMs for now, and this the result:

                               base   21%   the_wing_wingSG_base_color.1001.png
                         base_color   44%   the_wing_wingSG_base_color.1001.png
                  diffuse_roughness   39%   the_wing_wingSG_Roughness.1001.png
                          metalness   37%   the_wing_wingSG_Metalness.1001.png
                           specular   10%   the_wing_wingSG_BaseColor.1001.png
                     specular_color   33%   the_wing_wingSG_base_color.1001.png
                 specular_roughness   38%   the_wing_wingSG_Roughness.1001.png
                       specular_IOR    9%   the_wing_wingSG_BaseColor.1001.png
                specular_anisotropy   11%   the_wing_wingSG_BaseColor.1001.png
                  specular_rotation   20%   the_wing_wingSG_Metalness.1001.png
                       transmission   26%   the_wing_wingSG_Metalness.1001.png
                 transmission_color   34%   the_wing_wingSG_base_color.1001.png
                 transmission_depth   23%   the_wing_wingSG_Metalness.1001.png
               transmission_scatter   22%   the_wing_wingSG_Metalness.1001.png
    transmission_scatter_anisotropy   18%   the_wing_wingSG_Metalness.1001.png
            transmission_dispersion   21%   the_wing_wingSG_Metalness.1001.png
       transmission_extra_roughness   42%   the_wing_wingSG_Roughness.1001.png
                         subsurface   10%   the_wing_wingSG_Normal.1001.png
                   subsurface_color   31%   the_wing_wingSG_base_color.1001.png
                  subsurface_radius   20%   the_wing_wingSG_Roughness.1001.png
                   subsurface_scale   20%   the_wing_wingSG_Metalness.1001.png
              subsurface_anisotropy   23%   the_wing_wingSG_Height.1001.png
                              sheen   22%   the_wing_wingSG_Height.1001.png
                        sheen_color   39%   the_wing_wingSG_base_color.1001.png
                    sheen_roughness   49%   the_wing_wingSG_Roughness.1001.png
                               coat   11%   the_wing_wingSG_Normal.1001.png
                         coat_color   31%   the_wing_wingSG_base_color.1001.png
                     coat_roughness   42%   the_wing_wingSG_Roughness.1001.png
                    coat_anisotropy   26%   the_wing_wingSG_Height.1001.png
                      coat_rotation   21%   the_wing_wingSG_Metalness.1001.png
                           coat_IOR   10%   the_wing_wingSG_Height.1001.png
                        coat_normal   38%   the_wing_wingSG_Normal.1001.png
                  coat_affect_color   31%   the_wing_wingSG_base_color.1001.png
              coat_affect_roughness   40%   the_wing_wingSG_Roughness.1001.png
                thin_film_thickness   45%   the_wing_wingSG_Roughness.1001.png
                      thin_film_IOR   34%   the_wing_wingSG_Roughness.1001.png
                           emission   24%   the_wing_wingSG_BaseColor.1001.png
                     emission_color   37%   the_wing_wingSG_base_color.1001.png
                            opacity   11%   the_wing_wingSG_Normal.1001.png
                        thin_walled   40%   the_wing_wingSG_Metalness.1001.png
                             normal   32%   the_wing_wingSG_Normal.1001.png
                            tangent   26%   the_wing_wingSG_Height.1001.png

What do you think about that?

[jstone-lucasfilm]

This looks very promising, @Michaelredaa!

To reduce the impact of non-varying prefixes such as the_wing_wingSG_, you might consider searching for the longest prefix string that is present across all textures in the set, and removing this from the string that you run through sequence matching. Similarly, you can remove suffixes such as .1001 and .png that match common patterns for UDIM identifiers and filename extensions.

But otherwise, this looks like an approach that should provide good starting matches between textures and shading model inputs, with lots of possibilities for additional refinements over time.

Further in the future, we should consider including additional shading models such as Unreal Default Lit and Disney Principled in MaterialX, since both of those models would be even closer matches than Autodesk Standard Surface for your example texture set.

[Michaelredaa]

@jstone-lucasfilm I've implemented the initial logic and integrated it into the repository. Overall, it's functioning good for me with the exception of the roughness input on the standard_surface.
You can check it with:

python creatematerial -s UsdPreviewSurface /your/texture/dir 

let me know your thoughts about this.

[jstone-lucasfilm]

@Michaelredaa Is this new work merged into the current pull request, or should I look at another location to find it?

[Michaelredaa]

In the same PR https://github.com/Cinifreak/MaterialX/tree/create-material-from-textures/python/Scripts/creatematerial

[jstone-lucasfilm]

This is really starting to come together, @Michaelredaa, and one key simplification that I would recommend is to move all of the Python code into a single module such as python/Scripts/creatematerial.py, rather than creating and maintaining an external helper library.

This will help us to refine the script and its helper functions as a self-contained unit, and we can consider moving to a helper library pattern further in the future, as we learn more about which functionality is most beneficial to share across scripts.

For testing this new script, I would recommend the pattern we currently use for mxformat.py and generateshader.py, where the script is invoked with sample data in our GitHub Actions build matrix:

https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/.github/workflows/main.yml#L210

[jstone-lucasfilm]

I've had a chance to make a few minor refinements to this proposal, and I believe it's looking very close to the bar we'd need to hit for merging.

Here are the latest test results in GitHub Actions, where we're processing both standard_surface and gltf_pbr texture sets for validation:

Analyzing textures in the ..\resources\Materials\Examples\StandardSurface\chess_set folder for the standard_surface shading model.
Generated MaterialX document:
<?xml version="1.0"?>
<materialx version="1.[38](https://github.com/AcademySoftwareFoundation/MaterialX/actions/runs/7889813849/job/21530555437#step:14:39)">
  <nodegraph name="NG_material">
    <image name="chessboard_base_color" type="color3" colorspace="srgb_texture">
      <input name="file" type="filename" value="chessboard_base_color.jpg" />
    </image>
    <output name="chessboard_base_color_output" type="color3" nodename="chessboard_base_color" />
    <image name="chessboard_metallic" type="float">
      <input name="file" type="filename" value="chessboard_metallic.jpg" />
    </image>
    <output name="chessboard_metallic_output" type="float" nodename="chessboard_metallic" />
    <image name="chessboard_normal" type="vector3">
      <input name="file" type="filename" value="chessboard_normal.jpg" />
    </image>
    <normalmap name="chessboard_normal_normalmap" type="vector3">
      <input name="in" type="vector3" nodename="chessboard_normal" />
    </normalmap>
    <output name="chessboard_normal_output" type="vector3" nodename="chessboard_normal_normalmap" />
    <image name="chessboard_roughness" type="float">
      <input name="file" type="filename" value="chessboard_roughness.jpg" />
    </image>
    <output name="chessboard_roughness_output" type="float" nodename="chessboard_roughness" />
  </nodegraph>
  <standard_surface name="SR_material" type="surfaceshader">
    <input name="base_color" type="color3" output="chessboard_base_color_output" nodegraph="NG_material" />
    <input name="metalness" type="float" output="chessboard_metallic_output" nodegraph="NG_material" />
    <input name="normal" type="vector3" output="chessboard_normal_output" nodegraph="NG_material" />
    <input name="coat_roughness" type="float" output="chessboard_roughness_output" nodegraph="NG_material" />
  </standard_surface>
  <surfacematerial name="M_material" type="material">
    <input name="surfaceshader" type="surfaceshader" nodename="SR_material" />
  </surfacematerial>
</materialx>

Analyzing textures in the ..\resources\Materials\Examples\GltfPbr\boombox folder for the gltf_pbr shading model.
Generated MaterialX document:
<?xml version="1.0"?>
<materialx version="1.38">
  <nodegraph name="NG_material">
    <image name="BoomBox_baseColor" type="color3" colorspace="srgb_texture">
      <input name="file" type="filename" value="BoomBox_baseColor.png" />
    </image>
    <output name="BoomBox_baseColor_output" type="color3" nodename="BoomBox_baseColor" />
    <image name="BoomBox_emissive" type="color3" colorspace="srgb_texture">
      <input name="file" type="filename" value="BoomBox_emissive.png" />
    </image>
    <output name="BoomBox_emissive_output" type="color3" nodename="BoomBox_emissive" />
    <image name="BoomBox_normal" type="vector3">
      <input name="file" type="filename" value="BoomBox_normal.png" />
    </image>
    <output name="BoomBox_normal_output" type="vector3" nodename="BoomBox_normal" />
    <image name="BoomBox_occlusionRoughnessMetallic" type="float">
      <input name="file" type="filename" value="BoomBox_occlusionRoughnessMetallic.png" />
    </image>
    <output name="BoomBox_occlusionRoughnessMetallic_output" type="float" nodename="BoomBox_occlusionRoughnessMetallic" />
  </nodegraph>
  <gltf_pbr name="SR_material" type="surfaceshader">
    <input name="base_color" type="color3" output="BoomBox_baseColor_output" nodegraph="NG_material" />
    <input name="emissive" type="color3" output="BoomBox_emissive_output" nodegraph="NG_material" />
    <input name="normal" type="vector3" output="BoomBox_normal_output" nodegraph="NG_material" />
    <input name="sheen_roughness" type="float" output="BoomBox_occlusionRoughnessMetallic_output" nodegraph="NG_material" />
  </gltf_pbr>
  <surfacematerial name="M_material" type="material">
    <input name="surfaceshader" type="surfaceshader" nodename="SR_material" />
  </surfacematerial>
</materialx>

One shader input that the script still struggles with is "roughness", where it maps "chessboard_roughness.jpg" to coat_roughness in standard_surface.

There are a number of ways that we could potentially improve this, and one option that comes to mind is to define a "synonym" map in the script, where the isolated term "roughness" would be considered equivalent to "specular_roughness", following the common usage of "roughness" in computer graphics.

More specific terms such as "sheen_roughness" should ideally be left alone, so that textures including this more precise string can correctly find the "sheen_roughness" input of each shading model. At the moment, I believe the matching logic would only see the word "roughness" in this specific term, and we may want to refine the logic so that the name "sheen_roughness" is fully preserved.

Let me know what your thoughts are!

[Michaelredaa]

@jstone-lucasfilm Thanks for the review and for the updates.
I taught about it the way you think about it, to map it to take a specular_roughness as a default to roughness

[jstone-lucasfilm]

@Michaelredaa Since we're aiming to include this feature in MaterialX 1.39, would you mind making this a pull request for the dev_1.39 branch where active development on that version of MaterialX is ongoing?

[Michaelredaa]

To avoid making new pull request, maybe this request for @Cinifreak because he is the pull request owner and he have the ability to change the pull request merge branch.

[jstone-lucasfilm]

@Cinifreak If you have a chance to retarget this pull request to dev_1.39, that would be much appreciated, and we can continue review and refinement of the proposal there.

[Cinifreak]

@jstone-lucasfilm I tried to do that. but it seems i don't have permission to do that.

[jstone-lucasfilm]

@Cinifreak Feel free to create a new pull request to dev_1.39 if that's more straightforward, and there's no rush if you and @Michaelredaa are busy.

[jstone-lucasfilm]

Thanks @Michaelredaa, I'll go ahead and close this, as it's been superseded by #1746.