Include standard data libraries in Python package

[kwokcb]

Package the standard definition libraries as part of the Python package.
This removes the need for users to have to always search for the libraries, and the possibility to mistakenly use libraries which do not match the Python distribution.

• An additional MANIFEST.in is included to add in the libraries folder if setuptools is used instead of distutils.core.
• A convenience function getDefaultDataSearchPath() will provide a default search path to find libraries.
• A convenience function getDefaultDataLibraryFolders() will return a list containing data library folder names.
• The mxvalidate.py script has been update use the new API with loadLibraries(). The simplest usage would be
  this one-liner:

# Create the library document
stdlib = mx.createDocument()
# Load in the default data libraries
 mx.loadLibraries(mx.getDefaultDataLibraryFolders(), mx.getDefaultDataSearchPath(), stdlib)   

[kwokcb]

@jstone-lucasfilm , this is follow-up to your suggestion to allow std libs to be part of the Python package. Seems useful to
always have this available as it does not force usage of these libraries, but does provide consistency in releases. It is left it as a build option. Let me know what you think.

[jstone-lucasfilm]

Thanks for putting this proposal together, @kwokcb, and I think this is a great direction for the future. I wonder if we might ultimately want the data libraries to be included in all MaterialX Python builds, rather than making this optional? I'm interested in thoughts from other developers on this question, and we may want to discuss this at a future MaterialX TSC meeting.

[kwokcb]

Hi @jstone-lucasfilm ,
I had originally wrote this as always being there.

The pros I see are:

1. What has been discussed before. The libraries can always be found.
2. The Python package is self-contained. No additional "side-car" files are required.
3. The libraries are always consistent with the release. I'm sure folks could accidently grab the wrong libraries.
4. No more need for search libraries -- and possibly getting the wrong libraries due to searching.
5. This could help with USD integration as it's a set of libraries that is more discoverable.

Cons are:
A. It's static -- but this is offset by the fact that they are not automatically loaded so a developer can still load
in their own std libraries.
B. There is a degree of overhead as they are duplicated -- but this should be offset by the fact that they have to exist somewhere. Also you don't need the copies for C++ usage if your just working in Python.

In all, I think the pros out-weigh the cons (especially 1-3) , so I vote we not have an option at first, and then if there are issues
raised by developers add in a new option.

[jstone-lucasfilm]

We had a chance to discuss packaging of data libraries with MaterialX Python at today's MaterialX TSC meeting, and the idea had consistent support from the members of the TSC. Here are the full meeting notes for those who are interested:

https://academysoftwarefdn.slack.com/archives/C0230LWBE2X/p1676403077275179

[jstone-lucasfilm]

@kwokcb Taking this proposal forward, my sense is that we can safely remove the MATERIALX_INCLUDE_PYTHON_STDLIB option, and include the data libraries in all MaterialX Python packages. This would allow MaterialX Python developers to rely upon it being present, starting with the version of MaterialX in which this proposal is ultimately released.

[jstone-lucasfilm]

This looks great to me, thanks @kwokcb, and I'll merge the feature so that it's included in the upcoming MaterialX 1.38.7.