Maya: Improve lib.polyConstraint performance when Select tool is not the active tool context

[BigRoy]

Issue

The Maya validators Validate Mesh Non Zero Edge and Validate Mesh NGONs rely on using Maya's poly selection constraints to quickly retrieve the relevant components. However, since these rely on Maya selection of actual vertices/edges there are UI edges cases that can make this query very slow for high-res meshes.

The issue is basically that it is extremely slow when the user is NOT currently in the Select tool context. As if other manipulators do some extra processing during this step that introduces a huge performance drop. Previously it could literally take MINUTES for a very a bunch of very high res meshes to be processed.

This PR implements an optimization by ensuring the tool context is currently set to the the Select tool and not e.g. the Translate, Rotate, Scale or other tools since those produce the extreme slow down. I've seen performance increasements of over 100x speed difference. With this change I've not seen these validations take much longer than a second at most - and those were the most extreme cases - where previously they literally could take minutes.

With this speed optimization it makes these validators actually useful as opposed to "in the way".

Performance tests

With four cubes at 300x300x300 subdivs with Translate tool active:

New: 0.55668044090271
Old: 12.485342502593994

With four cubes at 300x300x300 subdivs with Select tool active. This is the case that didn't have speed issues before:

New: 0.06696057319641113
Old: 0.06696152687072754

Click to expand for script used to test performance difference.

from maya import cmds
import maya.api.OpenMaya as om
import contextlib

from openpype.hosts.maya.api.lib import *


@contextlib.contextmanager
def maintained_selection_api():
    """Maintain selection using the Maya Python API.

    Warning: This is *not* added to the undo stack.

    """
    original = om.MGlobal.getActiveSelectionList()
    try:
        yield
    finally:
        om.MGlobal.setActiveSelectionList(original)


@contextlib.contextmanager
def tool(context):
    """Set a tool context during the context manager.

    """
    original = cmds.currentCtx()
    try:
        if context != original:
            cmds.setToolTo(context)
        yield
    finally:
        cmds.setToolTo(original)


def polyConstraint(components, *args, **kwargs):
    """Return the list of *components* with the constraints applied.

    A wrapper around Maya's `polySelectConstraint` to retrieve its results as
    a list without altering selections. For a list of possible constraints
    see `maya.cmds.polySelectConstraint` documentation.

    Arguments:
        components (list): List of components of polygon meshes

    Returns:
        list: The list of components filtered by the given constraints.

    """

    kwargs.pop('mode', None)

    with no_undo(flush=False):
        # Reverting selection to the original selection using
        # `maya.cmds.select` can be slow in rare cases where previously
        # `maya.cmds.polySelectConstraint` had set constrain to "All and Next"
        # and the "Random" setting was activated. To work around this we
        # revert to the original selection using the Maya API. This is safe
        # since we're not generating any undo change anyway.
        with tool("selectSuperContext"):
            # Selection can be very slow when in a manipulator mode.
            # So we force the selection context which is fast.
            with maintained_selection_api():
                # Apply constraint using mode=2 (current and next) so
                # it applies to the selection made before it; because just
                # a `maya.cmds.select()` call will not trigger the constraint.
                with reset_polySelectConstraint():
                    cmds.select(components, r=1, noExpand=True)
                    cmds.polySelectConstraint(*args, mode=2, **kwargs)
                    result = cmds.ls(selection=True)
                    cmds.select(clear=True)
                    return result


def polyConstraint_old(components, *args, **kwargs):
    """Return the list of *components* with the constraints applied.

    A wrapper around Maya's `polySelectConstraint` to retrieve its results as
    a list without altering selections. For a list of possible constraints
    see `maya.cmds.polySelectConstraint` documentation.

    Arguments:
        components (list): List of components of polygon meshes

    Returns:
        list: The list of components filtered by the given constraints.

    """

    kwargs.pop('mode', None)

    with no_undo(flush=False):
        with maya.maintained_selection():
            # Apply constraint using mode=2 (current and next) so
            # it applies to the selection made before it; because just
            # a `maya.cmds.select()` call will not trigger the constraint.
            with reset_polySelectConstraint():
                cmds.select(components, r=1, noExpand=True)
                cmds.polySelectConstraint(*args, mode=2, **kwargs)
                result = cmds.ls(selection=True)
                cmds.select(clear=True)

    return result
    
    
import time

meshes = cmds.ls(type='mesh', long=True)

# Get all edges
edges = ['{0}.e[*]'.format(node) for node in meshes]

s = time.time()
edges = polyConstraint_old(edges,
               t=0x8000,  # type=edge
               length=1,
               lengthbound=(0, 0.001))
e = time.time()
print("Old: %s" % (e-s))

s = time.time()
edges = polyConstraint(edges,
               t=0x8000,  # type=edge
               length=1,
               lengthbound=(0, 0.001))
e = time.time()
print("New: %s" % (e-s))

Note

The speed comparison can be a bit tricky since running the same query multiple times within the same code block might execute faster on the separate run. For example, have a look at this time difference when running the Old first and then the New version. And running the code with the New first and the Old version after.

Old: 3.324089765548706
New: 0.14087533950805664

New: 0.15989446640014648
Old: 0.24584197998046875

It seems that if the newer version runs first that even the old one isn't as slow as it was. As if somehow the resulting value is "cached" already in memory and thus gives less issue with Manipulators being visible.

Nevertheless, the new code basically always performs optimally with just a minor decrease in speed for the case where it previously didn't encounter the speed issue so this should be a no brainer.

Also the speed difference seems to be much less in Maya 2022 (maybe the manipulators are optimized?) but still quite present.

Related

I originally posted about this a long time ago in the OpenPype discord. Here's a link to that.