Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core Solver - Use In-Memory Representations of Bundles and Cameras #114

Closed
david-cattermole opened this issue Jun 14, 2019 · 3 comments
Closed
Assignees
Labels
core solver Related to the low-level solver functions. enhancement maya plugin Maya API Plug-In (C++)
Milestone

Comments

@david-cattermole
Copy link
Owner

Feature

The solver currently sets Maya node attribute values and then queries the world-space position of the bundles and cameras. If a camera or bundle's attribute is being solved, and there is no hierarchy values being solved, it is possible to avoid setting Maya node attributes and instead creating Maya MTransform API objects "in-memory" and setting and querying these objects instead. This should prove faster than setting Maya node attributes directly as we will not need to query the Maya DG.

There are a number of conditions that a attribute must meet before being considered for this optimization:

  • The attribute must be a translate attribute on a Bundle node or a translate, rotate or focal length attribute on a camera.
  • The Bundle node's parent must be world/root, or cannot be changed by the solver (for example it is locked or connected).

The Bundle's parent node transform must be queried across all frames needing to be solved and used as the transform space for the MTransform object internally.

The Bundle's translation X, Y and Z attributes do not need to be added to the solver together, solving independent transform axis is possible.

It may be possible to extend this optimization feature to all transform node attributes, in the future, but the first priority is Bundles and cameras.

This feature should be able to be turned on/off with a flag given to maya.cmds.mmSolver.

Expected behavior:
This is a performance optimization only. The results of the solver output should not change if using or not using this feature.

This feature should greatly speed-up camera-based solves, since querying the Maya DG is by-far the slowest aspect of the solver.

@david-cattermole david-cattermole added enhancement core solver Related to the low-level solver functions. maya plugin Maya API Plug-In (C++) labels Jun 14, 2019
@david-cattermole david-cattermole added this to the v0.4.0 milestone Jun 14, 2019
@david-cattermole david-cattermole self-assigned this Jun 14, 2019
@david-cattermole
Copy link
Owner Author

This feature can be extended to a more general use of "in-memory" calculations, rather than using Maya's DAG network.

We should not limit ourselves to only world space attributes and nodes, but instead we should convert the entire Maya DAG / DG into an exact representation entirely by mmSolver defined classes.

As a starting point for this feature we should write a Maya API function that will convert a list of nodes into internal solver objects. This test command should then return a mapping of node attributes and values. These values must match Maya perfectly, and we will assert this in our unit tests.

After we are sure our test nodes match Maya we can start manipulation of the internal solver objects and create a flag to mmSolver command to allow using the "in memory" solver objects.

I expect by writing our own solver objects we will be able to gain a lot of performance.

Matrix classes should probably use the C++ Eigen headers, and we will build our own transform nodes on top of this.

david-cattermole added a commit that referenced this issue Dec 25, 2021
david-cattermole added a commit that referenced this issue Dec 25, 2021
david-cattermole added a commit that referenced this issue Dec 27, 2021
The library (crate) will not yet integrate and might even break the
project build, but this is the bare-bones for a Rust-only 3D scene
including camera, marker and bundle reprojection, with any transform
hierarchy.

Issue #114.
david-cattermole added a commit that referenced this issue Dec 28, 2021
Fix bug; The marker "links" to bundles and cameras were not
cleared when the SceneGraph.clear() was called.
These are now cleared as expected.

Issue #114
david-cattermole added a commit that referenced this issue Dec 28, 2021
david-cattermole added a commit that referenced this issue Dec 28, 2021
crate::scene::bake::bake_scene_graph() will create a FlatScene.

The FlatScene will contain the baked node hierarchy with data
structures to allow fast evaluation of the hierachy with different
attribute values.

Issue #114
david-cattermole added a commit that referenced this issue Dec 28, 2021
The Trait is no longer needed. All parenting information is stored
in the SceneGraph, not at the node level.

Issue #114
david-cattermole added a commit that referenced this issue Dec 28, 2021
This will ensure the Trait can also get the marker node weight.

Issue #114.
david-cattermole added a commit that referenced this issue Dec 28, 2021
Because the projection matrix input should always be a camera, not a
transform.

Issue #114.
david-cattermole added a commit that referenced this issue Dec 28, 2021
david-cattermole added a commit that referenced this issue Dec 28, 2021
david-cattermole added a commit that referenced this issue Dec 28, 2021
Reduced the number of function parameters by grouping attributes
together into camera and transform.

Filter out the bundles and cameras to correctly calculate the
world matrices needed.

Support bundle reprojection for multiple cameras in one scene
evaluation.

Issue #114.
david-cattermole added a commit that referenced this issue Dec 29, 2021
…ndle.

Add SceneGraph.get_camera_node_id_from_marker_node_id() and
get_bundle_node_id_from_marker_node_id() methods to query the
marker relationship data.

Issue #114
david-cattermole added a commit that referenced this issue Dec 29, 2021
LTO with codegen-units as 1 produces a faster binary.

Also removed some options that are default for release mode, such as
'debug' and 'debug-assertions'.

Issue #114
david-cattermole added a commit that referenced this issue Dec 31, 2021
A EvaluationObjects object was added to store all required nodes for
evaluation, rather than using 'rust::Vec' in C++ because it appears
it wont link to the library.

Link against 'Bcrypt' library because from Rust 1.57+ it's now required.
rust-lang/rust#91974

Issue #114.
david-cattermole added a commit that referenced this issue Jan 2, 2022
The simple test scene has no hierarchy - just flat data.

Added the .markers() method to simply see the differences
between the ground truth (marker) and reprojected point - easier
for debugging and places all the needed data in one place.

Issue #114
david-cattermole added a commit that referenced this issue Jan 2, 2022
david-cattermole added a commit that referenced this issue Jan 2, 2022
…ined.

This is for debugging mostly, to ensure - from the C++ outside world only
 - that our data still exists inside the AttrDataBlock object.

Issue #114.
david-cattermole added a commit that referenced this issue Jan 2, 2022
david-cattermole added a commit that referenced this issue Jan 2, 2022
Also adds the ability to create a FlatScene without any data, which is
needed so we can create FlatScene then pass it to a function as reference
to be filled (as we will do in Maya).

Issue #114
david-cattermole added a commit that referenced this issue Jan 2, 2022
This will avoid copying data structures - we assume that's a good thing
because the gain from any compilier optimisation here is probably
overkill (?). This is a mostly untested assumption!

Issue #114.
david-cattermole added a commit that referenced this issue Jan 2, 2022
This feature is currently hard-coded to be used, but can be disabled
and will be optional with an mmSolver command flag (to be added soon).

Current limitations are:
- No hierarchy of transforms is supported - only world space nodes.
- Smoothing and stiffness attributes are not supported.
- Constraints are not supported.
- Rivets are not supported (and cannot be supported by the current
  implementation of mmscenegraph.)

Issue #114
david-cattermole added a commit that referenced this issue Jan 2, 2022
Use '0' for Maya DAG (the default value for now), or use '1' for
mmSceneGraph.

The Maya DAG should be used for backwards compatibility and the full
feature set, but mmSceneGraph should provide improved performance
for simplier use-cases.

Issue #114.
david-cattermole added a commit that referenced this issue Jan 2, 2022
Introduced a new "FrameSolveMode" option to choose how to solve
the list of frames.

Issue #114.
david-cattermole added a commit that referenced this issue Jan 2, 2022
Solving with 'all frames at once' currently crashes with
MM Scene Graph.

Solving with the "Basic" solver tab will solve all frames
in a single call to the mmSolver command.

Issue #114.
david-cattermole added a commit that referenced this issue Jan 2, 2022
…lue.

Also removes unneeded doc-strings.

Issue #114
david-cattermole added a commit that referenced this issue Jan 3, 2022
The function was incorrectly calling 'link_marker_to_bundle'.

Issue #114
david-cattermole added a commit that referenced this issue Jan 3, 2022
Store dense animated attributes for frameLists with non-sequential
frame numbers.

Issue #114
@david-cattermole
Copy link
Owner Author

david-cattermole commented Jan 3, 2022

Here is the current state of this issue...

Lower-level core library:

  • Construct a flat Scene Graph matching Maya's DAG.
  • Construct a Scene Graph with hierarchy matching Maya's DAG.
  • Library (written in Rust) added to CMake build system.
  • Expose the Rust library features as C++ library to the mmSolver Maya plug-in.
  • Support static attributes
  • Support dense multi-frame animated attributes
  • Support sparse animation curve data.
  • Support connections between nodes.
  • Support trivial constraints between nodes.
  • Support custom nodes (by baking values only).
  • Support smoothness and stiffness deviation.

higher-level Maya wrapper:

  • Translate a basic Maya DAG into a MM Scene Graph representation.
  • Automatically detect Maya DAG to MM Scene Graph translation failures and revert to Maya DAG.
  • Allow the mmSolver command to optionally use the new Scene Graph or not.
  • Allow users to choose to use MM Scene Graph or Maya DAG (via the Solver UI)
  • Support hierarchies of nodes nodes by Maya wrapper.
  • Support multi-frame evaluation with arbitrary/non-sequential frame numbers (such as the "Standard" solver)
  • Support smoothness and stiffness attribute details.

david-cattermole added a commit that referenced this issue Jan 5, 2022
Issue #114.

# Conflicts:
#	src/CMakeLists.txt
#	src/MMSolverCmd.cpp
#	src/mayaUtils.h
david-cattermole added a commit that referenced this issue Jan 13, 2022
@david-cattermole
Copy link
Owner Author

Closing. This has been released in v0.4.0.

There are some missing features. These will be added as needed in future versions of mmSolver.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core solver Related to the low-level solver functions. enhancement maya plugin Maya API Plug-In (C++)
Projects
None yet
Development

No branches or pull requests

1 participant