InstancedSkinnedMesh #22667
InstancedSkinnedMesh #22667
Conversation
|
Um, do we really want to continue the creation of different mesh classes? I just say this because we have no How would the change look like if the existing |
@Mugen87 Yes probably Mesh and SkinnedMesh would need to be merged. This PR is super light on code because InstancedSkinnedMesh carries both the isInstancedMesh and isSkinnedMesh flags and, to my suprise tbh, that just worked with very minimal changes in the renderer. This actually is a question for @mrdoob, I just followed the existing patterns, imho it's no problem having 4 different classes for the mesh types and I can't imagine we'll be having more mesh classes than these. I don't really mind having different classes but what kills me is the performance of filling in the bone texture with matrices. In the demo it updates the bones for ~200 instances per frame and the time it takes to update the bones and compute the matrices is slightly more than what is required to render them. Since we repeatedly update a dummy object, a way to skip storing intermediate computations for bone animations and just have the mixer update the bone texture directly would give us huge wins here |
|
@makc unity treats it similarly to three in the sense that they have separate renderers(the equivalent of mesh for us) for static and skinned meshes. Instancing they do differently, you have to do minor additions to your shaders and set a flag on the material. Unity doesn't come with skinned instancing out of the box and because even the static instances need one to one representation in the scenegraph it will always be less efficient than a custom solution. Personally I prefer the three approach which gives you direct access to the instancing attributes/textures as it allows to get ultra efficient even using wasm to feed the instancing attributes/textures. The matrix math and scenegraph traversal we have are inherently slow because js works with doubles and heavy structures. Going to wasm+floats+simd to work on Float32Arrays directly and expose these arrays to be used for the instancing attributes/textures would speed things up an order of magnitude, better than what unity could deliver if they actually had instanced skinning because they'd still need to go through the scenegraph, which is more efficient for them than us since they work with floats and simd already. The code paths in the three renderer are pretty clean so as long as an object carries the appropriate flags like isInstancedMesh, isSkinnedMesh, isPoints and the appropriate structures for them (instanceMatrix, instanceColor, count, bindMatrix, skeleton, etc) things will just work, or at least we're not very far away from that code wise. So I guess we could have a method of compositing properties of a generic object. So isPoints + isInstanced along with the structures for both should give us an InstancePoints mesh. And we could have several classes with fixed configurations like InstancedMesh or SkinnedLines etc |
|
@Mugen87 @mrdoob how about a single Drawable class with flags and setters to configure it on the fly? ie Drawable.isSkinned = true would setup the appropriate structures in conjunction with the rest of the configuration. Some flags would be mutually exclusive like isMesh and isPoints etc. IsInstanced would apply to all |
|
Related issue: #22695 @mrdoob @Mugen87 turns out rendering the skinnedmeshes as points was very easy as suspected. With a very minor reshuffling and adding the skinning bits to points.glsl all we need to do is attach a PointMaterial and set (Instanced)SkinnedMesh.isPoints=true. I updated the example to toggle between mesh and points rendering by clicking. https://wizgrav.github.io/three.js/examples/webgl_instancing_skinning.html
|
|
Closing this to bring it back more comprehensive |
|
@wizgrav - do you plan to resurrect this? |
|
There is a solution for WebGPU: https://threejs.org/examples/webgpu_skinning_instancing |
|
@Mugen87 Hi, so awesome that its available in WebGPU but considering WebGPU is not in the mainstream as of yet to my knowledge, I think there would still be a lot of benefit to having a WebGL2 equivalent for the time being. @wizgrav 's solution looks pretty solid from what I can tell. If there are other solutions I have not found yet please let me know :) |
|
There's a way to do this in user-land like https://codesandbox.io/s/2yfgiu although it requires a hack for points rendering -- |
|
What would it take to have this in ThreeJS core? I'm willing to put time into a comprehensive PR if someone can point me in the right direction. |
|
@FarazzShaikh the API just needs to be clean and make lots of intuitive sense. Can't be a hodge podge like in some other engines. The functionality is already proven out above, but it is the way the classes and code are organized that make a difference (judging from maintainers' good taste for well-organized classes). So if you can come up with some clean layout for the feature, it may have a higher chance to go in. No one may know exactly what that layout is though. Maybe the WebGPU sample should be replicated? (Haven't looked) |
|
Out of curiosity, what are the per instance animation setting capabilties in this PR?
I might might be missing obvious technical challenges with doing that per instance, but without at least playing different actions per instance, the usability is quite limited. |
Similar to InstancedMesh, along with an example, basically repeatedly updating bone matrices and calling setBonesAt(index) to fill a texture with all bone matrices for all instances.
Shader code uses gl_InstanceID and texelFetch so it works on WebGL2 only. It can be made backwards compatible with 1 more attribute(for the lack of gl_InstanceID on WebGL1) and uglier code both on js and glsl side. Should we start standardizing on WebGL2, at least for new features?
@mrdoob the usefulness of this is limited by our capability to fill the bone matrix texture. It would be amazing if we could have an (Instanced)AnimationMixer directly filling the bone texture with matrices with as less copying as possible. Then we can reach thousands of independently animated meshes on screen.
Example live here: https://wizgrav.github.io/three.js/examples/webgl_instancing_skinning.html