Geometries attached to bone have strange transforms

[Nehon]

Hello, I think transforms of an object attached to a bone may be wrong.
consider this example
simple.zip
A simple rigged model. The last bone of the rig has a child geometry (knob). the last bone root is at 0,0,2, and its tip is at 0,0,3.
The knob geom is at 0,0,4 in world space.
If you look in the exported gltf everything is fine for the base rigged geom and the skeleton, and the knob geometry is a child of the last bone as expected, however, its transforms are like this

 {
            "mesh" : 0, 
            "name" : "Knob", 
            "rotation" : [
                -0.7071068286895752, 
                0.0, 
                -0.0, 
                0.7071068286895752
            ], 
            "translation" : [
                0.0, 
                0.0, 
                -1.0
            ]
}, 

First, it looks like the translation is taken from the tip of the bone, instead of the root of the bone. Which is a problem because we have no information about the tip in the gltf file.
Also there is this rotation that looks like a Z up to Y up correction rotation, but backward.
I don't know what to do with these transforms, to me they should be

 {
            "mesh" : 0, 
            "name" : "Knob",            
            "translation" : [
                0.0, 
                2.0, 
                0.0
            ]
}, 

Or any equivalent transforms. basically the local transforms of the geometry relative to the bone root it's attached to.

[UX3D-nopper]

Works as expected:

If you have a general question, how joints etc. work in glTF, please create an issue here:
https://github.com/KhronosGroup/glTF/

Also, I recommend to read the user and developer documentation, as they also can answer your question:
https://github.com/KhronosGroup/glTF-Blender-Exporter/blob/master/docs/user.md
https://github.com/KhronosGroup/glTF-Blender-Exporter/blob/master/docs/developer.md

Finally, I recommend to read, how rigging is done in Blender:
https://docs.blender.org/manual/de/dev/rigging/index.html

[Nehon]

Well according to your shot it doesn't work as expected at all, the Knob geometry is not even displayed.
Could you please reconsider this issue, reading carefully my description?

[UX3D-nopper]

You provided me this file:

[UX3D-nopper]

There are no knobs in the scene file.

[UX3D-nopper]

And if I am using your provided glTF 2.0 scene file, there is a "knob":

The exported glTF 2.0 scene file belongs to another Blender file.

[Nehon]

ho ... I messed up again I'm sorry... I always forget to save the blend file...
this one has all the information
simple.zip
re gltf scene : exactly, the knob is not in the right place

[UX3D-nopper]

The object is assigned to a bone. The mesh has no vertex groups. I think I have answered this question before and this is not supported at the moment.

[UX3D-nopper]

But it is nowhere listed :-)

[Nehon]

Oh ok, I thought it was supported because except for this issue it works... :p
Thanks, and sorry again for the bad first report.

[UX3D-nopper]

glTF 2.0 does not directly support the feature to assign a node to a joint.
To solve this, a vertex goup plus joints and weights has to be created.

[Nehon]

mhh doesn't it? Doesn't looks like it forbids it though... I'll post an issue in the spec project, this, at the very least, needs clarifications.

[McNopper]

There's the difference:
In glTF, a mesh can be assigned to a node, which is also a joint. But this is not possible in Blender: You cant put a mesh into a joint/bone hierarchy.
In this case, the node/joint tree is skeletal animated, but "only" the final matrix modifies the mesh transform.

The scene you provided me, the mesh object is still a child of the Armature plus there is a property, where the mesh object belongs to a joint/bone. And this is the tricky part and that's the reason why I provided you these links.

It is not a glTF 2.0 specification problem. And the problem is also solvable, but the solution is not straight forward. If you think so, go ahaed and implement it :-)

[Nehon]

Well, I'm not saying it's easy, I'm saying it almost works in current implementation, minus the porper transforms.
To me it sums up to this:

• A geometry is parented to a bone, and it's not skinned (no armature modifier). This is done by selecting the geometry then the bone hitting ctrl+P then choose "bone".
• When exported to gltf, the geometry appears in the children list of the parent bone. Implementations can make the difference between a child bone and a child geometry because geometries have a "mesh" attribute. Even if we have a more complex scene graph with multiple nodes (like empties) attached to a bone, implementers can sort it out, because those nodes are not referenced as joints in any skin's joint hierarchy.
• The transforms of the geometry must be its local transforms relative to the parent bone's root.
• This situation can be handled in different ways by loader implementations:
  - Handle the specific case by computing world transforms of the geometry on each frame by combining the world transforms of the bone and the local transforms of the geometry
  - Handle it as if the geometry was skinned, and create the proper buffers in the mesh, with every vertices pointing to the parent bone. The inverseBindmatrix can be computed from both the geometry and the parent bone transforms.

My point is, the exporter should not do any assumption on how the information should be interpreted, and export the data as there are in the blender file.

[McNopper]

Please consider this:
Blender has much more features than glTF. So, it is not just taking the data from Blender and writing it to glTF. For every export and feature, this has to be considered and the data in Blender needs to be converted to be glTF 2.0 compliant. The Blender exporter is not doing any assumptions. With this and other current issues, the exporter needs to work around the larger feature set of Blender.

In this spcific case, the mesh has still the Armature as the parent. So, with your description, if someone wants to render the scene without skinning - it will fail. So, which inverse matrix and/or transformation matrix will you take? The one from the current node or the one from the bone? What is happening, when you animate the armature, the mesh node and the bone?

But the exporter is open source. You are free to implement it and make a pull request.

[Nehon]

No you misunderstood what I meant, I'm no saying the blender importer does assumptions, I say it shouldn't. I'm not a native english speaker so sometimes I may sound more blunt than I intended, I'm sorry.

Anyway, you are right about the need to convert the blender's larger feature set.

I'm not really experienced neither in blender plugin dev nor python, but I can give it a try.

[McNopper]

The only proper solution is to create joints and weights for this mesh and give full weight to the bone it is assigned to.
Just look at the code and give it a try.

[Nehon]

Hehe, well I don't agree with that so before diving into the code I'd rather we reach to an agreement. I don't want to spend too much time on it and have my PR rejected.

Why do you think, that it's the only proper solution? Why handling it like any regular node would be a problem (child node with local transforms).

In jMonkeyEngine, we can handle both solutions, so in the end it will work for us, But the regular node solution is preferable to me as we have a special and optimized mechanism for this in the engine.

Alternatively, would it be conceivable to have an option checkbox "Consider object parented to a bone as skinned by this bone" or something along those lines. And support both ways.

[McNopper]

If it is working 100%, I will not reject it.

Because the mesh is assigned to a node and the node has the armature as a parent. Additionally, it is linked to a bone.

So, animate the armature, animate the meshes node, animate the whole bone. All animations are influencing the mesh - both from the object node and(!) the skeletal bone node tree. That's only possible - from my perspective - if you follow my suggested approach.

If you have another solution, please go ahead, I am always willing to learn.

The scene has one, exact behaviour. If you export it - saying you are supporting this feature - it has exectly to behave as seen in Blender. Supporting both ways is no option, as it just solves your problem in this specific case.

[donmccurdy]

@UX3D-nopper could you clarify — Can this not reasonably be implemented? Or is it not a recommended way to use Blender skinning, regardless of the export format?

If this is something Blender artists might typically do that the exporter or glTF simply don't support, I think we should do one of several things:

(a) Document it.
(b) Amend the glTF specification.
(c) Leave the issue open in the hope that a community member will implement export of nodes with armatures as parents.

With the issue closed I'm worried that issues like #155 will continue to be common. If the reason for closing is simply that we have no immediate plans to implement it, I'd prefer to have this open as a record.

[donmccurdy]

Reopening, I think documentation (and ideally a visible warning during export) would be worthwhile, even if no one can get to it right away.

[donmccurdy]

Tracking this issue on the new project here: KhronosGroup/glTF-Blender-IO#286.