Replacing a Mesh Resource with Blendshapes on a MeshInstance Node at Runtime has a Mesh corruption chance

[smix8]

Godot version:
3.2.0 stable.official
3.2.1 stable.official
3.2.2.rc.custom_build.4f00a2c64

OS/device including version:
Windows 10 x64
GTX980Ti, RTX2080Ti

Issue description:
Replacing a Mesh Resource with Blendshapes on a MeshInstance Node at Runtime has a Mesh corruption chance.

I try to give as much information as possible since I am unable to provide an example project that yields the error reliably.

Failure results are one of the following:

• Invisible Mesh
• Suddenly wrong, unnormalized blendshape mode (see image) but correct blendshape values from the loaded Mesh.
• Disfigured or completely exploded Mesh that lags and/or crashes game after some time.
• This one is odd ... one randomized range value gets applied to all Blendshapes. Those Blendshapes are visible on the Mesh while the Inspector insists that all Blendshape values are at zero and they can no longer be adjusted by sliders or through code at that point.

Only Mesh resources with blendshapes seem to be affected.
Doesn't matter if Mesh (sub)resources are unique or shared or local to scene.
It seems the failure chance is increased the more demanding the Mesh subresources are.
Mesh gets saved broken in cache, requiring a cache clear to fix mesh with a reload.
It occurs random but often enough to make a 3d game with e.g. character customization unstable and unreleaseable for the public. All while the project shows no performance issues and runs at 100+ fps.

Odd solution / Fix:
Immediatly after assigning the Mesh Resource looping all blendshapes with GDScript and getting and setting values redundantly a second time prevents all breaking cases.

MeshInstance.set_mesh(new_mesh_resource)
Result -> High chance of Mesh death with twisted blendshapes

MeshInstance.mesh = new_mesh_resource
loop_through_blendshapes_and_get_and_set_same_value_again_function()
Result -> Happy alive Mesh no matter the former circumstance of death

MeshInstance.mesh = new_mesh_resource
... waiting a few frames watching the dead Mesh
loop_through_blendshapes_and_get_and_set_same_value_again_function()
Result -> Rescue to late, Mesh patient stays dead and requires reload

A second, redundant loop to set the values again shouldn't be necessary at all to prevent breaking meshes added at runtime. Maybe it works cause the blendshape set function triggers a hidden engine notification in the background? Looping all blendshapes with GDScript is a silly solution and should be replaced by a proper way to prevent this errors in the future without a scripted workaround.

Expected behavior:
All Blendshape values should load at zero by default when the Mesh Resource changes on the MeshInstance Node ( e.g. set_mesh() ). If the new loaded Mesh Resource comes with saved Blendshape values those should be applied (afterwards?). By no means should a new mesh get corrupted blendshapes just because the asset is a bit more demanding to preload. Especially when not more demanding as anything commonly found in AAA games of the past 10 years.

Minimal reproduction project:
All attempts to recreate the problem reliably with a small, shareable testscene didn't work. The problem only appears more regularly with heavy assets but otherwise completely random. No idea about C++ engine territory but maybe from the described workaround the solution could be as simple as sending another update notification for the blendshapes after the mesh resource was changed?

[fire]

Can you retest on 3.2 branch and on master? I think 3.2 is still broken but master shouldn't be.

Post your versions.

[smix8]

@fire
Godot v4.0.dev.custom_build.fe93f6292
Godot v3.2.4.beta.custom_build.fb9f723bf

Issue is still present in Godot 3.x.

I had both colorful mesh explosions and mesh blendshape abominations immediatly on multiple characters after disabling the workaround. Workaround is just a loop function that resets all blendshape values to zero both before and after replacing a mesh resource on a mesh instance.

You can still see the working clothing meshes floating in the air that have the same character blendshapes as the destroyed character meshes.

This abomination is created by the engine when the mesh is added to the scenetree. All available blendshapes get random values applied and the blendshape mode gets twisted corrupting the mesh scale.

Unfortunately I am completely unable to import any mesh with blendshapes in a Godot 4.x.

Godot fails to import blendshape data from both gltf2.0 and .escn import files exported from blender 2.83/2.90/2.91.
I also exported as a textfile (which was painful slow) and blendshape data is included but Godot 4.x fails to read it.
The Godot 3.x mesh resources from the affected project are broken and fail to load in Godot 4.x.

Any other import ideas or waiting for Godot (fix)?

[fire]

When was Godot v4.0.dev.custom_build.fe93f6292 made and what is the build it was based on?

Can you post a sample?

[fire]

ecda989 Has a fix for blend shapes.

Edited:
Your build for fe93f62 is too old. Please use a recent build.

[smix8]

Oh sorry @fire ! I updated 1 hour before my posting but for whatever reason SCons build Godot 4.x with an old, cached version.

I now build Godot 4.x with the recent master commit 59b30e1
Godot 4.0.dev.custom_build.59b39e1d2

GLTF worked and I was able to import all meshes with blendshapes.
FBX import was broken and ignored all blendshapes.
ESCN import was broken, imported blendshapes and showed them in the inspector with sliders but all meshes became invisible.

With the GLTF import I had very mixed results.
I spawned 50+ fully animated and skinned LOD0 characters to the scene and none showed the issue. In Godot 3.x it is often enough to have 2-3 characters loaded at the same time to get the issue.

The following issues only happen with blendshape mode set to 'normalized'. I don't know if this is a related or new issue or is there a requirement/setup in Godot 4.x compared to 3.x that I am missing to make it work?

• With the 1rst blendshape everything seems to work fine in an otherwise empty test scene.

• With a 2nd blendshape active and a blendvalue between 0.9-1.0 the entire mesh gets insane shadow acne.

• With a 3rd blendshape with value between 0.05-1.0 the entire mesh receives inverted light as if directional light is rotated by 180° degree on the y-axis.

• With a 4rd blendshape value above 0 the entire mesh turns completely black or invisible (depending on the camera angle but even with cull disabled in material).

Blendshape mode 'relative' has no shadow or visibility issues but the giant mesh scale is not very useable.

I tried all available settings with the GLTF exporter (e.g. shapes with/without normal/tangent data) and Godot importer with no success. I also disabled the new mesh LOD system which created some black spots on the mesh but the issue remained.

To make sure the issue is gone in Godot 4.x I think I need to recreate a few parts of my 3.x project setup that deal with loading and mesh replacement as the issue only shows regularly when system is under heavy load but the GDscript 2.0 changes make it a little daunting for me to recreate things at the moment.

[fire]

I noticed that the gltf2 mesh instance 3d blend shape mode should be set to normalized. Changing the sliders by hand is tricky because of wrong scaling of the value interpolation. Try typing 1 and 0.

Also I'm not sure what the result of doing this for the whole body.

I'll try testing with the blend shapes in https://github.com/wonderunit/shot-generator-models

[kayomn]

IMO the notion of having the blend shapes as part of the mesh resource itself should be considered a bug.

Other engines have opted to apply blend shapes via their respective mesh renderer / mesh instance types so that they don't rely on modifying the source mesh itself.