Support displacement mapping via shaders

[janhohenheim]

I'm no expert on this, but we should be able to shift a vertex in the vertex shader by the displacement map's level in the normal direction

[nicopap]

Depending on your usecase, you might be interested in parallax mapping[1] (POM).

Usecase:

• 🟥 Terrain height map: do not use POM for height map, in this case, it is indeed better to use a vertex shader, or even use a dedicated terrain plugin[2] for very large maps.
• 🟩 Fancy textures: POM is good for stuff like cobblestone, rocks, brickwork. It's like normal mapping, but with actual depth.

links:

• [1] https://github.com/nicopap/bevy_mod_paramap/
• [2] https://github.com/kurtkuehnert/bevy_terrain

[janhohenheim]

Thanks for the suggestion, your crate looks perfect for my use-case! Is there a way to automatically use a displacement map that is included in a GLTF, like how StandardMaterial imports normal and roughness maps from the GLTF?

[nicopap]

Annoyingly, the base GLTF spec doesn't include an height map (or bump map) https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#reference-material. There is a proposal to extend the standard with such a texture here: KhronosGroup/glTF#2196. though it does look like a lot of work is needed to get it merged.

Furthermore, it is bevy that loads the GLTF, so we are stuck with whatever material & textures it loads. I think generating a height map based on the normal map is possible (see the TODO section of the README) But doing that at runtime is a bit beyond what I find acceptable in terms of performance. This is where a bevy asset preprocessing pipeline would be useful (planned for 0.11 I think)

[janhohenheim]

@nicopap do I see it right that bevyengine/bevy#5928 will do most of the lifting for this in Bevy 0.11? :)
Altough it seems the GLTF extension did not land yet, so the height maps have to be loaded at runtime, right?

[nicopap]

So if by "at runtime" you mean "in user code" the answer currently is "yes". As long as we don't somehow push the depth map (for parallax mapping, we should talk about "depth map" over height map) into the gltf model, we'll have to somehow tell bevy to load height maps. Currently the only way to tell bevy anything is through code. cart has been talking about "meta files" for the future asset pipeline which could also fulfills this role.

Though, time to plug my normal2depth map plugin. 🚀 If your gltf comes with a normal map but not a depth map, you can now generate it! Nothing easier, use the bevy_mod_n2dmap crate and call standard_material.generate_depth_map(&mut images_asset) and now your material should have a depth map, enabling directly parallax mapping on it. https://github.com/devildahu/n2dmap 🚀

[janhohenheim]

Woah, that crate sounds really useful, thanks! I'll be sure to use it once Bevy 0.11 rolls out.

[nicopap]

well, I've just started testing it and it doesn't work quite well yet. 😅 So I'd reserve judgement until it's complete.

[janhohenheim]

I decided that this is out of scope for the template