KHR_parallax_mapping #2196
Closed
KHR_parallax_mapping #2196
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I too would be interested in such an extension as a way to greatly increase apparent model detail without a large increase in model size/complexity.
The approach in GPU Gems 2 and many other places is not without caveats, but provides PM in a lightweight way:
https://developer.nvidia.com/gpugems/gpugems2/part-i-geometric-complexity/chapter-8-pixel-displacement-mapping-distance-functions
The "Relaxed Cone Stepping for Relief Mapping” in GPU Gems 3 is also interesting as a “cheap” displacement map technique:
https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping
Also interesting is the "Tessellation-Free Displacement Mapping for Ray Tracing” technique presented at SIGGRAPH Asia 2021, although more difficult to implement and not suitable for all backends:
https://perso.telecom-paristech.fr/boubek/papers/TFDM/
Cheers,
Bruce
On Aug 18, 2022, at 8:32 AM, Lars Maier ***@***.******@***.***>> wrote:
Hi, before I continue working on this, I would like to know if there is any chance that this will be accepted.
There is this<#948> thread (5yrs old by the way) that ends up in every google search and people are still asking, but it doesn't look like anything has been done.
The thread also mixes parallax mapping and actual displacement mapping. While I think both is nice to have, I don't think we want to have one extension that does everything. And since I'm more interested in parallax mapping, I went for it.
I'm doing this as a private person, I hope this is not a problem.
Any help is welcome!
…________________________________
You can view, comment on, or merge this pull request online at:
#2196
Commit Summary
* 0f08674<0f08674> First draft.
File Changes
(2 files<https://github.com/KhronosGroup/glTF/pull/2196/files>)
* A extensions/2.0/Khronos/KHR_materials_parallax_mapping/README.md<https://github.com/KhronosGroup/glTF/pull/2196/files#diff-0f8b396663224e5db193eef38ab2355027f243757f7849ae62127dcb679589fb> (56)
* A extensions/2.0/Khronos/KHR_materials_parallax_mapping/schema/parallaxMapping.schema.json<https://github.com/KhronosGroup/glTF/pull/2196/files#diff-6e1f8d3244eea8899049b8fe49458e8384d174728164f7f0d6ceaee85cbc637f> (23)
Patch Links:
* https://github.com/KhronosGroup/glTF/pull/2196.patch
* https://github.com/KhronosGroup/glTF/pull/2196.diff
—
Reply to this email directly, view it on GitHub<#2196>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACPU2VBOBKEUU6IUSF7ZSF3VZY3QVANCNFSM565HIFHQ>.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
|
Thanks for providing additional resources. |
|
Almost 3 weeks later. How do we proceed with this pull request? @UX3D-nopper is there an internal discussion about this? Is there anything I can provide to you? |
|
There is a dedicated group at Khronos, called PBR TSG, which is taking care of material extensions. As there are currently so much pull requests and suggestions, this will probably take a while. |
|
Thanks for your response! Then I will be patient. 🙂 |
|
I'd just add — one helpful way to contribute on proposed extensions can be to research the relevant feature support in (1) authoring tools, and (2) viewing engines. Standards are typically not designed "from scratch". Knowing the similarities and differences among popular existing or upcoming software helps to move things forward. |
|
Engine Support: In fact, if an engine provides support for custom shaders, parallax occlusion mapping is only limited by the rendering backend. Authoring Tools:
I think, that this is a common technique and is widely supported. |
nicopap
reviewed
Feb 25, 2023
Comment on lines
+26
to
+27
| The height at a given position is calculated by sampling the given texture and using the red channel. The sampled value is then multiplied with the `heightFactor`. | ||
| The result is used in an implementation defined way to offset all other texture lookups. (See Resources) |
There was a problem hiding this comment.
It might be worthwhile to specify the direction of the offset. (is white the topmost height or bottommost height?) Afaik Godot has white goes down, and calls the texture a "depth map"
There was a problem hiding this comment.
Good point! I will add that. Would it make sense to fix a certain behavior (i.e. 0 is lowest, 1 is highest) or have it configurable. Translating one to the other is trivial.
Alternatively, one could allow the heightFactor to be negative.
Not sure what is best. Any opinions/ideas?
There was a problem hiding this comment.
I spent the two last days on the question lol. Given it's a height map, of course white should be top (when height is at high value, it should be tall etc.). This goes against Godot's implementation (my own, and I imagine most people's), but it's just a question of doing 1.0 - height in the shader, which I doubt would introduce an unacceptable overhead, especially given that parallax mapping algorithm oftentimes sample multiple times at different positions the height map per fragment.
The alternative would be to call it depth_map. I actually think this might be worth considering. Given that parallax mapping oftentimes (at least unity/unreal/godot) insets the depth, rather that "push out". However, this depends on the implementation. Hull-based parallax mapping algorithms do push out the fake geometry, and would warrant the name height_map.
|
As #948 can attest, there is a lot of interest for the addition of height maps to the standard PBR material. As noted earlier in this discussion, parallax mapping is widely supported in existing engines, furthermore, modelling tools like Blender are well capable of exporting height maps (in any case, if a tool is capable of exporting a normal map, it is likely capable of exporting a height map), so the implementation burden for API consumers seem minimal. Some additional context from #948:
Note that all the questions around the materials_displacement extension was about the specific displacement mapping implementation which is not relevant for this proposition. Here, we are talking about parallax mapping, a technique that doesn't require any additional geometry. If the gltf rust crate were to support this as an extension, would it be enough of a proof of concept to start turning the gears of getting this into the standard? |
|
I've looked in depth at mmoeller's implementation for Blender and I have one additional comment:
|
nicopap
added a commit
to nicopap/bevy
that referenced
this pull request
Apr 2, 2023
Add a [parallax mapping] shader to bevy. Please note that this is a 3d technique, NOT a 2d sidescroller feature. - Add related fields to `StandardMaterial` - update the pbr shader - Add an example taking advantage of parallax mapping A pre-existing implementation exists at: https://github.com/nicopap/bevy_mod_paramap/ The implementation is derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28 Further discussion on literature is found in the `bevy_mod_paramap` README. Limitations ----------- - The mesh silhouette isn't affected by the depth map. - The depth of the pixel does not reflect its visual position, resulting in artifacts for depth-dependent features such as fog or SSAO - GLTF does not define a height map texture, so somehow the user will always need to work around this limitation, though [an extension is in the works][gltf] Future work ----------- - It's possible to update the depth in the depth buffer to follow the parallaxed texture. This would enable interop with depth-based visual effects, it also allows `discard`ing pixels of materials when computed depth is higher than the one in depth buffer - Cheap lower quality single-sample method using [offset limiting] - Add distance fading, to disable parallaxing (relatively expensive) on distant objects - GLTF extension to allow defining height maps. Or a workaround implemented through a blender plugin to the GLTF exporter that uses the `extras` field to add height map. - [Quadratic surface vertex attributes][oliveira_3] to enable parallax mapping on bending surfaces and allow clean silhouetting. - noise based sampling, to limit the pancake artifacts. - Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires preprocessing, increase depth map size, reduces sample count greatly. - [Quadtree parallax mapping][qpm] (also requires preprocessing) - Self-shadowing of parallax-mapped surfaces by modifying the shadow map https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4 --- - Add a `depth_map` field to the `StandardMaterial`, it is a greyscale image where white represents bottom and black the top. If `depth_map` is set, bevy's pbr shader will use it to do [parallax mapping] to give an increased feel of depth to the material. This is similar to a displacement map, but with infinite precision at fairly low cost. - The fields `parallax_mapping_method`, `parallax_depth` and `max_parallax_layer_count` allow finer grained control over the behavior of the parallax shader. - Add the `parallax_mapping` example to show off the effect. [parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping [oliveira_3]: https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf [GPU gems]: https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping [simcity]: https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/ [offset limiting]: https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf [gltf]: KhronosGroup/glTF#2196 [qpm]: https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf Co-authored-by: Robert Swain <robert.swain@gmail.com>
nicopap
added a commit
to nicopap/bevy
that referenced
this pull request
Apr 2, 2023
Add a [parallax mapping] shader to bevy. Please note that this is a 3d technique, NOT a 2d sidescroller feature. - Add related fields to `StandardMaterial` - update the pbr shader - Add an example taking advantage of parallax mapping A pre-existing implementation exists at: https://github.com/nicopap/bevy_mod_paramap/ The implementation is derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28 Further discussion on literature is found in the `bevy_mod_paramap` README. Limitations ----------- - The mesh silhouette isn't affected by the depth map. - The depth of the pixel does not reflect its visual position, resulting in artifacts for depth-dependent features such as fog or SSAO - GLTF does not define a height map texture, so somehow the user will always need to work around this limitation, though [an extension is in the works][gltf] Future work ----------- - It's possible to update the depth in the depth buffer to follow the parallaxed texture. This would enable interop with depth-based visual effects, it also allows `discard`ing pixels of materials when computed depth is higher than the one in depth buffer - Cheap lower quality single-sample method using [offset limiting] - Add distance fading, to disable parallaxing (relatively expensive) on distant objects - GLTF extension to allow defining height maps. Or a workaround implemented through a blender plugin to the GLTF exporter that uses the `extras` field to add height map. - [Quadratic surface vertex attributes][oliveira_3] to enable parallax mapping on bending surfaces and allow clean silhouetting. - noise based sampling, to limit the pancake artifacts. - Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires preprocessing, increase depth map size, reduces sample count greatly. - [Quadtree parallax mapping][qpm] (also requires preprocessing) - Self-shadowing of parallax-mapped surfaces by modifying the shadow map https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4 --- - Add a `depth_map` field to the `StandardMaterial`, it is a greyscale image where white represents bottom and black the top. If `depth_map` is set, bevy's pbr shader will use it to do [parallax mapping] to give an increased feel of depth to the material. This is similar to a displacement map, but with infinite precision at fairly low cost. - The fields `parallax_mapping_method`, `parallax_depth` and `max_parallax_layer_count` allow finer grained control over the behavior of the parallax shader. - Add the `parallax_mapping` example to show off the effect. [parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping [oliveira_3]: https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf [GPU gems]: https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping [simcity]: https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/ [offset limiting]: https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf [gltf]: KhronosGroup/glTF#2196 [qpm]: https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf Co-authored-by: Robert Swain <robert.swain@gmail.com>
nicopap
added a commit
to nicopap/bevy
that referenced
this pull request
Apr 2, 2023
Add a [parallax mapping] shader to bevy. Please note that this is a 3d technique, NOT a 2d sidescroller feature. - Add related fields to `StandardMaterial` - update the pbr shader - Add an example taking advantage of parallax mapping A pre-existing implementation exists at: https://github.com/nicopap/bevy_mod_paramap/ The implementation is derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28 Further discussion on literature is found in the `bevy_mod_paramap` README. Limitations ----------- - The mesh silhouette isn't affected by the depth map. - The depth of the pixel does not reflect its visual position, resulting in artifacts for depth-dependent features such as fog or SSAO - GLTF does not define a height map texture, so somehow the user will always need to work around this limitation, though [an extension is in the works][gltf] Future work ----------- - It's possible to update the depth in the depth buffer to follow the parallaxed texture. This would enable interop with depth-based visual effects, it also allows `discard`ing pixels of materials when computed depth is higher than the one in depth buffer - Cheap lower quality single-sample method using [offset limiting] - Add distance fading, to disable parallaxing (relatively expensive) on distant objects - GLTF extension to allow defining height maps. Or a workaround implemented through a blender plugin to the GLTF exporter that uses the `extras` field to add height map. - [Quadratic surface vertex attributes][oliveira_3] to enable parallax mapping on bending surfaces and allow clean silhouetting. - noise based sampling, to limit the pancake artifacts. - Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires preprocessing, increase depth map size, reduces sample count greatly. - [Quadtree parallax mapping][qpm] (also requires preprocessing) - Self-shadowing of parallax-mapped surfaces by modifying the shadow map https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4 --- - Add a `depth_map` field to the `StandardMaterial`, it is a greyscale image where white represents bottom and black the top. If `depth_map` is set, bevy's pbr shader will use it to do [parallax mapping] to give an increased feel of depth to the material. This is similar to a displacement map, but with infinite precision at fairly low cost. - The fields `parallax_mapping_method`, `parallax_depth` and `max_parallax_layer_count` allow finer grained control over the behavior of the parallax shader. - Add the `parallax_mapping` example to show off the effect. [parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping [oliveira_3]: https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf [GPU gems]: https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping [simcity]: https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/ [offset limiting]: https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf [gltf]: KhronosGroup/glTF#2196 [qpm]: https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf Co-authored-by: Robert Swain <robert.swain@gmail.com>
nicopap
added a commit
to nicopap/bevy
that referenced
this pull request
Apr 2, 2023
Add a [parallax mapping] shader to bevy. Please note that this is a 3d technique, NOT a 2d sidescroller feature. - Add related fields to `StandardMaterial` - update the pbr shader - Add an example taking advantage of parallax mapping A pre-existing implementation exists at: https://github.com/nicopap/bevy_mod_paramap/ The implementation is derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28 Further discussion on literature is found in the `bevy_mod_paramap` README. Limitations ----------- - The mesh silhouette isn't affected by the depth map. - The depth of the pixel does not reflect its visual position, resulting in artifacts for depth-dependent features such as fog or SSAO - GLTF does not define a height map texture, so somehow the user will always need to work around this limitation, though [an extension is in the works][gltf] Future work ----------- - It's possible to update the depth in the depth buffer to follow the parallaxed texture. This would enable interop with depth-based visual effects, it also allows `discard`ing pixels of materials when computed depth is higher than the one in depth buffer - Cheap lower quality single-sample method using [offset limiting] - Add distance fading, to disable parallaxing (relatively expensive) on distant objects - GLTF extension to allow defining height maps. Or a workaround implemented through a blender plugin to the GLTF exporter that uses the `extras` field to add height map. - [Quadratic surface vertex attributes][oliveira_3] to enable parallax mapping on bending surfaces and allow clean silhouetting. - noise based sampling, to limit the pancake artifacts. - Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires preprocessing, increase depth map size, reduces sample count greatly. - [Quadtree parallax mapping][qpm] (also requires preprocessing) - Self-shadowing of parallax-mapped surfaces by modifying the shadow map https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4 --- - Add a `depth_map` field to the `StandardMaterial`, it is a greyscale image where white represents bottom and black the top. If `depth_map` is set, bevy's pbr shader will use it to do [parallax mapping] to give an increased feel of depth to the material. This is similar to a displacement map, but with infinite precision at fairly low cost. - The fields `parallax_mapping_method`, `parallax_depth` and `max_parallax_layer_count` allow finer grained control over the behavior of the parallax shader. - Add the `parallax_mapping` example to show off the effect. [parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping [oliveira_3]: https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf [GPU gems]: https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping [simcity]: https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/ [offset limiting]: https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf [gltf]: KhronosGroup/glTF#2196 [qpm]: https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf Co-authored-by: Robert Swain <robert.swain@gmail.com>
nicopap
added a commit
to nicopap/bevy
that referenced
this pull request
Apr 2, 2023
Add a [parallax mapping] shader to bevy. Please note that this is a 3d technique, NOT a 2d sidescroller feature. - Add related fields to `StandardMaterial` - update the pbr shader - Add an example taking advantage of parallax mapping A pre-existing implementation exists at: https://github.com/nicopap/bevy_mod_paramap/ The implementation is derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28 Further discussion on literature is found in the `bevy_mod_paramap` README. Limitations ----------- - The mesh silhouette isn't affected by the depth map. - The depth of the pixel does not reflect its visual position, resulting in artifacts for depth-dependent features such as fog or SSAO - GLTF does not define a height map texture, so somehow the user will always need to work around this limitation, though [an extension is in the works][gltf] Future work ----------- - It's possible to update the depth in the depth buffer to follow the parallaxed texture. This would enable interop with depth-based visual effects, it also allows `discard`ing pixels of materials when computed depth is higher than the one in depth buffer - Cheap lower quality single-sample method using [offset limiting] - Add distance fading, to disable parallaxing (relatively expensive) on distant objects - GLTF extension to allow defining height maps. Or a workaround implemented through a blender plugin to the GLTF exporter that uses the `extras` field to add height map. - [Quadratic surface vertex attributes][oliveira_3] to enable parallax mapping on bending surfaces and allow clean silhouetting. - noise based sampling, to limit the pancake artifacts. - Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires preprocessing, increase depth map size, reduces sample count greatly. - [Quadtree parallax mapping][qpm] (also requires preprocessing) - Self-shadowing of parallax-mapped surfaces by modifying the shadow map https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4 --- - Add a `depth_map` field to the `StandardMaterial`, it is a greyscale image where white represents bottom and black the top. If `depth_map` is set, bevy's pbr shader will use it to do [parallax mapping] to give an increased feel of depth to the material. This is similar to a displacement map, but with infinite precision at fairly low cost. - The fields `parallax_mapping_method`, `parallax_depth` and `max_parallax_layer_count` allow finer grained control over the behavior of the parallax shader. - Add the `parallax_mapping` example to show off the effect. [parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping [oliveira_3]: https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf [GPU gems]: https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping [simcity]: https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/ [offset limiting]: https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf [gltf]: KhronosGroup/glTF#2196 [qpm]: https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf Co-authored-by: Robert Swain <robert.swain@gmail.com>
nicopap
added a commit
to nicopap/bevy
that referenced
this pull request
Apr 10, 2023
Add a [parallax mapping] shader to bevy. Please note that this is a 3d technique, NOT a 2d sidescroller feature. - Add related fields to `StandardMaterial` - update the pbr shader - Add an example taking advantage of parallax mapping A pre-existing implementation exists at: https://github.com/nicopap/bevy_mod_paramap/ The implementation is derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28 Further discussion on literature is found in the `bevy_mod_paramap` README. Limitations ----------- - The mesh silhouette isn't affected by the depth map. - The depth of the pixel does not reflect its visual position, resulting in artifacts for depth-dependent features such as fog or SSAO - GLTF does not define a height map texture, so somehow the user will always need to work around this limitation, though [an extension is in the works][gltf] Future work ----------- - It's possible to update the depth in the depth buffer to follow the parallaxed texture. This would enable interop with depth-based visual effects, it also allows `discard`ing pixels of materials when computed depth is higher than the one in depth buffer - Cheap lower quality single-sample method using [offset limiting] - Add distance fading, to disable parallaxing (relatively expensive) on distant objects - GLTF extension to allow defining height maps. Or a workaround implemented through a blender plugin to the GLTF exporter that uses the `extras` field to add height map. - [Quadratic surface vertex attributes][oliveira_3] to enable parallax mapping on bending surfaces and allow clean silhouetting. - noise based sampling, to limit the pancake artifacts. - Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires preprocessing, increase depth map size, reduces sample count greatly. - [Quadtree parallax mapping][qpm] (also requires preprocessing) - Self-shadowing of parallax-mapped surfaces by modifying the shadow map https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4 --- - Add a `depth_map` field to the `StandardMaterial`, it is a greyscale image where white represents bottom and black the top. If `depth_map` is set, bevy's pbr shader will use it to do [parallax mapping] to give an increased feel of depth to the material. This is similar to a displacement map, but with infinite precision at fairly low cost. - The fields `parallax_mapping_method`, `parallax_depth` and `max_parallax_layer_count` allow finer grained control over the behavior of the parallax shader. - Add the `parallax_mapping` example to show off the effect. [parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping [oliveira_3]: https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf [GPU gems]: https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping [simcity]: https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/ [offset limiting]: https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf [gltf]: KhronosGroup/glTF#2196 [qpm]: https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf Co-authored-by: Robert Swain <robert.swain@gmail.com>
nicopap
added a commit
to nicopap/bevy
that referenced
this pull request
Apr 12, 2023
Add a [parallax mapping] shader to bevy. Please note that this is a 3d technique, NOT a 2d sidescroller feature. - Add related fields to `StandardMaterial` - update the pbr shader - Add an example taking advantage of parallax mapping A pre-existing implementation exists at: https://github.com/nicopap/bevy_mod_paramap/ The implementation is derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28 Further discussion on literature is found in the `bevy_mod_paramap` README. Limitations ----------- - The mesh silhouette isn't affected by the depth map. - The depth of the pixel does not reflect its visual position, resulting in artifacts for depth-dependent features such as fog or SSAO - GLTF does not define a height map texture, so somehow the user will always need to work around this limitation, though [an extension is in the works][gltf] Future work ----------- - It's possible to update the depth in the depth buffer to follow the parallaxed texture. This would enable interop with depth-based visual effects, it also allows `discard`ing pixels of materials when computed depth is higher than the one in depth buffer - Cheap lower quality single-sample method using [offset limiting] - Add distance fading, to disable parallaxing (relatively expensive) on distant objects - GLTF extension to allow defining height maps. Or a workaround implemented through a blender plugin to the GLTF exporter that uses the `extras` field to add height map. - [Quadratic surface vertex attributes][oliveira_3] to enable parallax mapping on bending surfaces and allow clean silhouetting. - noise based sampling, to limit the pancake artifacts. - Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires preprocessing, increase depth map size, reduces sample count greatly. - [Quadtree parallax mapping][qpm] (also requires preprocessing) - Self-shadowing of parallax-mapped surfaces by modifying the shadow map https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4 --- - Add a `depth_map` field to the `StandardMaterial`, it is a greyscale image where white represents bottom and black the top. If `depth_map` is set, bevy's pbr shader will use it to do [parallax mapping] to give an increased feel of depth to the material. This is similar to a displacement map, but with infinite precision at fairly low cost. - The fields `parallax_mapping_method`, `parallax_depth` and `max_parallax_layer_count` allow finer grained control over the behavior of the parallax shader. - Add the `parallax_mapping` example to show off the effect. [parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping [oliveira_3]: https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf [GPU gems]: https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping [simcity]: https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/ [offset limiting]: https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf [gltf]: KhronosGroup/glTF#2196 [qpm]: https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf Co-authored-by: Robert Swain <robert.swain@gmail.com>
superdump
added a commit
to bevyengine/bevy
that referenced
this pull request
Apr 12, 2023
# Objective Add a [parallax mapping] shader to bevy. Please note that this is a 3d technique, NOT a 2d sidescroller feature. ## Solution - Add related fields to `StandardMaterial` - update the pbr shader - Add an example taking advantage of parallax mapping A pre-existing implementation exists at: https://github.com/nicopap/bevy_mod_paramap/ The implementation is derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28 Further discussion on literature is found in the `bevy_mod_paramap` README. ### Limitations - The mesh silhouette isn't affected by the depth map. - The depth of the pixel does not reflect its visual position, resulting in artifacts for depth-dependent features such as fog or SSAO - GLTF does not define a height map texture, so somehow the user will always need to work around this limitation, though [an extension is in the works][gltf] ### Future work - It's possible to update the depth in the depth buffer to follow the parallaxed texture. This would enable interop with depth-based visual effects, it also allows `discard`ing pixels of materials when computed depth is higher than the one in depth buffer - Cheap lower quality single-sample method using [offset limiting] - Add distance fading, to disable parallaxing (relatively expensive) on distant objects - GLTF extension to allow defining height maps. Or a workaround implemented through a blender plugin to the GLTF exporter that uses the `extras` field to add height map. - [Quadratic surface vertex attributes][oliveira_3] to enable parallax mapping on bending surfaces and allow clean silhouetting. - noise based sampling, to limit the pancake artifacts. - Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires preprocessing, increase depth map size, reduces sample count greatly. - [Quadtree parallax mapping][qpm] (also requires preprocessing) - Self-shadowing of parallax-mapped surfaces by modifying the shadow map - Generate depth map from normal map [link to slides], [blender question] https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4 [blender question]: https://blender.stackexchange.com/questions/89278/how-to-get-a-smooth-curvature-map-from-a-normal-map [link to slides]: https://developer.download.nvidia.com/assets/gamedev/docs/nmap2displacement.pdf [oliveira_3]: https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf [GPU gems]: https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping [simcity]: https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/ [offset limiting]: https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf [gltf]: KhronosGroup/glTF#2196 [qpm]: https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf --- ## Changelog - Add a `depth_map` field to the `StandardMaterial`, it is a grayscale image where white represents bottom and black the top. If `depth_map` is set, bevy's pbr shader will use it to do [parallax mapping] to give an increased feel of depth to the material. This is similar to a displacement map, but with infinite precision at fairly low cost. - The fields `parallax_mapping_method`, `parallax_depth_scale` and `max_parallax_layer_count` allow finer grained control over the behavior of the parallax shader. - Add the `parallax_mapping` example to show off the effect. [parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping --------- Co-authored-by: Robert Swain <robert.swain@gmail.com>
superdump
added a commit
to bevyengine/bevy
that referenced
this pull request
Apr 15, 2023
# Objective Add a [parallax mapping] shader to bevy. Please note that this is a 3d technique, NOT a 2d sidescroller feature. ## Solution - Add related fields to `StandardMaterial` - update the pbr shader - Add an example taking advantage of parallax mapping A pre-existing implementation exists at: https://github.com/nicopap/bevy_mod_paramap/ The implementation is derived from: https://web.archive.org/web/20150419215321/http://sunandblackcat.com/tipFullView.php?l=eng&topicid=28 Further discussion on literature is found in the `bevy_mod_paramap` README. ### Limitations - The mesh silhouette isn't affected by the depth map. - The depth of the pixel does not reflect its visual position, resulting in artifacts for depth-dependent features such as fog or SSAO - GLTF does not define a height map texture, so somehow the user will always need to work around this limitation, though [an extension is in the works][gltf] ### Future work - It's possible to update the depth in the depth buffer to follow the parallaxed texture. This would enable interop with depth-based visual effects, it also allows `discard`ing pixels of materials when computed depth is higher than the one in depth buffer - Cheap lower quality single-sample method using [offset limiting] - Add distance fading, to disable parallaxing (relatively expensive) on distant objects - GLTF extension to allow defining height maps. Or a workaround implemented through a blender plugin to the GLTF exporter that uses the `extras` field to add height map. - [Quadratic surface vertex attributes][oliveira_3] to enable parallax mapping on bending surfaces and allow clean silhouetting. - noise based sampling, to limit the pancake artifacts. - Cone mapping ([GPU gems], [Simcity (2013)][simcity]). Requires preprocessing, increase depth map size, reduces sample count greatly. - [Quadtree parallax mapping][qpm] (also requires preprocessing) - Self-shadowing of parallax-mapped surfaces by modifying the shadow map - Generate depth map from normal map [link to slides], [blender question] https://user-images.githubusercontent.com/26321040/223563792-dffcc6ab-70e8-4ff9-90d1-b36c338695ad.mp4 [blender question]: https://blender.stackexchange.com/questions/89278/how-to-get-a-smooth-curvature-map-from-a-normal-map [link to slides]: https://developer.download.nvidia.com/assets/gamedev/docs/nmap2displacement.pdf [oliveira_3]: https://www.inf.ufrgs.br/~oliveira/pubs_files/Oliveira_Policarpo_RP-351_Jan_2005.pdf [GPU gems]: https://developer.nvidia.com/gpugems/gpugems3/part-iii-rendering/chapter-18-relaxed-cone-stepping-relief-mapping [simcity]: https://community.simtropolis.com/omnibus/other-games/building-and-rendering-simcity-2013-r247/ [offset limiting]: https://raw.githubusercontent.com/marcusstenbeck/tncg14-parallax-mapping/master/documents/Parallax%20Mapping%20with%20Offset%20Limiting%20-%20A%20Per-Pixel%20Approximation%20of%20Uneven%20Surfaces.pdf [gltf]: KhronosGroup/glTF#2196 [qpm]: https://www.gamedevs.org/uploads/quadtree-displacement-mapping-with-height-blending.pdf --- ## Changelog - Add a `depth_map` field to the `StandardMaterial`, it is a grayscale image where white represents bottom and black the top. If `depth_map` is set, bevy's pbr shader will use it to do [parallax mapping] to give an increased feel of depth to the material. This is similar to a displacement map, but with infinite precision at fairly low cost. - The fields `parallax_mapping_method`, `parallax_depth_scale` and `max_parallax_layer_count` allow finer grained control over the behavior of the parallax shader. - Add the `parallax_mapping` example to show off the effect. [parallax mapping]: https://en.wikipedia.org/wiki/Parallax_mapping --------- Co-authored-by: Robert Swain <robert.swain@gmail.com>
9 tasks
|
Another 1.5 yrs later. Does anyone know if there is something going on inside khronos with respect to parallax mapping? Has this proposal or any variant of it been considered? |
|
Closed because stale and no interested from KHR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi, before I continue working on this, I would like to know if there is any chance that this will be accepted.
There is this thread (5yrs old by the way) that ends up in every google search and people are still asking, but it doesn't look like anything has been done.
The thread also mixes parallax mapping and actual displacement mapping. While I think both is nice to have, I don't think we want to have one extension that does everything. And since I'm more interested in parallax mapping, I went for it.
I'm doing this without being attached to any company, I hope this is not a problem.
Any help is welcome!