Instanced line rendering for gizmos based on `bevy_polyline` #8427

tim-blackbird · 2023-04-17T19:21:52Z

Objective

Adopt code from bevy_polyline for gizmo line-rendering.
This adds configurable width and perspective rendering for the lines.

Many thanks to @mtsr for the initial work on bevy_polyline. Thanks to @aevyrie for maintaining it, @nicopap for adding the depth_bias feature and the other contributors for squashing bugs and keeping bevy_polyline up-to-date.

Before

After - with line perspective

Line perspective is not on by default because with perspective there is no default line width that works for every scene.

After - without line perspective

Somewhat unexpectedly, the performance is improved with this PR.
At 200,000 lines in many_gizmos I get ~110 FPS on main and ~200 FPS with this PR.
I'm guessing this is a CPU side difference as I would expect the rendering technique to be more expensive on the GPU to some extent, but I am not entirely sure.

Co-authored-by: Jonas Matser <github@jonasmatser.nl> Co-authored-by: Aevyrie <aevyrie@gmail.com> Co-authored-by: Nicola Papale <nico@nicopap.ch>

aevyrie · 2023-04-17T20:06:50Z

Performance is better on my machine up until 220,000 lines in many_gizmos after which FPS suddenly tanks from ~200 down to ~60. Noting this so I can investigate later.

IIRC, you're going to want to batch lines to get really good performance. bevy_polyline can handle many millions of line segments by batching lines with the same material into the same polyline and separating each polyline in the single gpu struct with NaNs. See: ForesightMiningSoftwareCorporation/bevy_polyline#20.

You could probably aggregate all polylines during extract, and shove them into a single GPU uniform, where each set of vertices and colors are separated by a NaN, if you aren't doing this already. This would result in a single draw call for all polylines.

It's also possible bevy_polyline has a much higher ceiling because it doesn't include color information in instance data.

tim-blackbird · 2023-04-17T21:04:19Z

I should have clarified that bevy_gizmos is currently exclusively an immediate mode API.
This means that, unlike with bevy_polyline, those 220,000 lines are being sent every single frame to render them, so it works a little differently (i.e. I'm not expecting the same performance characteristics here).

many millions of line segments

I assumed above that this number is referring to static polylines. Is that right?

You could probably aggregate all polylines during extract, and shove them into a single GPU uniform, where each set of vertices and colors are separated by a NaN, if you aren't doing this already. This would result in a single draw call for all polylines.

I learned the NaN trick from @IceSentry iirc, wouldn't be surprised if they got it from you :)

All lines are split across just 2 batches. One for the NaN separated linestrip topology and one for linelist topology (Used only for single segment lines)
Those 220,000 lines are all in the linelist batch.

It's also possible bevy_polyline has a much higher ceiling because it doesn't include color information in instance data.

I wonder how much this might affect the performance of this implementation. I would like to keep the fancy color gradients q:

aevyrie · 2023-04-17T22:25:57Z

All lines are split across just 2 batches. One for the NaN separated linestrip topology and one for linelist topology (Used only for single segment lines)

Sorry, I didn't see that when I did a brief review of the changes. It's pretty awesome you're already doing this. 🙂

I wonder how much this might affect the performance of this implementation. I would like to keep the fancy color gradients q:

Yeah, it's a nice feature to have. You might have luck by treating it like mesh indices, and only storing colors as a list of indices into the list of colors. My guess is for most cases, this would significantly cut down on the amount of data that needs to be yeeted (yoten?) over to the GPU. You would be reducing memory use for color from 16 bytes ([f32; 4]) per vertex down to 2 bytes (u16). This would allow you to store at most 65k distinct colors across all polylines, which is probably a reasonable limitation (1MB of color data). Though you could have 4 billion colors if you use a u32 as your color index.

Alternatively, you could keep the same approach as now, but use run length encoding to losslessly compress color information for polylines that use a single color. Another upside of that is it would allow you to easily add per-vertex line width and depth bias without blowing up memory usage in most cases.

Anyway, these are obviously some potential optimizations for a future day. This PR is definitely a nice addition.

tim-blackbird · 2023-04-17T22:38:56Z

No need to apologize :) Thanks for all the pointers!

Another upside of that is it would allow you to easily add per-vertex line width and depth bias without blowing up memory usage in most cases.

It's good that you mention this. I was planning to enable more granular control over these settings in a future PR

IceSentry · 2023-04-18T06:13:32Z

This PR is pretty big, is there anything that could be done in a follow up PR in order to keep this one a bit smaller?

I learned the NaN trick from @IceSentry iirc, wouldn't be surprised if they got it from you :)

Yep 😉

tim-blackbird · 2023-04-18T11:02:33Z

This PR is pretty big, is there anything that could be done in a follow up PR in order to keep this one a bit smaller?

This is pretty much the minimal change required to do this line rendering technique. I already stripped down the code a quite a bit compared to bevy_polyline. Especially considering this includes both a 3d and a 2d pipeline.

# Objective Compute the `vertex_count` for indexed meshes as well as non-indexed meshes. I will need this in a future PR based on #8427 that adds a gizmo component that draws the normals of a mesh when attached to an entity ([branch](tim-blackbird/bevy@instanced-line-rendering...devil-ira:bevy:instanced-line-rendering-normals)). <details><summary>Example image</summary> <p> ![image](https://user-images.githubusercontent.com/29694403/233789526-cb5feb47-0aa7-4e69-90a2-e31ec24aadff.png) </p> </details> ## Solution Move `vertex_count` field from `GpuBufferInfo::NonIndexed` to `GpuMesh` ## Migration Guide `vertex_count` is now stored directly on `GpuMesh` instead of `GpuBufferInfo::NonIndexed`.

nicopap

LGTM. You say it's a perf upgrade compared to LineList? That's surprising, but good for us, because that would be amazing to have size (even if joints are still an open question). I'd just like to avoid some magic numbers.

crates/bevy_gizmos/src/lib.rs

nicopap · 2023-04-22T12:16:27Z

crates/bevy_gizmos/src/lib.rs

+struct LineGizmoUniform {
+    line_width: f32,
+    depth_bias: f32,
+    /// WebGL2 structs must be 16 byte aligned.
+    #[cfg(feature = "webgl")]
+    _padding: bevy_math::Vec2,
+}


So only a global line_width and depth_bias is possible? Given you only upload two instances.

Currently, yes. I'll look into more granular configuration in a future PR.

crates/bevy_gizmos/src/lib.rs

crates/bevy_gizmos/src/pipeline_3d.rs

Co-authored-by: Nicola Papale <nicopap@users.noreply.github.com>

hate · 2023-05-31T14:51:16Z

This looks great, the performance increase is a significant one

IceSentry

LGTM, there's a couple of things that could be improved in the future, but it's already in a really nice state for 0.11

alice-i-cecile · 2023-06-12T19:23:08Z

Merging for you: y'all have carefully considered this and have more than enough expertise between you. The code and changes look good from my end too.

nicopap · 2023-06-13T06:25:09Z

Dang merge failed

# Objective Gizmos are intended to draw over everything but for some reason I set the sort key to `0` during #8427 :v I didn't catch this mistake because it still draws over sprites with a Z translation of `0`. ## Solution Set the sort key to `f32::INFINITY`.

tim-blackbird and others added 2 commits April 17, 2023 20:43

Instanced line rendering

7b0bce7

Co-authored-by: Jonas Matser <github@jonasmatser.nl> Co-authored-by: Aevyrie <aevyrie@gmail.com> Co-authored-by: Nicola Papale <nico@nicopap.ch>

Rename thickness to line_width

8118296

alice-i-cecile added A-Rendering Drawing game state to the screen C-Performance A change motivated by improving speed, memory usage or compile times labels Apr 17, 2023

alice-i-cecile requested review from aevyrie, IceSentry and nicopap April 17, 2023 19:35

alice-i-cecile added this to the 0.11 milestone Apr 17, 2023

tim-blackbird added 2 commits April 18, 2023 10:52

cleanup

24f074d

cleanup

ee649d7

tim-blackbird added 4 commits April 18, 2023 20:14

Merge branch 'main' into instanced-line-rendering

593f2de

clippy

324e377

16 byte alignment in webgl

ba681d7

cleanup

1040a6e

tim-blackbird mentioned this pull request Apr 22, 2023

Compute vertex_count for indexed meshes on GpuMesh #8460

Merged

nicopap suggested changes Apr 23, 2023

View reviewed changes

tim-blackbird and others added 2 commits April 23, 2023 21:01

Remove Component derive from asset type

a531b5c

Co-authored-by: Nicola Papale <nicopap@users.noreply.github.com>

Replace magic numbers with calls to VertexFormat::size()

99cedfc

nicopap self-requested a review April 23, 2023 19:41

tim-blackbird added 4 commits April 23, 2023 21:56

Add GizmoLinePipelineKey

3b312a6

Naming pass

3311635

Don't duplicate uniform buffer binding :V

7d3d139

Don't use bevy_utils::default

6d4d283

tim-blackbird added 3 commits June 1, 2023 21:06

Merge branch 'main' into instanced-line-rendering

5caeb27

Fix plugin initialization

78b804b

Merge branch 'main' into instanced-line-rendering

fefaea6

IceSentry approved these changes Jun 11, 2023

View reviewed changes

nicopap added the S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it label Jun 11, 2023

alice-i-cecile added this pull request to the merge queue Jun 12, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 12, 2023

tim-blackbird added 3 commits June 12, 2023 22:00

Merge branch 'main' into instanced-line-rendering

8e9623e

Derive TypePath for LineGizmo

26e2928

Formatting

076b3f5

alice-i-cecile approved these changes Jun 12, 2023

View reviewed changes

alice-i-cecile added this pull request to the merge queue Jun 12, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 12, 2023

mockersf added this pull request to the merge queue Jun 12, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 12, 2023

alice-i-cecile added this pull request to the merge queue Jun 12, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 12, 2023

mockersf added this pull request to the merge queue Jun 13, 2023

Merged via the queue into bevyengine:main with commit 001b3eb Jun 13, 2023

tim-blackbird deleted the instanced-line-rendering branch June 14, 2023 13:22

mockersf mentioned this pull request Jun 21, 2023

2d_gizmos and 3d_gizmos are broke in WebGPU #8908

Closed

This was referenced Jun 28, 2023

News: Release 0.11 (wrong branch ... see #657) bevyengine/bevy-website#656

Closed

News: Release 0.11 bevyengine/bevy-website#657

Merged

tim-blackbird mentioned this pull request Jul 12, 2023

Fix gizmo draw order in 2D #9129

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Instanced line rendering for gizmos based on `bevy_polyline` #8427

Instanced line rendering for gizmos based on `bevy_polyline` #8427

tim-blackbird commented Apr 17, 2023 •

edited

Loading

aevyrie commented Apr 17, 2023 •

edited

Loading

tim-blackbird commented Apr 17, 2023

aevyrie commented Apr 17, 2023 •

edited

Loading

tim-blackbird commented Apr 17, 2023

IceSentry commented Apr 18, 2023

tim-blackbird commented Apr 18, 2023 •

edited

Loading

nicopap left a comment

nicopap Apr 22, 2023

tim-blackbird Apr 23, 2023

hate commented May 31, 2023

IceSentry left a comment

alice-i-cecile commented Jun 12, 2023

nicopap commented Jun 13, 2023

Instanced line rendering for gizmos based on bevy_polyline #8427

Instanced line rendering for gizmos based on bevy_polyline #8427

Conversation

tim-blackbird commented Apr 17, 2023 • edited Loading

Objective

Before

After - with line perspective

aevyrie commented Apr 17, 2023 • edited Loading

tim-blackbird commented Apr 17, 2023

aevyrie commented Apr 17, 2023 • edited Loading

tim-blackbird commented Apr 17, 2023

IceSentry commented Apr 18, 2023

tim-blackbird commented Apr 18, 2023 • edited Loading

nicopap left a comment

Choose a reason for hiding this comment

nicopap Apr 22, 2023

Choose a reason for hiding this comment

tim-blackbird Apr 23, 2023

Choose a reason for hiding this comment

hate commented May 31, 2023

IceSentry left a comment

Choose a reason for hiding this comment

alice-i-cecile commented Jun 12, 2023

nicopap commented Jun 13, 2023

Instanced line rendering for gizmos based on `bevy_polyline` #8427

Instanced line rendering for gizmos based on `bevy_polyline` #8427

tim-blackbird commented Apr 17, 2023 •

edited

Loading

aevyrie commented Apr 17, 2023 •

edited

Loading

aevyrie commented Apr 17, 2023 •

edited

Loading

tim-blackbird commented Apr 18, 2023 •

edited

Loading