drawing layer by layer

[ansis]

This could open up interesting possibilities like rendering some layers from a child tile and some from a parent tile, and maybe even cross-fade the two. We could then render most features from a tile without waiting for text placement. Together with streaming downloads we could potentially start rendering layers before the entire tile is downloaded.

We could also do compositing operations on data from different sources. For example, a satellite layer that is desaturated everywhere except parks.

I hackishly implemented this way back, and it was slightly slower (40 vs 42fps or something).

[kkaefer]

We could then render most features from a tile without waiting for text placement.

Can you elaborate on that? We could do the same today; just by deferring label placement to the last step and sending tile loaded signals with the current state back to the main thread.

streaming downloads

I've thought about those too, but since we're using the default browser XHR2, we can't access partially loaded data. WebSockets support streaming data, but that requires server/CDN support.

[ansis]

Things get more complicated when you are drawing on top of an overscaled tile, not on a blank canvas. But I think you're right, we could accomplish this just by toggling which layers of layer to draw and ordering tiles right, as long as text is always on top of other layers.

#115 and #194 are relavent

since we're using the default browser XHR2, we can't access partially loaded data.

I'm not 100% sure on this, but I think you can access partially loaded data with xhr. For example, https://github.com/tmcw/leaflet-geojson-stream

[ansis]

I think I might be misunderstanding whats going on there. Maybe @tmcw can clarify whether streaming xhr is actually possible

[tmcw]

Yes, it's possible - hyperquest wraps it (and takes up a lot of space), but you can do the same without the abstraction just by listening to readystatechange events with readyState of 3

[kkaefer]

Oh wow, didn't know about this feature. That's what happens when you don't know that something is not working ;)

[jfirebaugh]

I've rebooted @ansis's old branch in layer-by-layer and went a little further by starting to push the loop over tiles down into the draw functions, which avoids redundant shader switching/rebinds/uniform updating without needing to explicitly track all state (#145).

@kkaefer @ansis @mourner Does this look like a good approach?

[mourner]

@jfirebaugh looks good to me! Any idea about how it affects performance?

[jfirebaugh]

So far I haven't gotten any meaningful performance data -- both before and after are rendering at 60fps in the areas I tested. What do you do for performance testing?

For layer-by-layer rendering, it looks like efficient shader switching depends on the technique native uses for tile clipping, where each visible tile gets a unique ID which is written to the stencil buffer. That allows the stencil buffer to be written once, before rendering layers. Right now, js is using a single stencil buffer bit for clipping, which requires lots of shader switches when rendering layer-by-layer (switching between the fill shader used to draw the tile clipping stencil, and the shader used by the actual layer).

In turn, that clipping technique requires not using those stencil buffer bits for the fill rendering trick, i.e. tessellating polygons.

[jfirebaugh]

The increased shader switching is definitely harming performance compared to tile-by-tile rendering. In the default DC view, its smooth 60 fps tile-by-tile and janky 30-40 fps layer-by-layer.

[kkaefer]

It should be the other way around; when you draw layer-by-layer, there shouldn't be any shader switching for all tiles in the view. Curious whether there are any other effects.

[jfirebaugh]

@kkaefer Right, it's doing more shader switching now because it's not using unique-per-tile values in the stencil buffer for clipping, because it's using the stencil buffer bits for drawing fills (see my previous comment). Essentially, tessellating polygons is a prerequisite for drawing layer by layer.

[kkaefer]

Ah! makes sense.

[fnicollet]

Hello everyone,

Is somebody working on this issue? It is currently causing issues in production (see mapbox's example pages) and seem to be the source of a few other issues, at least #757 and #810.
There seem to be a branch by @ansis and @jfirebaugh but they look like discontinued. Does it involve big changes? Is there any way I can help fix this?

Thanks for your help

[jfirebaugh]

It's blocked by #682, which is blocked by mapbox/mapnik-vector-tile#53.

[lucaswoj]

Sounds like this is getting unblocked as #682 gets ready to land in the next few weeks. I'm interested to see how much work it'll take to rebase this on earcut-final and get it working. Thoughts on priority @jfirebaugh @mourner @ansis?

[jfirebaugh]

This isn't actually blocked by earcut. See #1197 for current status. Closing here to avoid future confusion.

[mourner]

This isn't actually blocked by earcut.

But with earcut in place, will we still need depth buffer support in headless-gl for tests?

[jfirebaugh]

@mourner Do you mean stencil buffer? We definitely need depth buffer for lots of things. And stencil buffer is also needed as long as some draw operations are clipped to tile bounds.

[mourner]

@jfirebaugh OK, so we're still blocked by stackgl/headless-gl#26, right? Maybe it would be worth disabling render tests on JS for some time and merging layer-by-layer just to unblock (would running render tests locally work with headless-gl)?