deck.gl layers were designed to be used with a Reactive programming paradigm. The challenge is of course that in the "reactive" model, every change to application state causes a full re-render, while in WebGL, potentially huge memory buffers describing the geometry (so called "vertex attributes", or just "attributes" for short) must be prepared in advance of any draw calls.
Creating new WebGL buffers before every draw call would quickly lead to unacceptable performance. Just like in React (which "renders" to the browser's slow-updating DOM) the challenge becomes to detect exactly what changes were made to each layer's properties and then limit both attribute recalculation and re-rendering to the minimum needed to ensure that the next draw cycle correctly reflects the changes.
Since the length of attributes are usually proportional of to the number of
data elements being visualized (hundreds of thousands or even millions of
elements are not uncommon in big data visualizations), efficient attribute
updates is critical. It is such an important use case that deck.gl provides an
AttributeManager class to simplify layer creation.
Note that it is possible for a layer to use custom code to manage attribute updates, however most layers rely on the AttributeManager class.
Automated attribute generation and management is suitable when a set of vertex shader attributes are generated by iteration over a data array, and updates to these attributes are needed either when the data itself changes, or when other data relevant to the calculations change.
- First the application registers descriptions of its dynamic vertex attributes using AttributeManager.add().
- Then, when any change that affects attributes is detected by the application, the app will call AttributeManager.invalidate().
- Finally before it renders, it calls AttributeManager.update() to ensure that attributes are automatically rebuilt if anything has been invalidated.
The application provided update functions describe how attributes should be updated from a data array and are expected to traverse that data array (or iterable) and fill in the attribute's typed array.
Note that the attribute manager intentionally does not do advanced change detection, but instead makes it easy to build such detection by offering the ability to "invalidate" each attribute separately.
The layer will expect each object to provide a number of "attributes" that it can use to set the GL buffers. By default, the layer will look for these attributes to be available as fields directly on the objects during iteration over the supplied data set. To gain more control of attribute access and/or to do on-the-fly calculation of attributes.
Note: A layer only renders when a property change is detected. For
performance reasons, property change detection uses shallow compare,
which means that mutating an element inside a buffer or a mutable data array
does not register as a property change, and thus does not trigger a re-render.
To force trigger a render after mutating buffers, simply increment the
renderCount property. To force trigger a buffer update after mutating data,
increment the updateCount property.
The AttributeManager class automated attribute allocation and updates.
Summary:
- keeps track of valid state for each attribute
- auto reallocates attributes when needed
- auto updates attributes with registered updater functions
- allows overriding with application supplied buffers
Limitations:
- The AttributeManager always reinitializes the entire typed array. There are currently no provisions for only invalidating a range of indices in an attribute.
- opts (Object) - named parameters
- opts.id (String, not required) - identifier (for debugging)
Adds attribute descriptions to the AttributeManager that describe the attributes that should be auto-calculated.
Takes a map of attribute descriptor objects
- keys are attribute names
- values are objects with attribute fields
-
attribute.size - number of elements per object
-
attribute.updater - number of elements
-
attribute.instanced=0 - is this is an instanced attribute (a.k.a. divisor)
-
attribute.noAlloc=false - if this attribute should not be allocated
attributeManager.add({ positions: {size: 2, update: calculatePositions} colors: { size: 4, type: GL.UNSIGNED_BYTE, update: calculateColors } });
Sets log functions to help trace or time attribute updates. Default logging uses the luma.gl logger.
Note that the app may not be in control of when update is called, so hooks are provided for update start and end.
- opts (Object) - named parameters
- opts.onLog (Function) - callback, called to print
- opts.onUpdateStart= (Function) - callback, called before update() starts
- opts.onUpdateEnd= (Function) - callback, called after update() ends
While most apps rely on their layers to automatically generate appropriate WebGL buffers from their props, it is possible for applications to take control of buffer generation and supply the buffers as properties.
While this allows for ultimate performance and control of updates, as well as potential sharing of buffers between layers, the application will need to generate attributes in exactly the format that the layer shaders expect, creating a strong coupling between the application and the layer.
Note: The application can provide some buffers and let others be managed
by the layer. As an example management of the instancePickingColors buffer is
normally left to the layer.
deck.gl layers use WebGL to draw to the screen, and WebGL draw calls need to be provided with on a carefully prepared description of the geometry to be rendered.
In WebGL, such geometry descriptions are called
vertex attributes and are specified using JavaScript typed arrays
(e.g. Float32Array or Uint8Array).
During rendering, these vertex attributes will become available layer's vertex shader executing on the GPU.
Part of designing a new WebGL layer is creating an elegant mapping from a set of data properties in JavaScript to a set of WebGL vertex attributes, and then implement the code that generates the typed arrays that represent the geometry so that GPU calls can be performed efficiently later when drawing the layer.
For performance reasons, each deck.gl layer aspires to use as few
GPU draw calls as possible to draw all the graphics representing
the visualization of all the objects in its data property. In fact,
the majority of deck.gl layers issue only a single GPU draw call.
Since most layers need to render thousands or even millions of similar objects, think circles in the ScatterplotLayer, hexagons in the HexagonLayer etc), how is this achieved?
The key is to use "instanced attributes". While some vertex attributes will still describe the geometry of each object (or instance), some vertex attributes will describe what is different between each object or instance. These attributes are called instanced attributes.
Note: While "instanced rendering" is technically an "extension" to WebGL,
(meaning that it is not guaranteed to be present in all browsers),
today the feature is supported by wide range of systems.
WebGL Stats for statistics on how big a percentage
of systems support various WebGL features, particularly the
ANGLE_instanced_arrays extension.
While you can certainly start consulting detailed WebGL/OpenGL resources to learn more about vertex attributes, be aware that many available resources can get quite technical, involving more concepts than you may need at this point.
If you are new to these concepts, we have found that a great way to learn more is simply to copy an existing deck.gl layer and start extending/modifying its functionality.