Allow data transitions when working with millions of data points?

[kevin-daniel-hunt]

I have an h3 hexagon layer. This layer consists of many millions of points of data represented each as a hexagon on the map. The initial load is not too bad, and once its loaded its performance is stellar.

I need to transition the data of H3Layer. I have accomplished this previously with two distinct layers, one with the from data and one with the to data and just swapping which is visible. This works well and is nearly instant. The only thing that changes between the data is the elevation of the hex cells (the two datasets share the same geolocations of h3indexes).

However, I would like this to occur with an actual smooth transition, showing the elevations change between the two data sets. This can be accomplished but the only way I can figure out how to do it, is by changing the data array of a single layer, and this introduces a ton of lag as the system has to recompute all of the attributes on the cpu. So there is a two-three second lag before the transition starts. I have offloaded the calculation of the new data onto a worker, and I am even using a binary array to further make the transition as lightweight as possible. Its not possible for me to bother with _dataDiff as almost all of the data is actually changing.

I believe I have exhausted the documented ways to enhance my performance here. Are there any other suggestions for how I might enable these transitions to occur simultaneously without the latency introduced during the data swap? Or is there a way to inform deck gl to transition between two existant layers I am missing?

The last option I have to consider is I could try and chunk the data update, and then feed the chunking update into the _dataDiff property to basically handle the new calculations over time, and just figure out what a good break point might be to unblock the main thread. This feels hacky and anti-pattern to me though. Any help greatly appreciated!

[Pessimistress]

I am not sure why you need to change data. Your layer should look something like this:

function renderLayer(dataVersion: string) {
	return new H3HexagonLayer<string>({
	  data,
	  getHexagon: (hex_id: string) => hex_id,
	  getElevation: (hex_id: string) => getCellElevation(hex_id),
	  updateTrigger: {
	    getElevation: [dataVersion]
	  },
	  transition: {
	    getElevation: 3000
	  }
	});
}

[kevin-daniel-hunt]

Hmm this seems like the best performance so far. I wasn't appropriately setting my update triggers, and this seems the best approach thus far!