-
Notifications
You must be signed in to change notification settings - Fork 2
Description
On files expensive to parse (large or badly optimized file type), even getting a small chunks can include a fixed cost.
Even reasonable fetch times (300ms) can lead to the DataGrid flashing when crossing t a new chunk.
On solution is prefetching the next chunks (both horizontal and vertical) before the moment it is actually needed.
In doing so, we can also prefetch a larger chunk to reduce the fixed costs of getting a chunk and storing it (reading file, metadata, schema is processed/stored every time in client...). With a prefetchFactor, getting a multiple of the chunk size make it easy to keep the current indexing: the chunk can be stored multiple times in the map, and only the in-chunk indexing need to be accounted for knowing the start row/col of the chunk.