Circumventing 6000+ data limit

[PriyaDoIT]

Due to browser constraints, OpenGrid only allows 1000 dots (6000 dots in WG) to appear on the map. This is a paint point for users because only the 1000 most recent points will populate in both the map and grid view. The limitation is only due to the map constraints, but not the grid view-- the grid view, can display more rows of data. However if we display more rows of data and the user then clicks on a row that doesn't have a corresponding point on the map already displayed, then the map would move and the refresh if only the geospatial filter is set on "Map Extent." The downside to this option is that the grid view and the map will not reflect the same data. The team has brainstormed a couple of options:

1. When a user performs an advanced search that will retrieve more than 1000 rows of data, we display a message stating that the data to be retrieved exceed the maximum dots that are allowed to be plotted on the map. User can select from two options-- either display the limited (1000 records) data and view the map OR display all rows of data and go directly the grid view. In the second option, the user would only have the ability to interact with the grid, but not the map.

2. When a user performs and advanced search that will retrieve more than 1000 rows of data, we don't present any options. Instead, we inform the user that the map will only display the 1000 most recent records based on the filters selected, and to view all data go to the grid view. If the user selects a row in the grid view that doesn't have a corresponding dot on the map, the map will move to area where the dot should be, populate the dot, and refresh the map for the new map extent. For this to work the Geospatial filter must selected to Map Extent.

[tomschenkjr]

I'd like to think about this in the context of dots versus heatmap. The limitation of 1,000 6,000 rows is challenging for dots. Right now, heatmaps are only generated based on dots displayed. Heatmaps have the advantage of showing "all the data" without the performance hit.

If we approach it like this, seems like we're limiting ourselves a bit. Perhaps we'd want the heatmap to generate from all available values where the dot map displays from the perspective mentioned above.