add API for mutating a layer's filter

[jfirebaugh]

Requested in #755 (comment).

[robschley]

Here are a couple use-cases that would benefit from dynamic manipulation of layer filters. All of our use-cases involve large amounts of complex geometries, e.g., 10K polygons is our base test set but we expect to operate in the range of 10K to 100K polygons within the map at one time… possibly more depending on how things shake out.

1. Visualizing temporal data — Our data shows the development of certain assets over time and our existing product allow our users to incrementally step through time and see how development unfolds in an area. They do this by manipulating a slider that filters our data to only show data recorded up to the current point. As they drag the slider around, the map will update and show all data recorded up to whatever date the slider corresponds to. All the data is loaded at once so it is really just a matter of hiding and showing the correct data points based on a date property they contain. We have a POC of this using GL JS using GeoJSON and setData() but because of the overhead of parsing the GeoJSON data and communicating with the workers, the interaction is not fluid. I was hoping we’d be able to solve this using layer filters and we can do the initial filtering but there doesn’t seem to be a clean way to change the filter as the user interacts with the slider.
2. Identifying individual objects — With the amount of data that we have in the map at any time, it is hard to identify individual objects (we don’t map points, we map polygons). We would like to be able to highlight or call out individual objects on the map by interacting with a separate data grid control. Presumably, the grid would call some JS to identify the objects within the map via unique IDs and preserve their visibility while hiding the rest of the objects. I could see this being implemented via a filter where we passed in a set of IDs to the “in” filter operator. Again, the amount of data we have makes it difficult to add/remove it quickly. It would be best if we did not have to deal with that overhead continuously but could apply filtering dynamically within the map since the data is already in there.

[robschley]

Another option, since the filtering is turned into functions anyway (in feature-filter code), would it be possible to just pass in a filter JavaScript function? I suppose the same limitation will exist unless there is a mechanism to reevaluate the functions periodically or forcefully. This makes me think of AngularJS's "digest cycle".

[jfirebaugh]

Thanks @robschley, that's very helpful. I've sketched out a basic implementation that follows the same pattern as get/setLayoutProperty. Right now it resets the whole source, so there's definitely room for performance improvements if necessary. We'll also probably want to follow a similar approach for flicker reduction for GeoJSON sources as @mourner is working on in #973.

[jfirebaugh]

To answer your question about JS functions as filters: there are two technical reasons why we don't do this. One is that the filters are evaluated on web workers, and functions can't be transferred from the main thread to a worker. The second is that we need to maintain parity with mapbox-gl-native. We're not going to embed a JS interpreter there, so that rules out supporting JS functions in the style specification itself, and it's not clear that we'd want to commit to supporting functions in whatever the "native" language is for each native SDK.

[robschley]

@jfirebaugh Thanks for this, it's a super simple API. I tried it out this morning and it works but it has the same flicker problem as in #973. I pulled down the latest master and the setData() flicker seems to have been fixed. I noticed that the filter-api branch was created after #973 was fixed. Would rebasing the filter-api branch off master fix this problem or would it require more work? Also, thanks for explaining the rational for having non-JS functions for filtering, that makes sense.

[jfirebaugh]

@robschley Try latest master, it contains both changes.

[jfirebaugh]

Performance should be pretty good after fe10ad4.

[robschley]

Thanks @jfirebaugh. I did see a bit of a performance uptick but I haven't had a chance to really dig into it.

[gmaclennan]

From one I can figure out, filtering is async, since the filtering is happening on the web worker, but I can't see an event I can listen for to know when it is done. Running this only updates the map once or twice:

for (var i=0; i<100; i++) {
  map.setFilter('markers', ['>', 'index', i])
}

[jfirebaugh]

@gmaclennan Can you open a separate ticket for async callbacks, and for the setFilter calls getting dropped if you think that's a bug?