Allow custom global properties to be used in expressions

[vincentvanderweele]

Motivation

Sometimes there is some global "map state" that I would like to use in expressions.

For example, the map I'm currently building has multiple floors and the user can select the floor to display. Most features have a showOnFloors array property, which specifies on which floors to show the feature.

Ideally, I would just be able to tell the map what the current floor is and I could use that floor as a variable in all filter expressions.

Design Alternatives

My current approach is to create a layer per floor with a hardcoded filter and manually implement the logic which layer to insert and which to remove based on the current floor. Alternatively, I could dynamically update the filter expression for these layers.

However, I have multiple layers per floor and also some layers that are independent of the current floor,. This makes that this logic is not trivial and I'm essentially duplicating part of the expression functionality.

Design

I would propose adding a global properties object on the map. This object could be accessed by the expression ["global"], similar to how ["properties"] behaves now. Optionally, we could add helpers like ["global-get", name], although I don't think that's really necessary.

There are already global parameters (zoom, most prominently), so I would imagine there is nothing preventing the use of globals in expressions.

The only real additional functionality is that expressions need to be re-evaluated when the global properties are updated. I would tie this to the function call on map (setGlobalProperties, or however it's called), so I would not watch the properties themselves, i.e.:

const globalProperties = { foo: { a: 1, b: 2}, bar: { c: 3 } };

// this triggers re-evaluating the expressions:
map.setGlobalProperties(globalProperties);

// this does not trigger:
globalProperties.foo.b = 4;

It might happen that the global property does not exist. I'm not familiar enough with all workflows users might have but I could imagine a map designer defines the layer with a filter referencing the global property but the map developer forgets to add that data to the map.

Some alternative ways to deal with that:

• just return undefined
• return undefined and allow providing a default value
• raise an error and stop evaluating the expression

My preference would be defensive and go for the default value but I'm not entirely sure if that is in line with the rest of the API.

Mock-Up

As already sketched above:

• an additional method on map: map.setGlobalProperties(object)
• an additional expression name: ["global"]
• optionally, additional helper expressions: ["global-get", name]

Concepts

I'm not entirely sure if "global properties" is the most descriptive name. I'm open to better alternatives. I think, however, that this is a pretty straightforward addition and the API documentation should be enough.

Implementation

I am not really familiar with the mapbox source code but I had a quick look and thought this object could be part of the evaluation context. I'm not sure yet how re-evaluating the expressions on changing data would work.

With some guidance from the maintainers I would be happy to give this a try and see if I can make this work.

[vincentvanderweele]

To elaborate a bit on the motivation:

Many mapbox examples are imperative. For instance, in this example the visibility of a layer is set on button click.

We are using mapbox with React and would like to approach this in a declarative way. Of course, in the end we need to call the mapbox functions but we try to minimize the size of the imperative wrapper around the map object.

We've been following the approach in this mapbox blog post to integrate the map in our React app. However, looking at an example, there are still some issues with this approach. The map wrapper needs to understand quite a lot of the data it is rendering in order to call the right methods. It needs to understand which properties can be changed on which layers:

componentDidUpdate() {
  this.setFill();
}

setFill() {
  const { property, stops } = this.state.active;
  this.map.setPaintProperty('countries', 'fill-color', {
    property,
    stops
  });    
}

Of course it's possible to generalize this but that would require the clients of this component to understand the internals of mapbox rendering.

---

If we would have global map state, we could instead do this:

class Map extends React.Component {
  componentDidUpdate() {
    this.map.setGlobalProperties(this.props.globalProperties);
  }

  componentDidMount() {
    this.map = new mapboxgl.Map({
      container: this.mapContainer,
      style: 'mapbox://styles/mapbox/streets-v9',
      center: [5, 34],
      zoom: 1.5
    });

    this.map.on('load', () => {
      for (let key in this.props.sources) {
        this.map.addSource(key, sources[key]);
      }

      for (let layer of this.props.layers) {
        this.map.addLayer(layer);  
      }
    });
  }

  // ... other methods
}

// Parent component:

const sources = {
  countries: {
    type: 'geojson',
    data,
  },
};

const layers = [{
  id: 'countries',
  type: 'fill',
  source: 'countries',
  paint: {
    'fill-color': [
      'match', ['get', 'filter_prop', ['global']],
      'pop_est',
      ['step', ['get', 'pop_est'],
        0, '#f8d5cc',
        1000000, '#f4bfb6',
        5000000, '#f1a8a5',
        10000000, '#ee8f9a',
        50000000, '#ec739b',
        100000000, '#dd5ca8',
        250000000, '#c44cc0',
        500000000, '#9f43d7',
        1000000000, '#6e40e6',
      ],
      'gdp_md_est',
      ['step', ['get', 'gdp_md_est'],
        0, '#f8d5cc',
        1000, '#f4bfb6',
        5000, '#f1a8a5',
        10000, '#ee8f9a',
        50000, '#ec739b',
        100000, '#dd5ca8',
        250000, '#c44cc0',
        5000000, '#9f43d7',
        10000000, '#6e40e6',
      ],
      '#ffffff',
    ],
  },
}];

const initialState = {
  filter_prop: 'pop_est',
};

class App extends React.Component {
  render() {
    return (
      <div>
        <Map sources={sources} layers={layers} globalProperties={initialState}/>
      </div>
    );
  }
}

The main advantage is that my map component no longer needs to understand anything about the layers. There could be one layer or 1000. The layer definition could be loaded from a backend or defined locally. All the map component needs to do is expose a globalProperties prop to its clients.

Also, the clients don't need to understand anything about mapbox internals. They just need to pass an prop globalProperties with key filter_prop and possible values 'pop_est' or 'gdp_md_est'.

[stevage]

Great write-up. Just copying in my proposed design from my very similar issue:

"line-color": ["case", ["variable", "dark-mode"], "dark-green", "green"]

map.setVariable("dark-mode", true)

I think it would make more sense to have something ilke setVariable() taking the name of the variable to set, rather than setting all globals in one object. Also, more similar to how setPaintProperty and setLayoutProperty work.

Mapbox-GL-JS is pretty amazing in how it makes it possible to have dynamic maps whose properties respond to changes in state (eg, what thing is being focused on, user preferences, other layers present...). But actually making that work can be pretty complex, and basically requires keeping a "shadow style" outside the map (akin to a shadow DOM). This feature, as noted above, would allow keeping more of that stuff in one single place - inside the map.

[vincentvanderweele]

I really like how simple your proposal is @stevage! How hard would you say it is to implement that?

I'm currently more and more running into the limits of my workaround*, so I'd be pretty motivated to make this work :)

---

*I create a layer per combination of property values. Right now I have 4 floors x 4 languages x 2 variations => 32 layers that could easily be replaced by a single layer if I would have map-wide variables.

[stevage]

I have no idea about implementing - I've never even looked at the mapbox-gl-js internals, much less done any coding.

[vincentvanderweele]

I spent about a day trying to get an MVP work but so far without any success.

The state management part is straightforward: I keep a state object in the style object and pass that in the EvaluationParameters object of style.update. I'm still not entirely sure what to do with all the cases where an EvaluationParameters object is created on the fly with what seems to be dummy parameters. Maybe map state should just not be supported in those cases?

I also registered a new composite expression in style-spec/expression/definitions/index.js, using the map state in its Evaluate function. But no matter what I do, I cannot make the expression evaluation mechanism call that function.

I've been trying to debug the expression evaluation in order to identify all the places where evaluation is optimised and will rewrite my expression to a constant. But even when possiblyEvaluate returns an expression rather than a constant, it will just not evaluate.

Could one of the maintainers give me a nudge in the right direction?

[vincentvanderweele]

I've been digging a bit deeper in how expressions are implemented and came up with two potential approaches. One I already discarded but I'll describe it for completeness anyway:

1. discarded approach: take map state into account when evaluating the expression.

• The main problem with this approach is that expressions can be evaluated in really many different places and under very different circumstances. Sometimes on the UI thread, sometimes in a worker, sometimes all global properties are passed, sometimes that isn't necessary. It would require a lot of plumbing to make the map state available in all those places.
• The rules of which property can use which type of expression are complicated already (and seem to mainly come from how the expression evaluation has been implemented). I would not like to mess with that. Also, conceptually I think map state should be available for every property that allows expressions, which seems hard to achieve with this approach.

2. currently under investigation: take map state into account while parsing the expression.

• This comes closer to my envisioned use of map state: the values are almost constants but might change occasionally over time.
• Expression parsing already has the functionality to simplify expressions whose value is known during parsing. If we would optimise map state away completely during parsing, map state would not impact rendering performance in any way.
• The problem to solve is then still how to change map state. My current idea under investigation consists of two parts:
  • We have a helper class (similar to source cache) that keeps track of all properties that use a certain state variable. When parsing an expression, we would keep track of all the variables it references and store them on this class.
  • When a map state variable changes, we simply re-parse all expressions that use that variable. This would be relatively expensive but map state should only change rarely. Without map state one would probably simulate this behavior by updating all the paint and layout properties that should change based on the new state. That would do the exact same: re-parse all the expressions for those properties.

I have been working on a POC of the second approach but haven't had too much time yet. I will make a PR and ask for feedback as soon as something simple works.

[asheemmamoowala]

@vincentvanderweele Seems we missed an opportunity to provide guidance in a previous post. Sorry for the delayed involvement.

take map state into account while parsing the expression....
This comes closer to my envisioned use of map state: the values are almost constants but might change occasionally over time.
[...]This would be relatively expensive but map state should only change rarely.

What is your expectation of the changes to the global variables during a map session?

One alternative approach you could take and be able to use immediately is to create a templated style.json file yourself and pre-process it before applying to the map with map.setStyle(..., diff: true). With this approach you will likely get similar performance with much less work. The main issue with this approach is that you wouldn't be able to upload and serve these style(s) from Mapbox Studio.

There have been internal discussions to tie in this work with #4225, but none of that is on the current roadmap

[vincentvanderweele]

Hi @asheemmamoowala, thanks for the response!

What is your expectation of the changes to the global variables during a map session?

The main use case for us is switching floors, which would happen regularly during a map session but typically only once per x seconds or more.

One alternative approach [...]

That's a pretty nice idea and definitely a lot simpler than what I had in mind 👍

My main concern is that that might conflict with layers we add to the map dynamically in the frontend (similar to the issues described in #4225). I suppose that I would lose those layers if I do something like:

map.setStyle(styleTemplate("<initial state>");
map.addLayer("<dynamic layer>");
map.setStyle(styleTemplate("<updated state>", diff: true);

which would mean I need to somehow add the dynamic layers to the template instead of adding them directly?

---

Another concern is that I am supposed to share our stylesheet with completely independent teams that might/should not be aware of the trickery we apply to the stylesheet, which is why I have been trying to stick to officially supported features only. I could possibly get around that by exposing two endpoints: one for the template and one with defaults applied (which would be a valid stylesheet again). I need to chew on that for a bit to see if that solves our needs.

---

[...] none of that is on the current roadmap

Just as some expectation management for us, is this a reasonable interpretation: even if I would manage to create a working version of my proposal above, the chances that that will make it to an actually released addition to the library in the near future are slim? Would the same changes need to be made to the native library before they could be released as part of Mapbox-js? And what about support for Mapbox Studio?

The answers to those would probably help me focus our efforts to make this work.

[asheemmamoowala]

My main concern is that that might conflict with layers we add to the map dynamically in the frontend

Yes, this will definitely be an issue.

even if I would manage to create a working version of my proposal above, the chances that that will make it to an actually released addition to the library in the near future are slim?

We would consider the changes for inclusion in the library. The best way to progress would be to put up an RFC of how you propose this will be exposed in the API and implemented, so that we can evaluate it before you go too far down the path of concrete implementation.

Would the same changes need to be made to the native library before they could be released as part of Mapbox-js? And what about support for Mapbox Studio?

You do not have to make the changes to the native library, but it may be a while before some one else picks it up. The changes would likely need to be available in the native platform for Studio to consider implementation so that the styles can be used everywhere.

cc @mapbox/studio

[kkaefer]

FWIW, we used to have this in the style spec, but dropped it in v8 to reduce complexity: mapbox/mapbox-gl-style-spec#308 (comment)

[underbluewaters]

This feature would be super helpful to me. I'm interested in providing what I'm calling smart basemaps in our platform where each map would have a handful of toggles that could be used to highlight or exaggerate certain features, show optional layers, or set layer opacity. The functionality and UI would be very similar to Style Components. I was trying to figure out how Studio implements that feature and it appears to use expressions like component-match and component-prop. Maybe the work necessary to implement this feature is already done and it just needs to be formally added to the spec and public client implementation?

[underbluewaters]

Here's an idea for a fun use-case. Say you want to build an inundation map with a slider to adjust sea level. Given a set of polygons with depth properties you could use an expression to show just those <= the slider value. With some metadata on acceptable input to drive the user interface it could be possible to make these sort of interactive maps work across different platforms.

[CptHolzschnauz]

What was the conclusion at the end?
I search a way to use a variable (from a ajax request to the db) to calculate a scale, something like this

'layout': {
                    'icon-image': '{layername}', // json referenz
                    'icon-size':

                        ['get', 'scale'] *valuefromdb,
                    'symbol-avoid-edges': true,
                    'icon-anchor': 'bottom',
                    'icon-allow-overlap': true

                }

Any hints?

[nzifnab]

Any movement on this? I'm finding I need some way to utilize a custom variable from within style expressions as well.

At first I thought maybe the second parameter on the get expression would work: "Retrieves a property value from the current feature's properties, or from another object if a second argument is provided." (emphasis mine), it says the syntax can be:

["get", string, object]: value

Although exactly zero of the examples use this variation of the syntax, so I'm not entirely sure what object is supposed to be.

So I tried sending it ["get", "zipActive", this]; in this context, this is my own class I control and have a variable on it called zipActive - well... I got a stack level too deep error in javascript instead, so clearly it didn't like that (I'm not even sure how I expected that to work, how would it know when my variable updated and to change the styling?)...

Some kind of global variables I can set on the map or layer and access in styling expressions would be very helpful :) My current workaround is to have two layers with the same source, and show one and hide the other when I need that state to change. Another option would be to loop through hundreds of features and set all the states to the same value, but that sounds horrible.

[CptHolzschnauz]

It's been a while, but:

 scale_com_loc = 0.15;
            map.addLayer({
                'id': 'com_layer_loc',
                'type': 'symbol',
                'source': 'commercial_loc',
                'minzoom': 16,
                'layout': {
                    'icon-image': '{layername}', 
                    'icon-size': scale_com_loc,
                    'symbol-avoid-edges': true,
                    'icon-anchor': 'bottom',
                    'icon-allow-overlap': true

                }
            });

Works for me. After that i can change the icon size in run time without reload the map

[zstadler]

@CptHolzschnauz ,
The above code is using a JavaScript variable, external to the style spec.
The request was to have this option within the expressions in the style itself without the need for JavaScript coding or access.

[tristen]

Configurations are now supported in styles. You can read more about it here along with its entry in the spec. For the original showOnFloors request, you can add something like this to the schema:

{
    "schema": {
        "showFloor": {
            "default": "first",
            "values": [ "basement", "first", "second", "third", "fourth" ]
        }
    }
}

And then the layer for a given floor could look like this:

{
  "id": "first-floor",
  "layout": {
     "visibility": [
        "match",
        ["config", "showFloor"],
        "first",
        "visible",
        "none" 
      ]
  } 
}

[3zzy]

@tristen How would I set the schema when using the classic style:

const map = new mapboxgl.Map({
    container: 'map',
    style: 'mapbox://styles/mapbox/streets-v12',
    center: [144.9632, -37.8136], 
    zoom: 11
});

Would really appreciate an example, my requirement is the same as #9709

[5andr0]

@tristen How would I set the schema when using the classic style:

const map = new mapboxgl.Map({
    container: 'map',
    style: 'mapbox://styles/mapbox/streets-v12',
    center: [144.9632, -37.8136], 
    zoom: 11
});

Would really appreciate an example, my requirement is the same as #9709

I was also looking for an example, but couldn't find one.
So I worked it out:

Option 1: Create your own style object with schema and import a mapbox style via the import field

map = new mapboxgl.Map({
    container: 'map',
    center: [144.9632, -37.8136],
    zoom: 11,
    style: {
        "version": 8,
        "imports": [
            {
                "id": "standard",
                "url": "mapbox://styles/mapbox/streets-v12",
            }
        ],
        "schema": {"showOnFloors": {"default": "first"}},
        "sources": {},
        "layers": []
    }
});

Option 2: Fetch the map as json and merge your options schema

fetch(`https://api.mapbox.com/styles/v1/mapbox/streets-v12?access_token=${mapboxgl.accessToken}`)
.then((response) => response.json())
.then((data) => {
    data.schema = { ...data.schema, "showOnFloors": {"default": "first"}};
    map = new mapboxgl.Map({
        container: 'map',
        center: [144.9632, -37.8136],
        zoom: 11,
        style: data
    });
})
.catch((error) => console.error(error));

Write/Read js:

map.setConfigProperty('basemap', 'showOnFloors', 'second');
console.log( map.getConfigProperty('basemap', 'showOnFloors'));

Access it in your expressions via ['config', 'showOnFloors', 'basemap']
The last parameter (scope) is required! The parameters aren't even defined in the docs.
@devs Please update the docs and give an example like this how to add options via schema.

Notes:
style.import.config only sets the already integrated basemap options. At the point of parsing the config field, the custom schema hasn't been processed yet.
Use the default values instead.

[stepankuzmin]

Hey @5andr0, thanks for flagging it. We'll add an API for reading and writing schema in one of the upcoming releases 👍

[emakarov]

hi @tristen
is there a possibility to use configuration parameters in the expression of a single feature expression in one of the layers as well as update the configuration parameter in runtime?
Let say some object may or may not be visible. one way is to generate new source with the attribute per each object (O(N) and another is to change a key in a configuration (O(1))

[tristen]

@emakarov Sure! Pretending the earlier example is a map you created call floors. You could create a style that imports it:

{
  "version": 8,
  "name": "My map",
  "imports": [{
    "id": "floors", // Arbitary name for the import
    "url": "mapbox://styles/my_username/floors",
    "config": {
      "showFloor": "first"
    }
  }]
}

Then, Initialize the style and call something like this during runtime in your code:

style.setConfigProperty('floors', 'showFloor', 'basement');

---

Note Support questions like these are best supported on StackOverflow if you want to continue the conversation there : )

[emakarov]

Thanks @tristen . That's interesting. Need to experiment on this to see how it works :)