Add support for slicing strings and arrays in expressions

[lbutler]

Motivation

Slicing of both strings and arrays is listed on the expression API master ticket #6484.

I currently have data sets where I need to extract part of a string before the underlying data can be used. A simple example for me would be trying to style an attribute like "100mm". Ideally, I'd like to strip the unit from the string and be left with only the integer. In more complex cases, I also need to convert the integer based on the unit that is found.

As this is a significant limiter for my own use case, if the below is OK, I'd like to try to implement the new expression and create a pull request.

Design Alternatives

At the moment I believe with expressions, there are no options to modify or change strings. Any modifications would need to be done to the base data before it is handed to the expression.

When search/regex is added it could be used as an alternative. However, that feature seems like it may be more complex to add at this time.

Design

Javascript contains multiple methods to extract part of a string or an array; as the master ticket specifically says slice and the methods are the same for strings and arrays I believe this is what should be implemented.

If we were just looking at strings you could go with a more verbose Left, Right & Mid expressions similar to Excel, which may be more approachable for newer programmers. However, you can achieve all the same from the single Slice method and it isn't a large learning curve with plenty of examples online.

Mock-Up

Slice Expression API:

["slice", string | array, beginIndex: number]: string | array,
["slice", string | array, beginIndex: number, endIndex: number]: string | array,

String examples as per MDN .slice() docs

["slice", "The morning is upon us.", 1, 8]  // OUTPUT: he morn
["slice", "The morning is upon us.", 4, -2] // OUTPUT: morning is upon u
["slice", "The morning is upon us.", 12]    // OUTPUT: is upon us.
["slice", "The morning is upon us.", 30]    // OUTPUT: ""

["slice", "The morning is upon us.", -3]     // OUTPUT: 'us.'
["slice", "The morning is upon us.", -3, -1] // OUTPUT: 'us'
["slice", "The morning is upon us.", 0, -1]  // OUTPUT: 'The morning is upon us'

Array examples as per MDN .slice() docs

["slice", ["literal", ['ant', 'bison', 'camel', 'duck', 'elephant']], 2]    // output: ["camel", "duck", "elephant"]
["slice", ["literal", ['ant', 'bison', 'camel', 'duck', 'elephant']], 2, 4] // output: ["camel", "duck"]
["slice", ["literal", ['ant', 'bison', 'camel', 'duck', 'elephant']], 1, 5] // output: ["bison", "camel", "duck", "elephant"]
["slice", ["literal", ['ant', 'bison', 'camel', 'duck', 'elephant']], -2]   // output: ["duck", "elephant"]

Concepts

As slice is in the master ticket, I believe this is what Mapbox wants to go forward with. Though as mentioned in the design section there are multiple methods, some of which are legacy, within Javascript to modify strings and arrays.

Implementation

I assume it would be modelled similar to the existing expression ["length"] at /src/style-spec/expression/definitions/length.js which handles arrays and strings.

I'm not familiar with the other SDKs that implement Mapbox expressions and if JS slice methods are handled in a similar way on those languages.

[arindam1993]

Thanks for the detailed writeup @lbutler !

I think this sounds like a good idea, and a PR is defnitely welcome! This would fit in well with the in expression that was added recently, but it seems like it would be hamstrung without an accompanying index-of expression, since otherwise you'd be restricted to hard coded indices.

Also for your use-case, if you have control over the data, I'd recommend preprocessing it and splitting up the data into it up into multiple properties. Its always better to store measurements in one unit system and convert to the desired one on the client based on user preference.

[lbutler]

Agree that including an index-of expression would make sense. The same expression could be used for both strings and arrays.

I'll start with a PR for slice and then do a separate PR for index-of (unless the consensus is they should both be done together).

Unfortunately for me, I'm working with user-supplied data from water utilities and most mix metric and imperial measurements for pipe diameters and store them in a single property as a string.

Also, my use-case is probably outside how most people would use expressions. I import @mapbox/mapbox-gl-style-spec directly into my app (not map-based) and use expressions as a way to let my users set filters and create custom attributes on their data before it exports to a priority format to create hydraulic models.

[ryanhamley]

Hey @lbutler thanks for this writeup. I just want to echo @arindam1993's thoughts above. We have a whole class of expressions that work on strings and arrays, so I think this is a great addition. index-of also seems extremely helpful, almost a pre-requisite as noted above.

The in expression which was added recently might be a good guide for this work. The PR for that is #8876 You can see below that the in expression directly makes use of the indexOf() method. It's ok to use the slice method internally because GL Native will implement the equivalent functionality with native methods. slice strikes me as the right choice since it works on strings and arrays while leaving the original input unchanged.

mapbox-gl-js/src/style-spec/expression/definitions/in.js

Line 75 in 96c4b66

return haystack.indexOf(needle) >= 0;

Also, my use-case is probably outside how most people would use expressions. I import @mapbox/mapbox-gl-style-spec directly into my app (not map-based) and use expressions as a way to let my users set filters and create custom attributes on their data before it exports to a priority format to create hydraulic models.

You may be interested in following along with #7670 which is about making expressions work outside of rendering

[lbutler]

Thanks @arindam1993 & @ryanhamley! I've pushed a PR for both slice and index-of based on your suggestions.

You may be interested in following along with #7670 which is about making expressions work outside of rendering

I've got it subscripted, will be awesome when it's available! I've been using the the gist that was linked for typescript for the time being.