-
Notifications
You must be signed in to change notification settings - Fork 1.3k
[core] Convert "legacy" filters directly into expressions #11610
Conversation
0b2cce0
to
4678133
Compare
4842bbd
to
ecf41db
Compare
621a177
to
daaefc1
Compare
mbgl::Value expressionValue = filter.expression.get()->serialize(); | ||
return gson::JsonElement::New(env, expressionValue); | ||
} else { | ||
return null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question for reviewers: Are we setting ourselves up for future NPEs by allowing filter.expression
to be null
? Is there a better solution?
platform/glfw/glfw_view.cpp
Outdated
args.push_back(std::make_unique<mbgl::style::expression::Literal>(mbgl::style::expression::Value(std::string("extrude")))); | ||
args.push_back(std::make_unique<mbgl::style::expression::Literal>(mbgl::style::expression::Value(std::string("true")))); | ||
mbgl::style::expression::ParsingContext parsingContext; | ||
extrusionLayer->setFilter(mbgl::style::Filter { std::move(*mbgl::style::expression::createCompoundExpression("filter-==", std::move(args), parsingContext)) }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question for future reviewer: should we create a more ergonomic syntax for instantiating expressions? Ideally something close to:
createExpression("filter-==", "extrude", "true")
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I definitely agree that creating filters (and, more generally, expressions) is very cumbersome in mbgl core. I'm just not sure how important it is to streamline this internal API, since its primary consumer is really ParsingContext
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran into this while working on #11247 -- I wanted to completely remove the *Function
constructors that take a Stops
argument. But I gave up because it was sooo cumbersome to convert everything that used them to expressions. That included tests and a few other things that I can't recall at the moment.
So I think it would be worthwhile to provide a more convenient API at least for the subset of expression operators needed by legacy filters and tests. It will probably look similar to the convenience API that Android has.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Specifically, this constructor. There’s a similar thing on iOS/macOS with +[NSExpression expressionWithMGLJSONObject:]
and/or the MGL_FUNCTION()
format string syntax.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added some convenience overloads to createCompoundExpression
and added createLiteral
function. These reduces the boilerplate significantly:
ParsingContext ctx;
extrusionLayer->setFilter(Filter(createCompoundExpression("filter-==", createLiteral("extrude"), createLiteral("true"), ctx)));
2c8d523
to
d588405
Compare
ab8cd73
to
4a4b44f
Compare
@anandthakker This ready for an initial review 😄 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good on the iOS/macOS side.
} | ||
} | ||
} | ||
|
||
#define MGLAssertEqualFilters(actual, expected, ...) \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This macro is unused.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
♻️
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lucaswoj still in progress, but wanted to post these before I close my browser
include/mbgl/style/filter.hpp
Outdated
class Filter : public FilterBase { | ||
public: | ||
using FilterBase::FilterBase; | ||
Filter():expression({}) {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of using nullptr
to model the absence of a filter, should we have expression
be optional<std::shared_ptr<Expression>>
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: surround the :
with spaces (here and just below)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there an advantage to using optional
instead of a nullptr
? For better or for worse, unique_ptr
has "optional" functionality built in and every callsite ought to handle it gracefully.
src/mbgl/style/conversion/filter.cpp
Outdated
|
||
namespace mbgl { | ||
namespace style { | ||
namespace conversion { | ||
|
||
using GeometryValue = mapbox::geometry::value; | ||
static bool isExpression(const Convertible& filter); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this static
necessary?
src/mbgl/style/conversion/filter.cpp
Outdated
|
||
namespace mbgl { | ||
namespace style { | ||
namespace conversion { | ||
|
||
using GeometryValue = mapbox::geometry::value; | ||
static bool isExpression(const Convertible& filter); | ||
std::unique_ptr<expression::Expression> convertLegacyFilter(const Convertible& values, Error& error); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: unnecessary indentation
src/mbgl/style/conversion/filter.cpp
Outdated
error = { parsingContext.getCombinedErrors() }; | ||
return {}; | ||
} | ||
expression = std::move(*parseResult); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this just be return { Filter { std::move(*parseResult) } };
, given the if(!parseResult)
check above?
src/mbgl/style/conversion/filter.cpp
Outdated
|
||
std::unique_ptr<expression::Expression> convertLiteral(const Convertible& convertible) { | ||
optional<mapbox::geometry::value> value = toValue(convertible); | ||
expression::Value expressionValue = value ? expression::toExpressionValue(*value) : expression::Null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like this will effectively coerce array or object values to expression::Null
, because toValue
is supposed to return "none" if the value is not a boolean, number, or string.
I think this means that instead of getting a validation error for ["==", "key", {"x": 1}]
, we'd get the equivalent of ["==", "key", null]
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah. Just seeing the Literal::parse
method now. That looks like the right one to use.
src/mbgl/style/conversion/filter.cpp
Outdated
*op == "none" ? createExpression("!", createExpression("any", convertLegacyFilterArray(values, error, 1), error), error) : | ||
*op == "in" ? convertLegacyInFilter(values, error) : | ||
*op == "!in" ? createExpression("!", convertLegacyInFilter(values, error), error) : | ||
*op == "has" ? convertLegacyHasFilter(*toString(arrayMember(values, 1)), error) : |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if there's an invalid filter of the form ["has", 1]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Then everything will 💥 ! Will fix shortly.
🤷♂️ I can see it both ways. IMO, the benefit of consistently modeling maybe-an-expression with |
src/mbgl/style/conversion/filter.cpp
Outdated
return { IdentifierFilterType {} }; | ||
std::unique_ptr<expression::Expression> convertLiteral(const Convertible& convertible, Error& error) { | ||
expression::ParsingContext parsingContext; | ||
expression::ParseResult parseResult = expression::Literal::parse(convertible, parsingContext); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this still does less type validation than was previously happening. For instance, Literal::parse()
will successfully parse any string, number, boolean, or null
value -- but previously, a legacy filter like ["==", "$type", null]
would have caused the error, "value for $type filter must be Point, LineString, or Polygon"; and ones like ["==", "property_key", null]
would have caused "filter expression value must be a boolean, number, or string".
define("filter-id->=", [](const EvaluationContext& params, double lhs) -> Result<bool> { | ||
auto rhs = featureIdAsDouble(params); | ||
return rhs ? rhs >= lhs : false; | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We also need string overloads of these comparison operators
d6dddde
to
d94cec4
Compare
ASSERT_TRUE(filter("[\"<=\", \"two\", \"2\"]", {{"two", std::string("2")}})); | ||
ASSERT_FALSE(filter("[\"<\", \"two\", \"1\"]", {{"two", std::string("2")}})); | ||
ASSERT_FALSE(filter("[\"==\", \"two\", 4]", {{"two", std::string("2")}})); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GL JS has tests for the expression version of any
and all
(to make sure that we're correctly parsing them as expressions as opposed to legacy filters): https://github.com/mapbox/mapbox-gl-js/blob/95bb4c0bcbf2da406da28795baf6a325c189cc04/test/unit/style-spec/feature_filter.test.js#L27-L37 -- would it make sense to add those?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed at: 01de0a9#diff-4574a618412773ee8823c6637c5221a2
@@ -141,6 +141,18 @@ ParseResult createCompoundExpression(const CompoundExpressionRegistry::Definitio | |||
ParseResult createCompoundExpression(const std::string& name, | |||
std::vector<std::unique_ptr<Expression>> args, | |||
ParsingContext& ctx); | |||
|
|||
ParseResult createCompoundExpression(const std::string& name, ParsingContext& ctx); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we stick a quick comment here indicating that these overloads are here to avoid inconvenience of having to create the args
vector before calling?
return { IdentifierFilterType {} }; | ||
std::unique_ptr<Expression> convertLiteral(const Convertible& convertible, Error& error) { | ||
ParsingContext parsingContext; | ||
ParseResult parseResult = Literal::parse(convertible, parsingContext); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we've still got the issue described in #11610 (comment) : I think right now, this would allow ["==", "$type", null]
or ["==", "property_key", null]
, whereas previously they would have caused errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking into the fix for $type
, but the null
property value case seems to be supported in gl-js: https://github.com/mapbox/mapbox-gl-js/blob/95bb4c0bcbf2da406da28795baf6a325c189cc04/test/unit/style-spec/feature_filter.test.js#L83-L84
Expression syntax also allows Null
as a comparable type :
mapbox-gl-native/src/mbgl/style/expression/equals.cpp
Lines 44 to 49 in 542713f
static bool isComparableType(const type::Type& type) { | |
return type == type::String || | |
type == type::Number || | |
type == type::Boolean || | |
type == type::Null; | |
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added some test to verify this issue, and there is a change in the error returned, but there is still protection from invalid filter structure. Even when the Literal
is parsed, the filter expression will not find a matching definition in the expression registry and complain about incorrect type or number of arguments.
["==", "$type"] //"Expected 1 arguments, but found 0 instead."
["==", "$type", null] //"[1]: Expected string but found null instead."
["==", "$type, "Point", 1] //"Expected 1 arguments, but found 2 instead."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ahhh, gotcha. I mistakenly thought that the filter-...
expressions were accepting a Value
argument, but yeah, as long as we're accepting/rejecting the same filters, I don't think it matters if the specific error message changed. 👍
define("!", [](bool e) -> Result<bool> { return !e; }); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: revert these whitespace changes -- I think we normally use Xcode's default, which to include indentation for blank lines.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As far as I know, the preference in mbgl is still to avoid introducing indented blank lines, while the preference in platform/{darwin,ios,macos}/ and Objective-C code elsewhere is to follow the Xcode default of indenting blank lines. It isn’t really worth fighting over invisible characters, but that goes both ways: not really worth stripping them out, and not really worth putting them back in. 🤷♂️
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It isn’t really worth fighting over invisible characters, but that goes both ways: not really worth stripping them out, and not really worth putting them back in
Yeah, that's basically my stance, too: the particular preference(s) matter less than avoiding whitespace-only changes (like these ^) in diffs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One nit in the unit test, but otherwise LGTM!
test/style/filter.test.cpp
Outdated
ASSERT_TRUE(filter("[\"any\", true]")); | ||
ASSERT_TRUE(filter("[\"any\",true, false]")); | ||
ASSERT_TRUE(filter("[\"any\", true, true]")); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think these will still end up testing the non-expression path -- how about just using the ones from the test immediately above, but replacing "foo"
with ["get", "foo"]
?
2ffbb69
to
364e9e3
Compare
…src/mbgl/style/expression/compound_expression.cpp
Fix android compillation
… need an arg vector
…use expression syntax, and port Fiter.EqualsNull test to match gl-js
364e9e3
to
0359c4c
Compare
Does this need changelog entries? |
Not sure. Is the change transparent to developers using the runtime styling API? |
Ideally. We are replacing all the filter evaluation logic so there’s a small chance that it’ll behave differently in some poorly defined edge cases. And the serialized output may be different.
(Sent from a remote ridge in Kings Canyon National Park)
|
I concur with the man on the ridge (ahoy @lucaswoj!): it should be
transparent (and if it isn’t then it’s a bug).
On Thu, May 10, 2018 at 9:36 PM Lucas Wojciechowski <
notifications@github.com> wrote:
… Ideally. We are replacing all the filter evaluation logic so there’s a
small chance that it’ll behave differently in some poorly defined edge
cases. And the serialized output may be different.
(Sent from a remote ridge in Kings Canyon National Park)
> On May 10, 2018, at 6:04 PM, Minh Nguyễn ***@***.***>
wrote:
>
> Not sure. Is the change transparent to developers using the runtime
styling API?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub <
#11610 (comment)>,
or mute the thread <
https://github.com/notifications/unsubscribe-auth/AARK2q-xdi8HkcI4CrF5WPh7q62XxRYHks5txOOfgaJpZM4TJRiX
>.
>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#11610 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AEvmRxELDOp3ZlyGE9Rt6rpO1UNqksquks5txOsmgaJpZM4TJRiX>
.
|
Follow up to #11429
Launch Checklist
filter-*
expressions from GL JS, adding them to src/mbgl/style/expression/compound_expression.cpp.ExpressionFilter
(using thefilter-*
types as needed for legacy syntax)mbgl::style::FilterEvaluator
, removing all the different*Filter
types and theFilter
variant and renamingExpressionFilter
toFilter
.release-boba