-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Implement Expression.serialize() (issue #10174) #11156
Conversation
: Expression(type_) | ||
, value(value_) | ||
, serialized(std::move(serialized_)) | ||
{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd say let's not add this extra bit of storage / complexity for now, and just serialized folded literals as-is.
@@ -35,6 +35,23 @@ ParseResult Assertion::parse(const Convertible& value, ParsingContext& ctx) { | |||
return ParseResult(std::make_unique<Assertion>(it->second, std::move(parsed))); | |||
} | |||
|
|||
const char* Assertion::getOperator() const { | |||
// TODO: Should we strip assertions entirely from serialization? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should leave them -- as of now, we don't track whether an assertion was automatically inferred or actually written as part of the expression. If the latter, it could easily affect the expression's meaning.
} else { | ||
assert(false); | ||
return ""; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can just be:
return type::toString(getType());
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not with the char *
interface because there wouldn't be anyone to own the string. But yeah maybe getOperator
should just return a std::string
...
} else { | ||
assert(false); | ||
return ""; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return getType().match(
[](const type::Number&) { return "to-number"; },
[](const type::Color&) { return "to-color"; },
[](const auto&) { assert(false); return ""; });
|
||
EvaluationResult apply(const EvaluationContext& evaluationParameters, const Args& args) const { | ||
return applyImpl(evaluationParameters, args, std::index_sequence_for<Params...>{}); | ||
} | ||
|
||
std::unique_ptr<Expression> makeExpression(const std::string& name, | ||
std::unique_ptr<Expression> makeExpression(const std::string& name_, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can remove the name
parameter from makeExpression
now that Signature
s know their own names.
serialized.emplace_back(std::vector<mbgl::Value>{{ cubicBezierTag, p1.first, p1.second, p2.first, p2.second }}); | ||
} else { | ||
assert(false); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
interpolator.match(
[&](const ExponentialInterpolator& exponential) {
serialized.emplace_back(std::vector<mbgl::Value>{{ exponentialTag, exponential.base }});
},
[&](const CubicBezierInterpolator& cubicBezier) {
// ...
});
if (interpolator.is<ExponentialInterpolator>()) { | ||
const ExponentialInterpolator& exponential = interpolator.get<ExponentialInterpolator>(); | ||
// TODO: Could do a separate serialization for "linear", but probably no need? | ||
serialized.emplace_back(std::vector<mbgl::Value>{{ exponentialTag, exponential.base }}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any downside to just saying std::vector<mbgl::Value>{{ "exponential", exponential.base }}
? The string's gotta get copied into the vector anyway, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The strings are really just there to help the std::vector<mbgl::Value>
constructor deduce the type -- a plain char* is ambiguous for it.
// static const std::string literalTag = "literal"; | ||
// result.emplace_back(literalTag); | ||
// result.emplace_back(serialized ? *serialized : *fromExpressionValue<mbgl::Value>(value)); | ||
// return result; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
^ yeah, this is what we need for type::Array
and type::Object
literals.
src/mbgl/style/expression/match.cpp
Outdated
}; | ||
serialized.emplace_back(otherwise->serialize()); | ||
return serialized; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that this could make serialize some expressions much more verbosely:
[
"match",
[1, 2, 3, 4, 5], some_complex_expression,
"otherwise"
]
would become:
[
"match",
1, some_complex_expression,
2, some_complex_expression,
3, some_complex_expression,
4, some_complex_expression,
5, some_complex_expression,
"otherwise"
]
That said, I'm not sure whether it's really worth it to produce the more compact form.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I bucketed this under "OK as long as it's semantically equivalent". I'm inclined to leave as is in the name of simplicity, unless you have a hunch that we may actually see a fair number of expressions that blow up in size with the flattened format...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unless you have a hunch that we may actually see a fair number of expressions that blow up in size with the flattened format
I wasn't thinking we needed to worry about this, but on second thought, I think we actually probably should consolidate 😬
a5ffb38
to
a3cc7b4
Compare
I've hacked There's one test failure to deal with right now:
I could just make the test harness ignore types, but not sure if we want to treat a top-level type as being more meaningful. |
@ChrisLoer yeah, we can probably just ignore the type for the roundtripped expression. (We could use checkSubtype to check that its type satisfies the expected type without necessarily being identical to it... but that's probably unnecessary) I still think we should add a |
Nothing jumps out at me from the benchmark results: Benchmark results on
Benchmark results on
|
Even bootstrapping the expected serialization exposed what I think is a legitimate problem (surprisingly not related to the "weirdjsonissue" in the test):
Our serialization of |
Possibly I did some looking for cases where the order might matter and didn't find any, but I came across |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ChrisLoer this is looking great to me overall, but I'm having doubts about the match
collapsing issue...
mbgl::Value ArrayAssertion::serialize() const { | ||
std::vector<mbgl::Value> serialized; | ||
serialized.emplace_back(getOperator()); | ||
getType().match( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this case, since we know it's a type::Array
, you can do const auto& array = getType().get<type::Array>()
instead of going through match
@@ -176,14 +175,14 @@ struct Signature<Lambda, std::enable_if_t<std::is_class<Lambda>::value>> | |||
using Definition = CompoundExpressionRegistry::Definition; | |||
|
|||
template <typename Fn> | |||
static std::unique_ptr<detail::SignatureBase> makeSignature(Fn evaluateFunction) { | |||
return std::make_unique<detail::Signature<Fn>>(evaluateFunction); | |||
static std::unique_ptr<detail::SignatureBase> makeSignature(Fn evaluateFunction, const std::string& name) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we use a non-reference std::string
and std::move
here (and similarly in the Signature
constructors above)?
}, | ||
[&](const auto&) { | ||
assert(false); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This noop branch shouldn't be necessary, because the specific cases above cover every member type of the variant.
src/mbgl/style/expression/match.cpp
Outdated
}; | ||
serialized.emplace_back(otherwise->serialize()); | ||
return serialized; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unless you have a hunch that we may actually see a fair number of expressions that blow up in size with the flattened format
I wasn't thinking we needed to worry about this, but on second thought, I think we actually probably should consolidate 😬
OK, I implemented label coalescing for shared output expressions in |
Sorry, I mentioned this transformation in #10726 but should also mentioned it in #10714 and #10944. I view |
I think I'd have to talk to @anandthakker to get a brain dump on what "expression transformations" might mean -- definitely interested in taking that on after landing this PR. My first reaction would be to try to do something built on top of the serialization, as you suggested in #10944 (comment). But I think maybe I need to understand more use cases -- right now all I have in mind is "find and replace string values within the expression". |
Issue #10714 - Each expression stores its operator as a string, and default serialization is [operator, serialize(child1), ...] - Custom implementations of `serialize` for Expression types that don't follow the pattern - expression::Value -> mbgl::Value converter - node_expression bindings to expose `serialize`
- Round-tripping expressions through serialization and checking that outputs don't change - Checking expression serialization against expected value from fixture
fab05eb
to
13102dd
Compare
Fix to issue #10714.
Signature
objectserialize
for Expression types that don't follow the patternserialize
TODO:
test.json
and wire up the harness to check the serializations.serialize
implementations? Figure out what to do with unusedgetOperator
? Do a pass to minimize unnecessary copy construction.../cc @anandthakker @1ec5