Add parsing for InternalScriptedMetric aggregation #24738

cbuescher · 2017-05-17T12:46:52Z

This adds parsing of the scripted_metric aggregation that we need for the high level java rest client.
While the parsing code itself is straight forward, it wasn't completely clear which kind of values we
should support to parse back from xContent. The documentation mentions: primitive types, String, Map, Array (containing elements of only the types listed here) so that's what is parsed and tested now. It might be possible that other kinds of values also currently work when using the transport client (e.g. StreamOutput#writeGenericValue() also supports GeoPoint, so that might work via transport but it gets rendered as a json object). I included one test to cover these cases with the example of GeoPoint as well.

javanna

left a few minors

javanna · 2017-05-17T13:15:27Z

...c/main/java/org/elasticsearch/search/aggregations/metrics/scripted/ParsedScriptedMetric.java

+    @Override
+    public Object aggregation() {
+        if (aggregation.size() != 1) {
+            throw new IllegalStateException("aggregation was not reduced");


is this error up-to-date?

I think so, at least in InternalScriptedMetric. I think in the parsed version we can ommit it.

yea that was my point, it doesn't apply to the parsed version of it.

I left an assert for tests and a comment though, otherwise this might look like a mistake later on.

javanna · 2017-05-17T13:16:19Z

...c/main/java/org/elasticsearch/search/aggregations/metrics/scripted/ParsedScriptedMetric.java

+        if (token == XContentParser.Token.VALUE_NULL) {
+            return null;
+        } else if (token.isValue()) {
+            return XContentParserUtils.parseStoredFieldsValue(parser);


I think that it feels weird to use this method from here, given its name, as it was really meant to parse stored fields values based on assumptions made around the possible types we have there. I wouldn't want changes made there to be reflected here in the future. Shall we copy what it does over here?

Sure, thats what I had before. I agree that a little copy/paste is better than to introduce unneeded dependencies in this case. Glad to change back.

javanna · 2017-05-17T13:21:43Z

...java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetricTests.java


 package org.elasticsearch.search.aggregations.metrics.scripted;

+import com.google.common.base.Supplier;


is this what you meant to use?

No, good catch. Seems to do the same thing as java.util.function.Supplier apparently though.

javanna · 2017-05-17T13:22:06Z

...java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetricTests.java

+            () -> new ArrayList<>() };
+
+
+


can you remove some of these empty lines?

javanna · 2017-05-17T13:22:32Z

...java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetricTests.java

+
+
+    @Override
+    @Before


I don't think you need @before here, the parent method already has it.

javanna · 2017-05-17T13:32:25Z

...java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetricTests.java

+    }
+
+    @SuppressWarnings("unchecked")
+    private static Object randomValue(Supplier<Object>[] valueTypes, int level) {


I wonder if it's worth having this in RandomObjects instead. But given that it is only used here I don't have a strong opinion on that.

it is also very specialized given that before dance that creates the suppliers, maybe wise to leave it here for now.

javanna · 2017-05-17T13:36:28Z

...java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetricTests.java

+        assertValues(aggregation.aggregation(), parsed.aggregation());
+    }
+
+    private void assertValues(Object expected, Object actual) {


can this be static?

javanna · 2017-05-17T13:38:25Z

core/src/main/java/org/elasticsearch/common/xcontent/ObjectParser.java

        OBJECT_ARRAY_OR_STRING(START_OBJECT, START_ARRAY, VALUE_STRING),
-        VALUE(VALUE_BOOLEAN, VALUE_NULL, VALUE_EMBEDDED_OBJECT, VALUE_NUMBER, VALUE_STRING);
+        VALUE(VALUE_BOOLEAN, VALUE_NULL, VALUE_EMBEDDED_OBJECT, VALUE_NUMBER, VALUE_STRING),
+        VALUE_OBJECT_ARRAY(VALUE_BOOLEAN, VALUE_NULL, VALUE_EMBEDDED_OBJECT, VALUE_NUMBER, VALUE_STRING, START_OBJECT, START_ARRAY);


given that we add this to ObjectParser we should add tests for it as part of the object parser tests too. Maybe even commit it upstream from a separate PR?

I don't see why this is necessary, those enums are merely combinations of token types and the parsing test added in this PR would fail if this wasn't working (indeed it did before I added this new constant includinf ARRAY_START and OBJECT_START). We also don't use any of the enums other than STRING and FLOAT in ObjectParserTests currently.
As for adding this as a separate PR upstream, that PR would only add an enum thats not used? I think we can add it together with merging the feature branch.

cbuescher · 2017-05-17T14:38:05Z

@javanna thanks for the review, I pushed a commit that should address most of your comments and left some response for the others.

tlrx

LGTM, I left some comments but feel free to implement them or not.

tlrx · 2017-05-17T14:52:13Z

...java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetricTests.java

+    @SuppressWarnings("unchecked")
+    private static Object randomValue(Supplier<Object>[] valueTypes, int level) {
+        Supplier<Object> sup = valueTypes[level];
+        Object value = sup.get();


You can save a line and go Object value = valueTypes[level].get(); directly

tlrx · 2017-05-17T14:52:24Z

...java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetricTests.java

+        Supplier<Object> sup = valueTypes[level];
+        Object value = sup.get();
+        if (value instanceof Map) {
+            int elements = randomIntBetween(1,5);


space after comma

tlrx · 2017-05-17T14:52:42Z

...java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetricTests.java

+        if (value instanceof Map) {
+            int elements = randomIntBetween(1,5);
+            for (int i = 0; i < elements; i++) {
+                ((Map<String, Object>) value).put(randomAlphaOfLength(5), randomValue(valueTypes, level + 1));


Could you cast once before the for loop?

tlrx · 2017-05-17T14:52:57Z

...java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetricTests.java

+        } else if (value instanceof List) {
+            int elements = randomIntBetween(1,5);
+            for (int i = 0; i < elements; i++) {
+                ((List<Object>) value).add(randomValue(valueTypes, level + 1));


Same here, cast before then use in loop?

tlrx · 2017-05-17T14:57:12Z

...java/org/elasticsearch/search/aggregations/metrics/scripted/InternalScriptedMetricTests.java

+            GeoPoint point = (GeoPoint) expected;
+            Map<String, Object> pointMap = (Map<String, Object>) actual;
+            assertEquals(point.getLat(), pointMap.get("lat"));
+            assertEquals(point.getLon(), pointMap.get("lon"));


So here we have a difference between transport client and our rest client. Would it make sense to add an extra post-parsing treatment for maps that in case of a map of 2 keys lat and lon converts it back to a GeoPoint?

I don't know, this won't be the only case but one of many. I don't think we should be forced to treat each of those as a special case in the rest client, the documentation says we don't support them anyway. Maybe @colings86 has an opinion about this though...

I think we should draw a line here. Users need to understand that we are going through REST and some things have to change as the protocol is different. I am not sure that patching this to make things work like with the transport client will work, there will be edge cases that we don't handle anyways. After all it should be easy for people to migrate, and a few users may be hit by this right?

I agree, really all we can do is provide a best effort for compatibility between the two and there will inevitably be differences. I think this is a case where the extra effort to make the two the same is not worth it

Good for me, I didn't have a strong feeling about it.

javanna

LGTM

Add parsing for InternalScriptedMetric

7bb3a2c

cbuescher added :Java High Level REST Client labels May 17, 2017

cbuescher requested review from javanna and tlrx May 17, 2017 12:46

javanna requested changes May 17, 2017

View reviewed changes

Addressing comments

2dfcf87

tlrx approved these changes May 17, 2017

View reviewed changes

Adress tlrx comments

2f301d4

javanna approved these changes May 17, 2017

View reviewed changes

cbuescher merged commit 9fc9db2 into elastic:feature/client_aggs_parsing May 17, 2017

This was referenced May 17, 2017

Java High Level REST Client plan for first release #23331

Closed

Add aggs parsers for high level REST Client #24824

Merged

javanna pushed a commit to javanna/elasticsearch that referenced this pull request May 23, 2017

Add parsing for InternalScriptedMetric aggregation (elastic#24738)

eb5e83b

javanna mentioned this pull request May 23, 2017

Backport aggs parsers for high level REST Client #24844

Merged

cbuescher deleted the addParsing-internalScriptedMetric branch November 27, 2025 10:28


		package org.elasticsearch.search.aggregations.metrics.scripted;

		import com.google.common.base.Supplier;

Add parsing for InternalScriptedMetric aggregation #24738

Add parsing for InternalScriptedMetric aggregation #24738

Uh oh!

Conversation

cbuescher commented May 17, 2017

Uh oh!

javanna left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

javanna May 17, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cbuescher commented May 17, 2017

Uh oh!

tlrx left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tlrx May 17, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

javanna left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

javanna May 17, 2017 •

edited

Loading

tlrx May 17, 2017 •

edited

Loading