Rest test fixes #27354

clintongormley · 2017-11-12T17:44:53Z

These are various fixes I had to make to get Perl running with the rest tests.

Adjust percentile tests to work with Perl number handling
Fix bad YAML in search/110_field_collapsing.yml
Force dummy scripts to be strings, not numbers
Fixed bad YAML comments
Removed unneeded "headers" from bulk/10_basic.yml
Rename remote.info to cluster.remote_info
Rename render_search_template to indices.render_search_template
Rename ingest.processor.grok to ingest.processor_grok

karmi · 2017-11-13T08:22:31Z

Thanks for looking into that, @clintongormley! I've run the Ruby test suite against the patch and I'm getting more failures than on current master :), I'll investigate more in the next couple of days.

karmi · 2017-11-13T08:26:03Z

.../src/main/resources/rest-api-spec/test/search.aggregation/180_percentiles_tdigest_metric.yml

-        "95.0": 143.49999999999997
-        "99.0": 149.5
+
+  # - match:


The fixes made by @fxdgear in #26905, and my subsequent fixes in #27355 worked nicely for me in Ruby. I must admit I like more the original notation. Also, some tests in the proposed notation (aggregations.percentiles_int.values.5\.0) fail for me, not sure how the Ruby runner handles it, must debug...

The test runners are supposed to support dots escaped with \.

The original notation is a non-starter for me, for instance Perl gets back 143.5 and keeps it as that value, rather than converting to 143.49999999999997. But then it sees 1.0 in the response and reads that as 1(then the test fails when it tries to compare it to 1.0).

I can do numeric comparisons when there is a single numeric-like value on the right, but not with an object.

What I could do (keeping the key-to-numeric-value format) is allow for approximate numeric matching (ie within a certain tolerance) so that I can remove the lte/gte

I'm thinking we may just want to remove the values like 143.49999999999997. I didn't think about it at the time when I added the test, but these are likely subject to how each language interprets json floats and internal float vs double, etc.

Would simplify everyone's life if we stuck to nice "discrete" floats (149.5, etc). The point of these REST tests are to just do rough validation of the body, not really the values, so I think it's fine if we omit the problematic percentiles.

Mpdreamz

Other than a small nitpick LGTM

Mpdreamz · 2017-11-13T10:04:13Z

rest-api-spec/src/main/resources/rest-api-spec/api/indices.render_search_template.json

@@ -1,5 +1,5 @@
 {
-  "render_search_template": {
+  "indices.render_search_template": {


But nit picky perhaps but I think this actually makes more sense unnamespaced like other search related API's. Do not feel strongly about this though.

Thanks, you're right - don't know what i was thinking

javanna · 2017-11-13T13:11:07Z

rest-api-spec/src/main/resources/rest-api-spec/test/bulk/10_basic.yml

  - do:
      catch: /Malformed action\/metadata line \[3\], expected FIELD_NAME but found \[END_OBJECT\]/
-      headers:
-        Content-Type: application/json


why did you remove these? I'm afraid they were there for a reason. This may work for clients without the header as they always use json content-type anyway, but providing the header disables the content-type randomization that the java yaml test runner usually does.

OK - I can revert this and add a features skip clause instead. The problem was that it had headers, but hadn't declared the need for the headers feature.

Ah, I see that the headers feature has already been added, but it's impossible to ignore without also adding parsing to skip it... which i shall duly do (and revert this change)

you mean that you would expect two skip clauses here? or the reason also highlighting why the headers skip?

ok sorry we were kind of typing at the same time, ignore my last message ;)

I think I've fixed this in #26896, but I might have forgot to cherry-pick that commit into the 6.0 branch, which I did yesterday.

javanna · 2017-11-13T13:13:43Z

modules/ingest-common/src/test/resources/rest-api-spec/test/ingest/120_grok.yml

 "Test Grok Patterns Retrieval":
  - do:
-      ingest.processor.grok: {}
+      ingest.processor_grok: {}


wondering if you were the first one ever running these tests...

I think I don't follow this change, why the underscore instead of the dot? the API is called ingest.processor.grok in the spec right? This should fail the build I think.

I renamed ingest.processor.grok to ingest.processor_grok in the spec too, so this should work. The reason for the underscore instead of the dot is that an underscore returns a new namespace in the REST clients, but we have only one thing in this namespace, so makes more sense just to have it at the same level as the other ingest methods.

Yes, I see that now. And the name of the api didn't match the name of the file. All good. I am going to open a PR to validate that the name of the api matches the filename, I've seen enough of these errors.

clintongormley · 2017-11-13T14:32:35Z

jenkins test this

This reverts commit ae13733.

…template" This reverts commit 49d578a.

This reverts commit dccea2b.

clintongormley · 2017-11-14T10:12:13Z

thanks @Mpdreamz @javanna @karmi for the reviews. merging

* REST: Rename ingest.processor.grok to ingest.processor_grok * REST: Rename remote.info to cluster.remote_info * REST: Fixed bad YAML comments * REST: Force dummy scripts to be strings, not numbers * REST: Fix bad YAML in search/110_field_collapsing.yml * REST: Adjust percentile tests to work with Perl number handling

* es/master: Logging: Unify log rotation for index/search slow log (#27298) wildcard query on _index (#27334) REST spec: Validate that api name matches file name that contains it (#27366) Revert "Reduce synchronization on field data cache" Create new handlers for every new request in GoogleCloudStorageService (#27339) Rest test fixes (#27354)

* es/6.x: Updated 6.0.0 release notes to include all changes from alpha, beta, RC wildcard query on _index (#27334) REST spec: Validate that api name matches file name that contains it (#27366) Revert "Reduce synchronization on field data cache" Create new handlers for every new request in GoogleCloudStorageService (#27339) Rest test fixes (#27354)

clintongormley added :Core/Infra/REST API REST infrastructure and utilities >bug v6.0.0-alpha1 v6.1.0 v7.0.0 labels Nov 12, 2017

clintongormley requested review from Mpdreamz, honzakral, javanna, karmi and polyfractal November 12, 2017 17:45

karmi reviewed Nov 13, 2017

View reviewed changes

Mpdreamz approved these changes Nov 13, 2017

View reviewed changes

javanna reviewed Nov 13, 2017

View reviewed changes

clintongormley added 14 commits November 13, 2017 17:38

REST: Rename ingest.processor.grok to ingest.processor_grok

3de6c7e

REST: Rename render_search_template to indices.render_search_template

232bf82

REST: Rename remote.info to cluster.remote_info

6c9f83f

REST: Removed unneeded "headers" from bulk/10_basic.yml

2fae729

REST: Fixed bad YAML comments

22721c6

REST: Force dummy scripts to be strings, not numbers

2aa322d

REST: Fix bad YAML in search/110_field_collapsing.yml

9845d68

REST: Adjust percentil tests to work with Perl number handling

d956913

Fixed ingest test to use ingest.processor_grok

fc67d4d

Use approximate numeric matching in Perl to deal with float precision

70fe301

Use indices.render_search_template in mustache tests

e70476e

Revert "Use indices.render_search_template in mustache tests"

853360b

This reverts commit ae13733.

Revert "REST: Rename render_search_template to indices.render_search_…

009a32c

…template" This reverts commit 49d578a.

Revert "REST: Removed unneeded "headers" from bulk/10_basic.yml"

5c0c2a8

This reverts commit dccea2b.

clintongormley force-pushed the rest_spec branch from 8a7664c to 5c0c2a8 Compare November 13, 2017 16:49

Fixed use of remote.info in QA tests to use cluster.remote_info

2ec50a2

clintongormley merged commit c4e8597 into 6.0 Nov 14, 2017

clintongormley added v6.0.0 and removed v6.0.0-alpha1 labels Nov 14, 2017

clintongormley deleted the rest_spec branch November 14, 2017 12:14

lcawl removed the v6.1.0 label Dec 12, 2017

colings86 added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

Rest test fixes #27354

Rest test fixes #27354

Uh oh!

Conversation

clintongormley commented Nov 12, 2017

Uh oh!

karmi commented Nov 13, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Mpdreamz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clintongormley commented Nov 13, 2017

Uh oh!

clintongormley commented Nov 14, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants