From 735db8199dfb0c0f9520f74da98efccc2010e65c Mon Sep 17 00:00:00 2001 From: Nik Everett Date: Wed, 4 Sep 2019 15:19:20 -0400 Subject: [PATCH] Build: Enable testing without magic comments (backports #46180) (#46325) Previously we only turned on tests if we saw either `// CONSOLE` or `// TEST`. These magic comments are difficult for the docs build to deal with so it has moved away from using them where possible. We should catch up. This adds another trigger to enable testing: marking a snippet with the `console` language. It looks like this: ``` [source,console] ---- GET / ---- ``` This saves a line which is nice, I guess. But it is more important to me that this is consistent with the way the docs build works now. Similarly this enables response testing when you mark a snippet with the language `console-result`. That looks like: ``` [source,console-result] ---- { "result": "0.1" } ---- ``` `// TESTRESPONSE` is still available for situations like `// TEST`: when the response isn't *in* the console-result language (like `_cat`) or when you want to perform substitutions on the generated test. Should unblock #46159. --- .../doc/RestTestsFromSnippetsTask.groovy | 5 +-- docs/README.asciidoc | 35 +++++++++++++------ .../painless-execute-script.asciidoc | 18 ++++------ docs/reference/docs/index_.asciidoc | 3 +- 4 files changed, 34 insertions(+), 27 deletions(-) diff --git a/buildSrc/src/main/groovy/org/elasticsearch/gradle/doc/RestTestsFromSnippetsTask.groovy b/buildSrc/src/main/groovy/org/elasticsearch/gradle/doc/RestTestsFromSnippetsTask.groovy index d89ed12b8167b..54db2cd716289 100644 --- a/buildSrc/src/main/groovy/org/elasticsearch/gradle/doc/RestTestsFromSnippetsTask.groovy +++ b/buildSrc/src/main/groovy/org/elasticsearch/gradle/doc/RestTestsFromSnippetsTask.groovy @@ -202,11 +202,12 @@ public class RestTestsFromSnippetsTask extends SnippetsTask { previousTest = snippet return } - if (snippet.testResponse) { + if (snippet.testResponse || snippet.language == 'console-result') { response(snippet) return } - if (snippet.test || snippet.console) { + if (snippet.test || snippet.console || + snippet.language == 'console') { test(snippet) previousTest = snippet return diff --git a/docs/README.asciidoc b/docs/README.asciidoc index 9cd2c0b4e7bdc..6ec7083fba85d 100644 --- a/docs/README.asciidoc +++ b/docs/README.asciidoc @@ -3,10 +3,14 @@ Elasticsearch documentation build process. See: https://github.com/elastic/docs -Snippets marked with `// CONSOLE` are automatically annotated with "VIEW IN -CONSOLE" and "COPY AS CURL" in the documentation and are automatically tested -by the command `gradle :docs:check`. To test just the docs from a single page, -use e.g. `./gradlew :docs:integTestRunner --tests "*rollover*"`. +Snippets marked with `[source,console]` are automatically annotated with +"VIEW IN CONSOLE" and "COPY AS CURL" in the documentation and are automatically +tested by the command `./gradlew -pdocs check`. To test just the docs from a +single page, use e.g. `./gradlew -ddocs integTestRunner --tests "*rollover*"`. + +NOTE: Previously we use `// CONSOLE` instead of `[source,console]`. This worked +well for a long time so you'll see it all over early branches but we're phasing +it out because it requires some unpleasant hackery on the docs build side. NOTE: If you have an elasticsearch-extra folder alongside your elasticsearch folder, you must temporarily rename it when you are testing 6.3 or later branches. @@ -45,10 +49,21 @@ for its modifiers: header. If the response doesn't include a `Warning` header with the exact text then the test fails. If the response includes `Warning` headers that aren't expected then the test fails. -* `// TESTRESPONSE`: Matches this snippet against the body of the response of - the last test. If the response is JSON then order is ignored. If you add - `// TEST[continued]` to the snippet after `// TESTRESPONSE` it will continue - in the same test, allowing you to interleave requests with responses to check. +* `[source,console-result]`: Matches this snippet against the body of the + response of the last test. If the response is JSON then order is ignored. If + you add `// TEST[continued]` to the snippet after `[source,console-result]` + it will continue in the same test, allowing you to interleave requests with + responses to check. +* `// TESTRESPONSE`: Explicitly marks a snippet as a test response even without + `[source,console-result]`. Similarly to `// TEST` this is mostly used for + its modifiers. + * You can't use `[source,console-result]` immediately after `// TESTSETUP`. + Instead, consider using `// TEST[continued]` or rearrange your snippets. + + NOTE: Previously we only used `// TESTRESPONSE` instead of + `[source,console-result]` so you'll see that a lot in older branches but we + prefer `[source,console-result]` now. + * `// TESTRESPONSE[s/foo/bar/]`: Substitutions. See `// TEST[s/foo/bar]` for how it works. These are much more common than `// TEST[s/foo/bar]` because they are useful for eliding portions of the response that are not pertinent @@ -62,8 +77,6 @@ for its modifiers: "figures out" the path. This is especially useful for making sweeping assertions like "I made up all the numbers in this example, don't compare them" which looks like `// TESTRESPONSE[s/\d+/$body.$_path/]`. - * You can't use `// TESTRESPONSE` immediately after `// TESTSETUP`. Instead, - consider using `// TEST[continued]` or rearrange your snippets. * `// TESTRESPONSE[non_json]`: Add substitutions for testing responses in a format other than JSON. Use this after all other substitutions so it doesn't make other substitutions difficult. @@ -98,7 +111,7 @@ endyaml ``` This allows slightly more expressive testing of the snippets. Since that syntax -is not supported by CONSOLE the usual way to incorporate it is with a +is not supported by `[source,console]` the usual way to incorporate it is with a `// TEST[s//]` marker like this: ``` diff --git a/docs/painless/painless-guide/painless-execute-script.asciidoc b/docs/painless/painless-guide/painless-execute-script.asciidoc index fc5a6bf71d14a..02c15cb75dd1a 100644 --- a/docs/painless/painless-guide/painless-execute-script.asciidoc +++ b/docs/painless/painless-guide/painless-execute-script.asciidoc @@ -30,7 +30,7 @@ If no context is specified then this context is used by default. Request: -[source,js] +[source,console] ---------------------------------------------------------------- POST /_scripts/painless/_execute { @@ -43,17 +43,15 @@ POST /_scripts/painless/_execute } } ---------------------------------------------------------------- -// CONSOLE Response: -[source,js] +[source,console-result] -------------------------------------------------- { "result": "0.1" } -------------------------------------------------- -// TESTRESPONSE ===== Filter context @@ -69,7 +67,7 @@ index:: The name of an index containing a mapping that is compatible with the do *Example* -[source,js] +[source,console] ---------------------------------------------------------------- PUT /my-index { @@ -99,17 +97,15 @@ POST /_scripts/painless/_execute } } ---------------------------------------------------------------- -// CONSOLE Response: -[source,js] +[source,console-result] -------------------------------------------------- { "result": true } -------------------------------------------------- -// TESTRESPONSE ===== Score context @@ -125,7 +121,7 @@ query:: If `_score` is used in the script then a query can specified that will b *Example* -[source,js] +[source,console] ---------------------------------------------------------------- PUT /my-index { @@ -159,14 +155,12 @@ POST /_scripts/painless/_execute } } ---------------------------------------------------------------- -// CONSOLE Response: -[source,js] +[source,console-result] -------------------------------------------------- { "result": 0.8 } -------------------------------------------------- -// TESTRESPONSE diff --git a/docs/reference/docs/index_.asciidoc b/docs/reference/docs/index_.asciidoc index 4aaa22fbe23cd..0ef40ddb8fcc0 100644 --- a/docs/reference/docs/index_.asciidoc +++ b/docs/reference/docs/index_.asciidoc @@ -140,7 +140,7 @@ automatic creation of indices that match specified patterns, or set it to comma-separated list of patterns you want to allow, or prefix each pattern with `+` or `-` to indicate whether it should be allowed or blocked. -[source,js] +[source,console] -------------------------------------------------- PUT _cluster/settings { @@ -163,7 +163,6 @@ PUT _cluster/settings } } -------------------------------------------------- -// CONSOLE <1> Allow auto-creation of indices called `twitter` or `index10`, block the creation of indices that match the pattern `index1*`, and allow creation of