Merge branch '6.x' into ccr-6.x

* 6.x:
  Watcher: Make settings reloadable (#31746)
  [Rollup] Histo group config should support scaled_floats (#32048)
  lazy snapshot repository initialization (#31606)
  Add secure setting for watcher email password (#31620)
  Watcher: cleanup ensureWatchExists use (#31926)
  Add second level of field collapsing (#31808)
  Added lenient flag for synonym token filter (#31484) (#31970)
  Test: Fix a second case of bad watch creation
  [Rollup] Use composite's missing_bucket (#31402)
  Docs: Restyled cloud link in getting started
  Docs: Change formatting of Cloud options
  Re-instate link in StringFunctionUtils javadocs
  Correct spelling of AnalysisPlugin#requriesAnalysisSettings (#32025)
  Fix problematic chars in javadoc
  [ML] Move open job failure explanation out of root cause (#31925)
  [ML] Switch ML native QA tests to use a 3 node cluster (#32011)

docs/reference/analysis/tokenfilters/synonym-graph-tokenfilter.asciidoc

@@ -50,7 +50,49 @@ PUT /test_index
 The above configures a `search_synonyms` filter, with a path of
 `analysis/synonym.txt` (relative to the `config` location). The
 `search_synonyms` analyzer is then configured with the filter.
-Additional settings are: `expand` (defaults to `true`).
+
+Additional settings are:
+
+* `expand` (defaults to `true`).
+* `lenient` (defaults to `false`). If `true` ignores exceptions while parsing the synonym configuration. It is important
+to note that only those synonym rules which cannot get parsed are ignored. For instance consider the following request:
+
+[source,js]
+--------------------------------------------------
+PUT /test_index
+{
+    "settings": {
+        "index" : {
+            "analysis" : {
+                "analyzer" : {
+                    "synonym" : {
+                        "tokenizer" : "standard",
+                        "filter" : ["my_stop", "synonym_graph"]
+                    }
+                },
+                "filter" : {
+                	"my_stop": {
+                		"type" : "stop",
+                		"stopwords": ["bar"]
+                	},
+                    "synonym_graph" : {
+                        "type" : "synonym_graph",
+                        "lenient": true,
+                        "synonyms" : ["foo, bar => baz"]
+                    }
+                }
+            }
+        }
+    }
+}
+--------------------------------------------------
+// CONSOLE
+With the above request the word `bar` gets skipped but a mapping `foo => baz` is still added. However, if the mapping
+being added was "foo, baz => bar" nothing would get added to the synonym list. This is because the target word for the
+mapping is itself eliminated because it was a stop word. Similarly, if the mapping was "bar, foo, baz" and `expand` was
+set to `false` no mapping would get added as when `expand=false` the target mapping is the first word. However, if
+`expand=true` then the mappings added would be equivalent to `foo, baz => foo, baz` i.e, all mappings other than the
+stop word.
 
 [float]
 ==== `tokenizer` and `ignore_case` are deprecated

docs/reference/analysis/tokenfilters/synonym-tokenfilter.asciidoc

@@ -33,12 +33,55 @@ PUT /test_index
 
 The above configures a `synonym` filter, with a path of
 `analysis/synonym.txt` (relative to the `config` location). The
-`synonym` analyzer is then configured with the filter. Additional
-settings is: `expand` (defaults to `true`).
+`synonym` analyzer is then configured with the filter.
 
 This filter tokenize synonyms with whatever tokenizer and token filters
 appear before it in the chain.
 
+Additional settings are:
+
+* `expand` (defaults to `true`).
+* `lenient` (defaults to `false`). If `true` ignores exceptions while parsing the synonym configuration. It is important
+to note that only those synonym rules which cannot get parsed are ignored. For instance consider the following request:
+
+[source,js]
+--------------------------------------------------
+PUT /test_index
+{
+    "settings": {
+        "index" : {
+            "analysis" : {
+                "analyzer" : {
+                    "synonym" : {
+                        "tokenizer" : "standard",
+                        "filter" : ["my_stop", "synonym"]
+                    }
+                },
+                "filter" : {
+                	"my_stop": {
+                		"type" : "stop",
+                		"stopwords": ["bar"]
+                	},
+                    "synonym" : {
+                        "type" : "synonym",
+                        "lenient": true,
+                        "synonyms" : ["foo, bar => baz"]
+                    }
+                }
+            }
+        }
+    }
+}
+--------------------------------------------------
+// CONSOLE
+With the above request the word `bar` gets skipped but a mapping `foo => baz` is still added. However, if the mapping
+being added was "foo, baz => bar" nothing would get added to the synonym list. This is because the target word for the
+mapping is itself eliminated because it was a stop word. Similarly, if the mapping was "bar, foo, baz" and `expand` was
+set to `false` no mapping would get added as when `expand=false` the target mapping is the first word. However, if
+`expand=true` then the mappings added would be equivalent to `foo, baz => foo, baz` i.e, all mappings other than the
+stop word.
+
+
 [float]
 ==== `tokenizer` and `ignore_case` are deprecated
 

docs/reference/getting-started.asciidoc

@@ -104,10 +104,13 @@ With that out of the way, let's get started with the fun part...
 
 == Installation
 
+[TIP]
+==============
 You can skip installation completely by using our hosted
 Elasticsearch Service on https://www.elastic.co/cloud[Elastic Cloud], which is
 available on AWS and GCP. You can
 https://www.elastic.co/cloud/elasticsearch-service/signup[try out the hosted service] for free.
+==============
 
 Elasticsearch requires at least Java 8. Specifically as of this writing, it is recommended that you use the Oracle JDK version {jdk}. Java installation varies from platform to platform so we won't go into those details here. Oracle's recommended installation documentation can be found on http://docs.oracle.com/javase/8/docs/technotes/guides/install/install_overview.html[Oracle's website]. Suffice to say, before you install Elasticsearch, please check your Java version first by running (and then install/upgrade accordingly if needed):
 

docs/reference/search/request/collapse.asciidoc

@@ -116,3 +116,105 @@ The default is based on the number of data nodes and the default search thread p
 
 WARNING: `collapse` cannot be used in conjunction with <<search-request-scroll, scroll>>,
 <<search-request-rescore, rescore>> or <<search-request-search-after, search after>>.
+
+==== Second level of collapsing
+
+Second level of collapsing is also supported and is applied to `inner_hits`.
+For example, the following request finds the top scored tweets for
+each country, and within each country finds the top scored tweets
+for each user.
+
+[source,js]
+--------------------------------------------------
+GET /twitter/_search
+{
+    "query": {
+        "match": {
+            "message": "elasticsearch"
+        }
+    },
+    "collapse" : {
+        "field" : "country",
+        "inner_hits" : {
+            "name": "by_location",
+            "collapse" : {"field" : "user"},
+            "size": 3
+        }
+    }
+}
+--------------------------------------------------
+// NOTCONSOLE
+
+
+Response:
+[source,js]
+--------------------------------------------------
+{
+    ...
+    "hits": [
+        {
+            "_index": "twitter",
+            "_type": "_doc",
+            "_id": "9",
+            "_score": ...,
+            "_source": {...},
+            "fields": {"country": ["UK"]},
+            "inner_hits":{
+                "by_location": {
+                    "hits": {
+                       ...,
+                       "hits": [
+                          {
+                            ...
+                            "fields": {"user" : ["user124"]}
+                          },
+                          {
+                            ...
+                            "fields": {"user" : ["user589"]}
+                          },
+                          {
+                            ...
+                             "fields": {"user" : ["user001"]}
+                          }
+                       ]
+                    }
+                 }
+            }
+        },
+        {
+            "_index": "twitter",
+            "_type": "_doc",
+            "_id": "1",
+            "_score": ..,
+            "_source": {...},
+            "fields": {"country": ["Canada"]},
+            "inner_hits":{
+                "by_location": {
+                    "hits": {
+                       ...,
+                       "hits": [
+                          {
+                            ...
+                            "fields": {"user" : ["user444"]}
+                          },
+                          {
+                            ...
+                            "fields": {"user" : ["user1111"]}
+                          },
+                          {
+                            ...
+                             "fields": {"user" : ["user999"]}
+                          }
+                       ]
+                    }
+                 }
+            }
+
+        },
+        ....
+    ]
+}
+--------------------------------------------------
+// NOTCONSOLE
+
+NOTE: Second level of of collapsing doesn't allow `inner_hits`.

docs/reference/setup/install.asciidoc

@@ -1,11 +1,16 @@
 [[install-elasticsearch]]
 == Installing Elasticsearch
 
+[float]
+=== Hosted Elasticsearch
 Elasticsearch can be run on your own hardware or using our hosted
 Elasticsearch Service on https://www.elastic.co/cloud[Elastic Cloud], which is
 available on AWS and GCP. You can
 https://www.elastic.co/cloud/elasticsearch-service/signup[try out the hosted service] for free.
 
+[float]
+=== Installing Elasticsearch Yourself
+
 Elasticsearch is provided in the following package formats:
 
 [horizontal]

modules/analysis-common/src/main/java/org/elasticsearch/analysis/common/CommonAnalysisPlugin.java

@@ -135,7 +135,7 @@
 import java.util.Map;
 import java.util.TreeMap;
 
-import static org.elasticsearch.plugins.AnalysisPlugin.requriesAnalysisSettings;
+import static org.elasticsearch.plugins.AnalysisPlugin.requiresAnalysisSettings;
 
 public class CommonAnalysisPlugin extends Plugin implements AnalysisPlugin {
 
@@ -201,11 +201,11 @@ public Map<String, AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
         filters.put("cjk_width", CJKWidthFilterFactory::new);
         filters.put("classic", ClassicFilterFactory::new);
         filters.put("czech_stem", CzechStemTokenFilterFactory::new);
-        filters.put("common_grams", requriesAnalysisSettings(CommonGramsTokenFilterFactory::new));
+        filters.put("common_grams", requiresAnalysisSettings(CommonGramsTokenFilterFactory::new));
         filters.put("decimal_digit", DecimalDigitFilterFactory::new);
         filters.put("delimited_payload_filter", LegacyDelimitedPayloadTokenFilterFactory::new);
         filters.put("delimited_payload", DelimitedPayloadTokenFilterFactory::new);
-        filters.put("dictionary_decompounder", requriesAnalysisSettings(DictionaryCompoundWordTokenFilterFactory::new));
+        filters.put("dictionary_decompounder", requiresAnalysisSettings(DictionaryCompoundWordTokenFilterFactory::new));
         filters.put("dutch_stem", DutchStemTokenFilterFactory::new);
         filters.put("edge_ngram", EdgeNGramTokenFilterFactory::new);
         filters.put("edgeNGram", EdgeNGramTokenFilterFactory::new);
@@ -216,11 +216,11 @@ public Map<String, AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
         filters.put("german_normalization", GermanNormalizationFilterFactory::new);
         filters.put("german_stem", GermanStemTokenFilterFactory::new);
         filters.put("hindi_normalization", HindiNormalizationFilterFactory::new);
-        filters.put("hyphenation_decompounder", requriesAnalysisSettings(HyphenationCompoundWordTokenFilterFactory::new));
+        filters.put("hyphenation_decompounder", requiresAnalysisSettings(HyphenationCompoundWordTokenFilterFactory::new));
         filters.put("indic_normalization", IndicNormalizationFilterFactory::new);
-        filters.put("keep", requriesAnalysisSettings(KeepWordFilterFactory::new));
-        filters.put("keep_types", requriesAnalysisSettings(KeepTypesFilterFactory::new));
-        filters.put("keyword_marker", requriesAnalysisSettings(KeywordMarkerTokenFilterFactory::new));
+        filters.put("keep", requiresAnalysisSettings(KeepWordFilterFactory::new));
+        filters.put("keep_types", requiresAnalysisSettings(KeepTypesFilterFactory::new));
+        filters.put("keyword_marker", requiresAnalysisSettings(KeywordMarkerTokenFilterFactory::new));
         filters.put("kstem", KStemTokenFilterFactory::new);
         filters.put("length", LengthTokenFilterFactory::new);
         filters.put("limit", LimitTokenCountFilterFactory::new);
@@ -229,8 +229,8 @@ public Map<String, AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
         filters.put("multiplexer", MultiplexerTokenFilterFactory::new);
         filters.put("ngram", NGramTokenFilterFactory::new);
         filters.put("nGram", NGramTokenFilterFactory::new);
-        filters.put("pattern_capture", requriesAnalysisSettings(PatternCaptureGroupTokenFilterFactory::new));
-        filters.put("pattern_replace", requriesAnalysisSettings(PatternReplaceTokenFilterFactory::new));
+        filters.put("pattern_capture", requiresAnalysisSettings(PatternCaptureGroupTokenFilterFactory::new));
+        filters.put("pattern_replace", requiresAnalysisSettings(PatternReplaceTokenFilterFactory::new));
         filters.put("persian_normalization", PersianNormalizationFilterFactory::new);
         filters.put("porter_stem", PorterStemTokenFilterFactory::new);
         filters.put("remove_duplicates", RemoveDuplicatesTokenFilterFactory::new);
@@ -241,10 +241,10 @@ public Map<String, AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
         filters.put("serbian_normalization", SerbianNormalizationFilterFactory::new);
         filters.put("snowball", SnowballTokenFilterFactory::new);
         filters.put("sorani_normalization", SoraniNormalizationFilterFactory::new);
-        filters.put("stemmer_override", requriesAnalysisSettings(StemmerOverrideTokenFilterFactory::new));
+        filters.put("stemmer_override", requiresAnalysisSettings(StemmerOverrideTokenFilterFactory::new));
         filters.put("stemmer", StemmerTokenFilterFactory::new);
         filters.put("trim", TrimTokenFilterFactory::new);
-        filters.put("truncate", requriesAnalysisSettings(TruncateTokenFilterFactory::new));
+        filters.put("truncate", requiresAnalysisSettings(TruncateTokenFilterFactory::new));
         filters.put("unique", UniqueTokenFilterFactory::new);
         filters.put("uppercase", UpperCaseTokenFilterFactory::new);
         filters.put("word_delimiter_graph", WordDelimiterGraphTokenFilterFactory::new);
@@ -256,8 +256,8 @@ public Map<String, AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
     public Map<String, AnalysisProvider<CharFilterFactory>> getCharFilters() {
         Map<String, AnalysisProvider<CharFilterFactory>> filters = new TreeMap<>();
         filters.put("html_strip", HtmlStripCharFilterFactory::new);
-        filters.put("pattern_replace", requriesAnalysisSettings(PatternReplaceCharFilterFactory::new));
-        filters.put("mapping", requriesAnalysisSettings(MappingCharFilterFactory::new));
+        filters.put("pattern_replace", requiresAnalysisSettings(PatternReplaceCharFilterFactory::new));
+        filters.put("mapping", requiresAnalysisSettings(MappingCharFilterFactory::new));
         return filters;
     }
 

modules/rank-eval/src/test/java/org/elasticsearch/index/rankeval/ExpectedReciprocalRankTests.java

@@ -69,7 +69,7 @@ public void testProbabilityOfRelevance() {
      * 4    | 1         | 0.03125   | 0.078125 | 0.00244140625 |
      * }</pre>
      *
-     * err => sum of last column
+     * err = sum of last column
      */
     public void testERRAt() {
         List<RatedDocument> rated = new ArrayList<>();
@@ -94,7 +94,7 @@ public void testERRAt() {
      * 4    | 1         | 0.03125   | 0.125    | 0.00390625 |
      * }</pre>
      *
-     * err => sum of last column
+     * err = sum of last column
      */
     public void testERRMissingRatings() {
         List<RatedDocument> rated = new ArrayList<>();