Merge branch 'master' into switch-to-core-application-service

docs/maps/heatmap-layer.asciidoc

@@ -2,15 +2,12 @@
 [[heatmap-layer]]
 == Heat map layer
 
-In the heat map layer, point data is clustered to show locations with higher densities.
+Heat map layers cluster point data to show locations with higher densities.
 
 [role="screenshot"]
 image::maps/images/heatmap_layer.png[]
 
-You can create a heat map layer from the following data source:
-
-*Grid aggregation*:: Geospatial data grouped in grids with metrics for each gridded cell.
-Set *Show as* to *heat map*.
+To add a heat map layer to your map, click *Add layer*, then select the *Heat map* layer.
 The index must contain at least one field mapped as {ref}/geo-point.html[geo_point].
 
 NOTE: Only count, sum, unique count metric aggregations are available with the grid aggregation source and heat map layers.

docs/maps/index.asciidoc

@@ -30,6 +30,7 @@ include::tile-layer.asciidoc[]
 include::vector-layer.asciidoc[]
 include::maps-aggregations.asciidoc[]
 include::search.asciidoc[]
+include::map-settings.asciidoc[]
 include::connect-to-ems.asciidoc[]
 include::geojson-upload.asciidoc[]
 include::indexing-geojson-data-tutorial.asciidoc[]

docs/maps/indexing-geojson-data-tutorial.asciidoc

@@ -87,14 +87,13 @@ hot spots are. An advantage of having indexed
 lightning strikes is that you can perform aggregations on the data.
 
 . Click *Add layer*.
-. From the list of layer types, click *Grid aggregation*.
+. From the list of layer types, click *Heat map*.
 +
 Because you indexed `lightning_detected.geojson` using the index name and
 pattern `lightning_detected`, that data is available as a {ref}/geo-point.html[geo_point]
 aggregation.
 
 . Select `lightning_detected`.
-. Click *Show as* and select `heat map`.
 . Click *Add layer* to add the heat map layer
 "Lightning intensity".
 +

docs/maps/map-settings.asciidoc

@@ -0,0 +1,39 @@
+[role="xpack"]
+[[maps-settings]]
+== Map settings
+
+Elastic Maps offers settings that let you configure how a map is displayed.
+To access these settings, click *Map settings* in the application toolbar.
+
+[float]
+[[maps-settings-navigation]]
+=== Navigation
+
+*Zoom range*::
+Constrain the map to the defined zoom range.
+
+*Initial map location*::
+Configure the initial map center and zoom.
+* *Map location at save*: Use the map center and zoom from the map position at the time of the latest save.
+* *Fixed location*: Lock the map center and zoom to fixed values.
+* *Browser location*: Set the initial map center to the browser location.
+
+[float]
+[[maps-settings-spatial-filters]]
+=== Spatial filters
+
+Use spatial filter settings to configure how <<maps-spatial-filters, spatial filters>> are displayed.
+
+image::maps/images/spatial_filters.png[]
+
+*Show spatial filters on map*::
+Clear the checkbox so <<maps-spatial-filters, spatial filters>> do not appear on the map.
+
+*Opacity*::
+Set the opacity of spatial filters.
+
+*Fill color*::
+Set the fill color of spatial filters.
+
+*Border color*::
+Set the border color of spatial filters.

docs/maps/maps-aggregations.asciidoc

@@ -37,7 +37,7 @@ image::maps/images/grid_to_docs.gif[]
 [[maps-grid-aggregation]]
 === Grid aggregation
 
-The *Grid aggregation* source uses {ref}/search-aggregations-bucket-geotilegrid-aggregation.html[GeoTile grid aggregation] to group your documents into grids. You can calculate metrics for each gridded cell.
+*Grid aggregation* layers use {ref}/search-aggregations-bucket-geotilegrid-aggregation.html[GeoTile grid aggregation] to group your documents into grids. You can calculate metrics for each gridded cell.
 
 Symbolize grid aggregation metrics as:
 
@@ -48,13 +48,13 @@ The cluster location is the weighted centroid for all geo-points in the gridded
 
 *Heat map*:: Creates a <<heatmap-layer, heat map layer>> that clusters the weighted centroids for each gridded cell.
 
-To enable grid aggregation:
+To enable a grid aggregation layer:
 
-. Click *Add layer*, then select the *Grid aggregation* source.
+. Click *Add layer*, then select the *Clusters and grids* or *Heat map* layer.
 
 To enable a blended layer that dynamically shows clusters or documents:
 
-. Click *Add layer*, then select the *Documents* source.
+. Click *Add layer*, then select the *Documents* layer.
 . Configure *Index pattern* and the *Geospatial field*. To enable clustering, the *Geospatial field* must be set to a field mapped as {ref}/geo-point.html[geo_point].
 . In *Scaling*, select *Show clusters when results exceed 10000*.
 
@@ -69,7 +69,7 @@ then accumulates the most relevant documents based on sort order for each entry
 
 To enable top hits:
 
-. Click *Add layer* button and select *Documents* source.
+. Click *Add layer*, then select the *Documents* layer.
 . Configure *Index pattern* and *Geospatial field*.
 . In *Scaling*, select *Show top hits per entity*.
 . Set *Entity* to the field that identifies entities in your documents.
@@ -99,7 +99,7 @@ image::maps/images/point_to_point.png[]
 
 Use term joins to augment vector features with properties for <<maps-vector-style-data-driven, data driven styling>> and richer tooltip content.
 
-Term joins are available for <<vector-layer, vector layers>> with the following sources:
+Term joins are available for the following <<vector-layer, vector layers>>:
 
 * Configured GeoJSON
 * Documents

docs/maps/maps-getting-started.asciidoc

@@ -62,10 +62,10 @@ The first layer you'll add is a choropleth layer to shade world countries
 by web log traffic. Darker shades symbolize countries with more web log traffic,
 and lighter shades symbolize countries with less traffic.
 
-==== Add a vector layer from the Elastic Maps Service source
+==== Add a vector layer to display world country boundaries
 
-. In the map legend, click *Add layer*.
-. Click the *EMS Boundaries* data source.
+. Click *Add layer*.
+. Select the *EMS Boundaries* layer.
 . From the *Layer* dropdown menu, select *World Countries*.
 . Click the *Add layer* button.
 . Set *Name* to `Total Requests by Country`.
@@ -112,16 +112,16 @@ To avoid overwhelming the user with too much data at once, you'll add two layers
 
 * The first layer will display individual documents.
 The layer will appear when the user zooms in the map to show smaller regions.
-* The second layer will show aggregated data that represents many documents.
+* The second layer will display aggregated data that represents many documents.
 The layer will appear when the user zooms out the map to show larger amounts of the globe.
 
-==== Add a vector layer from the document source
+==== Add a vector layer to display individual documents
 
 This layer displays web log documents as points.
 The layer is only visible when users zoom in the map past zoom level 9.
 
-. In the map legend, click *Add layer*.
-. Click the *Documents* data source.
+. Click *Add layer*.
+. Click the *Documents* layer.
 . Set *Index pattern* to *kibana_sample_data_logs*.
 . Click the *Add layer* button.
 . Set *Name* to `Actual Requests`.
@@ -137,7 +137,7 @@ Your map now looks like this between zoom levels 9 and 24:
 [role="screenshot"]
 image::maps/images/gs_add_es_document_layer.png[]
 
-==== Add a vector layer from the grid aggregation source
+==== Add a vector layer to display aggregated data
 
 Aggregations group {es} documents into grids. You can calculate metrics
 for each gridded cell.
@@ -154,10 +154,9 @@ image::maps/images/grid_metrics_both.png[]
 
 ===== Add the layer
 
-. In the map legend, click *Add layer*.
-. Click the *Grid aggregation* data source.
+. Click *Add layer*.
+. Click the *Clusters and grids* layer.
 . Set *Index pattern* to *kibana_sample_data_logs*.
-. Set *Show as* to *clusters*.
 . Click the *Add layer* button.
 . Set *Name* to `Total Requests and Bytes`.
 . Set *Visibility* to the range [0, 9].

docs/maps/search.asciidoc

@@ -10,13 +10,13 @@ You can create a layer that requests data from {es} from the following:
 
 * <<vector-layer>> with:
 
-** Documents source
+** Documents
 
-** Grid aggregation source
+** Clusters and grid
 
 ** <<terms-join>>
 
-* <<heatmap-layer>> with Grid aggregation source
+* <<heatmap-layer>>
 
 [role="screenshot"]
 image::maps/images/global_search_bar.png[]

docs/maps/tile-layer.asciidoc

@@ -2,12 +2,12 @@
 [[tile-layer]]
 == Tile layer
 
-The tile layer displays image tiles served from a tile server.
+Tile layers display image tiles served from a tile server.
 
 [role="screenshot"]
 image::maps/images/tile_layer.png[]
 
-You can create a tile layer from the following data sources:
+To add a tile layer to your map, click *Add layer*, then select one of the following layers:
 
 *Configured Tile Map Service*:: Tile map service configured in kibana.yml.
 See map.tilemap.url in <<settings>> for details.

docs/maps/vector-layer.asciidoc

@@ -2,12 +2,15 @@
 [[vector-layer]]
 == Vector layer
 
-The vector layer displays points, lines, and polygons.
+Vector layers display points, lines, and polygons.
 
 [role="screenshot"]
 image::maps/images/vector_layer.png[]
 
-You can create a vector layer from the following sources:
+To add a vector layer to your map, click *Add layer*, then select one of the following layers:
+
+*Clusters and grids*:: Geospatial data grouped in grids with metrics for each gridded cell.
+The index must contain at least one field mapped as {ref}/geo-point.html[geo_point].
 
 *Configured GeoJSON*:: Vector data from hosted GeoJSON configured in kibana.yml.
 See map.regionmap.* in <<settings>> for details.
@@ -18,15 +21,13 @@ The index must contain at least one field mapped as {ref}/geo-point.html[geo_poi
 NOTE: Document results are limited to the `index.max_result_window` index setting, which defaults to 10000.
 Use <<maps-aggregations, aggregations>> to plot large data sets.
 
-*Grid aggregation*:: Geospatial data grouped in grids with metrics for each gridded cell.
-Set *Show as* to *grid rectangles* or *clusters*.
-The index must contain at least one field mapped as {ref}/geo-point.html[geo_point].
-
 *EMS Boundaries*:: Administrative boundaries from https://www.elastic.co/elastic-maps-service[Elastic Maps Service].
 
 *Point to point*:: Aggregated data paths between the source and destination.
 The index must contain at least 2 fields mapped as {ref}/geo-point.html[geo_point], source and destination.
 
+*Upload Geojson*:: Index GeoJSON data in Elasticsearch.
+
 include::vector-style.asciidoc[]
 include::vector-style-properties.asciidoc[]
 include::vector-tooltips.asciidoc[]

docs/maps/vector-style.asciidoc

@@ -86,7 +86,7 @@ Qualitative data driven styling is available for the following styling propertie
 * *Label color*
 * *Label border color*
 
-To ensure symbols are consistent as you pan, zoom, and filter the map, qualitative data driven styling uses a {ref}/search-aggregations-bucket-terms-aggregation.html[terms aggregation]. The term aggregation retrieves the top nine categories for the property. Feature values within the top categories are assigned a unique style. Feature values outside of the top categories are grouped into the *Other* category. A feature is assigned the *Other* category when the property value is undefined.
+To ensure symbols are consistent as you pan, zoom, and filter the map, qualitative data driven styling uses a {ref}/search-aggregations-bucket-terms-aggregation.html[terms aggregation]. The term aggregation retrieves the top categories for the property. Feature values within the top categories are assigned a unique style. Feature values outside of the top categories are grouped into the *Other* category. A feature is assigned the *Other* category when the property value is undefined.
 
 To configure the terms aggregation, click the gear icon image:maps/images/gear_icon.png[]. Clear the *Get categories from indice* checkbox to turn off the terms aggregation request.
 

src/dev/ci_setup/setup_env.sh

@@ -126,6 +126,7 @@ export PATH="$PATH:$yarnGlobalDir"
 # use a proxy to fetch chromedriver/geckodriver asset
 export GECKODRIVER_CDNURL="https://us-central1-elastic-kibana-184716.cloudfunctions.net/kibana-ci-proxy-cache"
 export CHROMEDRIVER_CDNURL="https://us-central1-elastic-kibana-184716.cloudfunctions.net/kibana-ci-proxy-cache"
+export CYPRESS_DOWNLOAD_MIRROR="https://us-central1-elastic-kibana-184716.cloudfunctions.net/kibana-ci-proxy-cache/cypress"
 
 
 export CHECKS_REPORTER_ACTIVE=false

x-pack/plugins/infra/common/http_api/metrics_explorer.ts

@@ -58,6 +58,7 @@ export const metricsExplorerRequestBodyOptionalFieldsRT = rt.partial({
   limit: rt.union([rt.number, rt.null, rt.undefined]),
   filterQuery: rt.union([rt.string, rt.null, rt.undefined]),
   forceInterval: rt.boolean,
+  dropLastBucket: rt.boolean,
 });
 
 export const metricsExplorerRequestBodyRT = rt.intersection([

x-pack/plugins/infra/common/inventory_models/container/index.ts

@@ -26,7 +26,7 @@ export const container: InventoryModel = {
   fields: {
     id: 'container.id',
     name: 'container.name',
-    ip: 'continaer.ip_address',
+    ip: 'container.ip_address',
   },
   metrics,
   requiredMetrics: [

x-pack/plugins/infra/public/alerting/metric_threshold/components/expression_chart.tsx

@@ -35,6 +35,7 @@ import { getChartTheme } from '../../../pages/metrics/metrics_explorer/component
 import { createFormatterForMetric } from '../../../pages/metrics/metrics_explorer/components/helpers/create_formatter_for_metric';
 import { calculateDomain } from '../../../pages/metrics/metrics_explorer/components/helpers/calculate_domain';
 import { useMetricsExplorerChartData } from '../hooks/use_metrics_explorer_chart_data';
+import { getMetricId } from '../../../pages/metrics/metrics_explorer/components/helpers/get_metric_id';
 
 interface Props {
   context: AlertsContextValue<AlertContextMeta>;
@@ -120,7 +121,7 @@ export const ExpressionChart: React.FC<Props> = ({
     rows: firstSeries.rows.map(row => {
       const newRow: MetricsExplorerRow = { ...row };
       thresholds.forEach((thresholdValue, index) => {
-        newRow[`metric_threshold_${index}`] = thresholdValue;
+        newRow[getMetricId(metric, `threshold_${index}`)] = thresholdValue;
       });
       return newRow;
     }),
@@ -140,7 +141,8 @@ export const ExpressionChart: React.FC<Props> = ({
 
   const isAbove = [Comparator.GT, Comparator.GT_OR_EQ].includes(expression.comparator);
   const opacity = 0.3;
-  const timeLabel = TIME_LABELS[expression.timeUnit];
+  const { timeSize, timeUnit } = expression;
+  const timeLabel = TIME_LABELS[timeUnit];
 
   return (
     <>
@@ -255,8 +257,8 @@ export const ExpressionChart: React.FC<Props> = ({
           <EuiText size="xs" color="subdued">
             <FormattedMessage
               id="xpack.infra.metrics.alerts.dataTimeRangeLabelWithGrouping"
-              defaultMessage="Last 20 {timeLabel} of data for {id}"
-              values={{ id: series.id, timeLabel }}
+              defaultMessage="Last {lookback} {timeLabel} of data for {id}"
+              values={{ id: series.id, timeLabel, lookback: timeSize * 20 }}
             />
           </EuiText>
         ) : (

x-pack/plugins/infra/public/alerting/metric_threshold/hooks/use_metrics_explorer_chart_data.ts

@@ -26,6 +26,7 @@ export const useMetricsExplorerChartData = (
     () => ({
       limit: 1,
       forceInterval: true,
+      dropLastBucket: false,
       groupBy,
       filterQuery,
       metrics: [

x-pack/plugins/infra/public/pages/metrics/metrics_explorer/hooks/use_metrics_explorer_data.ts

@@ -60,6 +60,7 @@ export function useMetricsExplorerData(
             method: 'POST',
             body: JSON.stringify({
               forceInterval: options.forceInterval,
+              dropLastBucket: options.dropLastBucket != null ? options.dropLastBucket : true,
               metrics:
                 options.aggregation === 'count'
                   ? [{ aggregation: 'count' }]

x-pack/plugins/infra/public/pages/metrics/metrics_explorer/hooks/use_metrics_explorer_options.ts

@@ -41,6 +41,7 @@ export interface MetricsExplorerOptions {
   filterQuery?: string;
   aggregation: MetricsExplorerAggregation;
   forceInterval?: boolean;
+  dropLastBucket?: boolean;
 }
 
 export interface MetricsExplorerTimeOptions {

x-pack/plugins/infra/server/routes/metrics_explorer/lib/create_metrics_model.ts

@@ -15,12 +15,15 @@ const percentileToVaue = (agg: 'p95' | 'p99') => {
 };
 
 export const createMetricModel = (options: MetricsExplorerRequestBody): TSVBMetricModel => {
+  // if dropLastBucket is set use the value otherwise default to true.
+  const dropLastBucket: boolean = options.dropLastBucket != null ? options.dropLastBucket : true;
   return {
     id: 'custom',
     requires: [],
     index_pattern: options.indexPattern,
     interval: options.timerange.interval,
     time_field: options.timerange.field,
+    drop_last_bucket: dropLastBucket,
     type: 'timeseries',
     // Create one series per metric requested. The series.id will be used to identify the metric
     // when the responses are processed and combined with the grouping request.