Merge branch 'master' into alerting/consumer-based-rbac

* master: (38 commits)
  Support migrating from reserved feature privileges (elastic#68504)
  add `preference` field to SavedObjectsFindOptions (elastic#68620)
  [ILM] Add "wait for snapshot" policy field to Delete phase (elastic#68505)
  Cleanup old license overwrites (elastic#68744)
  Bump TypeScript to v3.9 (elastic#67666)
  [APM] Service maps - adds new storybook stories to test out various data sets (elastic#68727)
  Fix vega specification parsing (elastic#67963)
  docs: add more api information (elastic#68717)
  [APM] Don't show annotations on charts with no data (elastic#68829)
  [Metrics UI] Fix Inventory View sorting by handling null values (elastic#67889)
  skip flaky suite (elastic#68836)
  [SIEM][Detections Engine] - Fix reference rule url overflow (elastic#68640)
  Index pattern public api => common (elastic#68289)
  [APM] Lazy-load alert triggers (elastic#68806)
  [DOCS] Fix table formatting in ingest manager settings (elastic#68824)
  [Endpoint] Functional Tests cleanup (elastic#68756)
  revert previous commit which was unintentional
  Use Github token instead for project assignments
  [SIEM][Exceptions] - ExceptionsViewer cleanup (elastic#68739)
  move @kbn/storybook to devDeps (elastic#68791)
  ...

.eslintrc.js

@@ -197,9 +197,12 @@ module.exports = {
  errorMessage: `Plugins may only import from src/core/server and src/core/public.`,
  },
  {
- target: ['(src|x-pack)/plugins/*/public/**/*'],
- from: ['(src|x-pack)/plugins/*/server/**/*'],
- errorMessage: `Public code can not import from server, use a common directory.`,
+ target: [
+ '(src|x-pack)/plugins/*/server/**/*',
+ '!x-pack/plugins/apm/**/*', // https://github.com/elastic/kibana/issues/67210
+ ],
+ from: ['(src|x-pack)/plugins/*/public/**/*'],
+ errorMessage: `Server code can not import from public, use a common directory.`,
  },
  {
  target: ['(src|x-pack)/plugins/*/common/**/*'],
@@ -589,8 +592,11 @@ module.exports = {
  * Security Solution overrides
  */
  {
- // front end typescript and javascript files only
- files: ['x-pack/plugins/security_solution/public/**/*.{js,ts,tsx}'],
+ // front end and common typescript and javascript files only
+ files: [
+ 'x-pack/plugins/security_solution/public/**/*.{js,ts,tsx}',
+ 'x-pack/plugins/security_solution/common/**/*.{js,ts,tsx}',
+ ],
  rules: {
  'import/no-nodejs-modules': 'error',
  'no-restricted-imports': [
@@ -766,6 +772,23 @@ module.exports = {
  /**
  * Lists overrides
  */
+ {
+ // front end and common typescript and javascript files only
+ files: [
+ 'x-pack/plugins/lists/public/**/*.{js,ts,tsx}',
+ 'x-pack/plugins/lists/common/**/*.{js,ts,tsx}',
+ ],
+ rules: {
+ 'import/no-nodejs-modules': 'error',
+ 'no-restricted-imports': [
+ 'error',
+ {
+ // prevents UI code from importing server side code and then webpack including it when doing builds
+ patterns: ['**/server/*'],
+ },
+ ],
+ },
+ },
  {
  // typescript and javascript for front and back end
  files: ['x-pack/plugins/lists/**/*.{js,ts,tsx}'],

.github/CODEOWNERS

@@ -222,8 +222,7 @@
 /x-pack/plugins/endpoint/ @elastic/endpoint-app-team @elastic/siem
 /x-pack/test/api_integration/apis/endpoint/ @elastic/endpoint-app-team @elastic/siem
 /x-pack/test/endpoint_api_integration_no_ingest/ @elastic/endpoint-app-team @elastic/siem
-/x-pack/test/functional_endpoint/ @elastic/endpoint-app-team @elastic/siem
-/x-pack/test/functional_endpoint_ingest_failure/ @elastic/endpoint-app-team @elastic/siem
+/x-pack/test/security_solution_endpoint/ @elastic/endpoint-app-team @elastic/siem
 /x-pack/test/functional/es_archives/endpoint/ @elastic/endpoint-app-team @elastic/siem
 /x-pack/test/plugin_functional/plugins/resolver_test/ @elastic/endpoint-app-team @elastic/siem
 /x-pack/test/plugin_functional/test_suites/resolver/ @elastic/endpoint-app-team @elastic/siem

.gitignore

@@ -4,6 +4,7 @@
 /.es
 .DS_Store
 .node_binaries
+.native_modules
 node_modules
 !/src/dev/npm/integration_tests/__fixtures__/fixture1/node_modules
 !/src/dev/notice/__fixtures__/node_modules

docs/apm/api.asciidoc

@@ -15,8 +15,39 @@ Some APM app features are provided via a REST API:
 [[apm-api-example]]
 === Using the APIs
 
-Users interacting with APM APIs must have <<apm-app-api-user,sufficient privileges>>.
-In addition, there are request headers to be aware of, like `kbn-xsrf: true`, and `Content-Type: applicaton/json`.
+// The following content is reused throughout the API docs
+// tag::using-the-APIs[]
+Interact with APM APIs using cURL or another API tool.
+All APM APIs are Kibana APIs, not Elasticsearch APIs;
+because of this, the Kibana dev tools console cannot be used to interact with APM APIs.
+
+For all APM APIs, you must use a request header.
+Supported headers are `Authorization`, `kbn-xsrf`, and `Content-Type`.
+
+`Authorization: ApiKey {credentials}`::
+Kibana supports token-based authentication with the Elasticsearch API key service.
+The API key returned by the {ref}/security-api-create-api-key.html[Elasticsearch create API key API]
+can be used by sending a request with an `Authorization` header that has a value of `ApiKey` followed by the `{credentials}`,
+where `{credentials}` is the base64 encoding of `id` and `api_key` joined by a colon.
++
+Alternatively, you can create a user and use their username and password to authenticate API access: `-u $USER:$PASSWORD`.
++
+Whether using `Authorization: ApiKey {credentials}`, or `-u $USER:$PASSWORD`,
+users interacting with APM APIs must have <<apm-app-api-user,sufficient privileges>>.
+
+`kbn-xsrf: true`::
+ By default, you must use `kbn-xsrf` for all API calls, except in the following scenarios:
+
+* The API endpoint uses the `GET` or `HEAD` operations
+* The path is whitelisted using the <<settings, `server.xsrf.whitelist`>> setting
+* XSRF protections are disabled using the `server.xsrf.disableProtection` setting
+
+`Content-Type: application/json`::
+ Applicable only when you send a payload in the API request.
+ {kib} API requests and responses use JSON.
+ Typically, if you include the `kbn-xsrf` header, you must also include the `Content-Type` header.
+// end::using-the-APIs[]
+
 Here's an example CURL request that adds an annotation to the APM app:
 
 [source,curl]
@@ -38,9 +69,6 @@ curl -X POST \
  }'
 ----
 
-The Kibana <<api,REST API reference>> provides additional information on how to use Kibana APIs,
-required request headers, and token-based authentication options.
-
 ////
 *******************************************************
 ////
@@ -59,7 +87,15 @@ The following Agent configuration APIs are available:
 * <<apm-list-config>> to list all Agent configurations.
 * <<apm-search-config>> to search for an Agent configuration.
 
-See <<apm-app-api-config-manager>> for information on the privileges required to use this API endpoint.
+[float]
+[[use-agent-config-api]]
+==== How to use APM APIs
+
+.Expand for required headers, privileges, and usage details
+[%collapsible%closed]
+======
+include::api.asciidoc[tag=using-the-APIs]
+======
 
 ////
 *******************************************************
@@ -100,7 +136,7 @@ See <<apm-app-api-config-manager>> for information on the privileges required to
 [[apm-update-config-example]]
 ===== Example
 
-[source,console]
+[source,curl]
 --------------------------------------------------
 PUT /api/apm/settings/agent-configuration
 {
@@ -150,7 +186,7 @@ PUT /api/apm/settings/agent-configuration
 [[apm-delete-config-example]]
 ===== Example
 
-[source,console]
+[source,curl]
 --------------------------------------------------
 DELETE /api/apm/settings/agent-configuration
 {
@@ -228,7 +264,7 @@ DELETE /api/apm/settings/agent-configuration
 [[apm-list-config-example]]
 ===== Example
 
-[source,console]
+[source,curl]
 --------------------------------------------------
 GET /api/apm/settings/agent-configuration
 --------------------------------------------------
@@ -293,7 +329,7 @@ GET /api/apm/settings/agent-configuration
 [[apm-search-config-example]]
 ===== Example
 
-[source,console]
+[source,curl]
 --------------------------------------------------
 POST /api/apm/settings/agent-configuration/search
 {
@@ -317,17 +353,25 @@ POST /api/apm/settings/agent-configuration/search
 The Annotation API allows you to annotate visualizations in the APM app with significant events, like deployments,
 allowing you to easily see how these events are impacting the performance of your existing applications.
 
+By default, annotations are stored in a newly created `observability-annotations` index.
+The name of this index can be changed in your `config.yml` by editing `xpack.observability.annotations.index`.
+
 The following APIs are available:
 
 * <<apm-annotation-create>> to create an annotation for APM.
 // * <<obs-annotation-create>> POST /api/observability/annotation
 // * <<obs-annotation-get>> GET /api/observability/annotation/:id
 // * <<obs-annotation-delete>> DELETE /api/observability/annotation/:id
 
-By default, annotations are stored in a newly created `observability-annotations` index.
-The name of this index can be changed in your `config.yml` by editing `xpack.observability.annotations.index`.
+[float]
+[[use-annotation-api]]
+==== How to use APM APIs
 
-See <<apm-app-api-annotation-manager>> for information on the privileges required to use this API endpoint.
+.Expand for required headers, privileges, and usage details
+[%collapsible%closed]
+======
+include::api.asciidoc[tag=using-the-APIs]
+======
 
 ////
 *******************************************************
@@ -374,19 +418,20 @@ While you can add additional tags, you cannot remove the `apm` tag.
 
 The following example creates an annotation for a service named `opbeans-java`.
 
-[source,console]
+[source,curl]
 --------------------------------------------------
-POST /api/apm/services/opbeans-java/annotation
-{
- "@timestamp": "2020-05-08T10:31:30.452Z",
- "service": {
- "version": "1.2"
- },
- "message": "Deployment 1.2",
- "tags": [
- "elastic.co", "customer"
- ]
-}
+curl -X POST \
+ http://localhost:5601/api/apm/services/opbeans-java/annotation \
+-H 'Content-Type: application/json' \
+-H 'kbn-xsrf: true' \
+-H 'Authorization: Basic YhUlubWZhM0FDbnlQeE6WRtaW49FQmSGZ4RUWXdX' \
+-d '{
+ "@timestamp": "2020-05-08T10:31:30.452Z",
+ "service": {
+ "version": "1.2"
+ },
+ "message": "Deployment 1.2"
+ }'
 --------------------------------------------------
 
 [[apm-annotation-config-body]]

docs/apm/deployment-annotations.asciidoc

@@ -18,7 +18,7 @@ Alternatively, you can explicitly create deployment annotations with our annotat
 The API can integrate into your CI/CD pipeline,
 so that each time you deploy, a POST request is sent to the annotation API endpoint:
 
-[source,console]
+[source,curl]
 ----
 curl -X POST \
  http://localhost:5601/api/apm/services/${SERVICE_NAME}/annotation \ <1>

docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.bulkcreate.md

@@ -9,5 +9,5 @@ Creates multiple documents at once
 <b>Signature:</b>
 
 ```typescript
-bulkCreate: (objects?: SavedObjectsBulkCreateObject<unknown>[], options?: SavedObjectsBulkCreateOptions) => Promise<SavedObjectsBatchResponse<unknown>>;
+bulkCreate: (objects?: SavedObjectsBulkCreateObject[], options?: SavedObjectsBulkCreateOptions) => Promise<SavedObjectsBatchResponse<unknown>>;
 ```

docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.bulkget.md

@@ -9,10 +9,10 @@ Returns an array of objects by id
 <b>Signature:</b>
 
 ```typescript
-bulkGet: (objects?: {
+bulkGet: (objects?: Array<{
  id: string;
  type: string;
- }[]) => Promise<SavedObjectsBatchResponse<unknown>>;
+ }>) => Promise<SavedObjectsBatchResponse<unknown>>;
 ```
 
 ## Example

docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.delete.md

@@ -9,5 +9,5 @@ Deletes an object
 <b>Signature:</b>
 
 ```typescript
-delete: (type: string, id: string) => Promise<{}>;
+delete: (type: string, id: string) => ReturnType<SavedObjectsApi['delete']>;
 ```

docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.find.md

@@ -9,5 +9,5 @@ Search for objects
 <b>Signature:</b>
 
 ```typescript
-find: <T = unknown>(options: Pick<SavedObjectFindOptionsServer, "search" | "filter" | "type" | "page" | "perPage" | "sortField" | "fields" | "searchFields" | "hasReference" | "defaultSearchOperator">) => Promise<SavedObjectsFindResponsePublic<T>>;
+find: <T = unknown>(options: SavedObjectsFindOptions) => Promise<SavedObjectsFindResponsePublic<T>>;
 ```

docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.md

@@ -20,11 +20,11 @@ The constructor for this class is marked as internal. Third-party code should no
 
 | Property | Modifiers | Type | Description |
 | --- | --- | --- | --- |
-| [bulkCreate](./kibana-plugin-core-public.savedobjectsclient.bulkcreate.md) | | <code>(objects?: SavedObjectsBulkCreateObject&lt;unknown&gt;[], options?: SavedObjectsBulkCreateOptions) =&gt; Promise&lt;SavedObjectsBatchResponse&lt;unknown&gt;&gt;</code> | Creates multiple documents at once |
-| [bulkGet](./kibana-plugin-core-public.savedobjectsclient.bulkget.md) | | <code>(objects?: {</code><br/><code> id: string;</code><br/><code> type: string;</code><br/><code> }[]) =&gt; Promise&lt;SavedObjectsBatchResponse&lt;unknown&gt;&gt;</code> | Returns an array of objects by id |
+| [bulkCreate](./kibana-plugin-core-public.savedobjectsclient.bulkcreate.md) | | <code>(objects?: SavedObjectsBulkCreateObject[], options?: SavedObjectsBulkCreateOptions) =&gt; Promise&lt;SavedObjectsBatchResponse&lt;unknown&gt;&gt;</code> | Creates multiple documents at once |
+| [bulkGet](./kibana-plugin-core-public.savedobjectsclient.bulkget.md) | | <code>(objects?: Array&lt;{</code><br/><code> id: string;</code><br/><code> type: string;</code><br/><code> }&gt;) =&gt; Promise&lt;SavedObjectsBatchResponse&lt;unknown&gt;&gt;</code> | Returns an array of objects by id |
 | [create](./kibana-plugin-core-public.savedobjectsclient.create.md) | | <code>&lt;T = unknown&gt;(type: string, attributes: T, options?: SavedObjectsCreateOptions) =&gt; Promise&lt;SimpleSavedObject&lt;T&gt;&gt;</code> | Persists an object |
-| [delete](./kibana-plugin-core-public.savedobjectsclient.delete.md) | | <code>(type: string, id: string) =&gt; Promise&lt;{}&gt;</code> | Deletes an object |
-| [find](./kibana-plugin-core-public.savedobjectsclient.find.md) | | <code>&lt;T = unknown&gt;(options: Pick&lt;SavedObjectFindOptionsServer, &quot;search&quot; &#124; &quot;filter&quot; &#124; &quot;type&quot; &#124; &quot;page&quot; &#124; &quot;perPage&quot; &#124; &quot;sortField&quot; &#124; &quot;fields&quot; &#124; &quot;searchFields&quot; &#124; &quot;hasReference&quot; &#124; &quot;defaultSearchOperator&quot;&gt;) =&gt; Promise&lt;SavedObjectsFindResponsePublic&lt;T&gt;&gt;</code> | Search for objects |
+| [delete](./kibana-plugin-core-public.savedobjectsclient.delete.md) | | <code>(type: string, id: string) =&gt; ReturnType&lt;SavedObjectsApi['delete']&gt;</code> | Deletes an object |
+| [find](./kibana-plugin-core-public.savedobjectsclient.find.md) | | <code>&lt;T = unknown&gt;(options: SavedObjectsFindOptions) =&gt; Promise&lt;SavedObjectsFindResponsePublic&lt;T&gt;&gt;</code> | Search for objects |
 | [get](./kibana-plugin-core-public.savedobjectsclient.get.md) | | <code>&lt;T = unknown&gt;(type: string, id: string) =&gt; Promise&lt;SimpleSavedObject&lt;T&gt;&gt;</code> | Fetches a single object |
 
 ## Methods

docs/development/core/public/kibana-plugin-core-public.savedobjectsfindoptions.md

@@ -21,6 +21,7 @@ export interface SavedObjectsFindOptions extends SavedObjectsBaseOptions
 | [hasReference](./kibana-plugin-core-public.savedobjectsfindoptions.hasreference.md) | <code>{</code><br/><code> type: string;</code><br/><code> id: string;</code><br/><code> }</code> | |
 | [page](./kibana-plugin-core-public.savedobjectsfindoptions.page.md) | <code>number</code> | |
 | [perPage](./kibana-plugin-core-public.savedobjectsfindoptions.perpage.md) | <code>number</code> | |
+| [preference](./kibana-plugin-core-public.savedobjectsfindoptions.preference.md) | <code>string</code> | An optional ES preference value to be used for the query \* |
 | [search](./kibana-plugin-core-public.savedobjectsfindoptions.search.md) | <code>string</code> | Search documents using the Elasticsearch Simple Query String syntax. See Elasticsearch Simple Query String <code>query</code> argument for more information |
 | [searchFields](./kibana-plugin-core-public.savedobjectsfindoptions.searchfields.md) | <code>string[]</code> | The fields to perform the parsed query against. See Elasticsearch Simple Query String <code>fields</code> argument for more information |
 | [sortField](./kibana-plugin-core-public.savedobjectsfindoptions.sortfield.md) | <code>string</code> | |

docs/development/core/public/kibana-plugin-core-public.savedobjectsfindoptions.preference.md

@@ -0,0 +1,13 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-core-public](./kibana-plugin-core-public.md) &gt; [SavedObjectsFindOptions](./kibana-plugin-core-public.savedobjectsfindoptions.md) &gt; [preference](./kibana-plugin-core-public.savedobjectsfindoptions.preference.md)
+
+## SavedObjectsFindOptions.preference property
+
+An optional ES preference value to be used for the query \*
+
+<b>Signature:</b>
+
+```typescript
+preference?: string;
+```

docs/development/core/public/kibana-plugin-core-public.scopedhistory.createhref.md

@@ -11,5 +11,5 @@ Creates an href (string) to the location. If `prependBasePath` is true (default)
 ```typescript
 createHref: (location: LocationDescriptorObject<HistoryLocationState>, { prependBasePath }?: {
  prependBasePath?: boolean | undefined;
- }) => string;
+ }) => Href;
 ```