Merge branch 'main' into deprecate_endpoints

.gitignore

@@ -66,6 +66,7 @@ npm-debug.log*
 .ci/bash_standard_lib.sh
 .gradle
 .vagrant
+.envrc
 
 ## @cypress/snapshot from apm plugin
 /snapshots.js

docs/CHANGELOG.asciidoc

@@ -21,8 +21,6 @@ Review important information about the {kib} 8.0.0 releases.
 [[release-notes-8.0.0]]
 == {kib} 8.0.0
 
-coming::[8.0.0]
-
 Review the {kib} 8.0.0 changes, then use the {kibana-ref-all}/7.17/upgrade-assistant.html[Upgrade Assistant] to complete the upgrade.
 
 [float]
@@ -78,6 +76,34 @@ To review the deprecations in previous versions, refer to the following:
 
 <<deprecations-8.0.0-rc1,8.0.0-rc1>> | <<deprecations-8.0.0-alpha1,8.0.0-alpha1>>
 
+[float]
+[[known-issue-8.0.0]]
+=== Known issue
+
+[discrete]
+[[known-issue-123550]]
+.Importing and copying saved objects causes weak links to break
+[%collapsible]
+====
+*Details* +
+{kib} supports weak links in some saved objects. For example, a dashboard may include a Markdown panel that contains a relative URL to
+another dashboard. Weak links are defined by free text, _not_ the saved object's relationships, and can break if **both** of the following
+conditions are true:
+
+* You are importing saved objects into multiple spaces, _OR_ you are copying saved objects into another space
+* Before you upgraded to 8.0.0, the saved objects did not already exist in the destinations
+
+In 8.0.0 and later, weak links break because <<saved-object-ids-impact-when-using-import-and-copy,saved object IDs can change during import or copy>>.
+This applies to both the UI and the API.
+This issue will be fixed 8.0.1 and 8.1.0. For more information, refer to {kibana-issue}123550[#123550].
+
+*Impact* +
+Saved objects in 7.x that are migrated during upgrade are **not** impacted.
+Only _new_ saved objects that are imported or copied _multiple times_ (causing object IDs to change) are impacted.
+If you are impacted, you can re-import or re-copy your saved objects after the fix is
+implemented to preserve the weak links.
+====
+
 [float]
 [[features-8.0.0]]
 === Features
@@ -153,7 +179,7 @@ To review the breaking changes in previous versions, refer to the following:
 
 <<breaking-changes-8.0.0-rc1,8.0.0-rc1>> | <<breaking-changes-8.0.0-beta1,8.0.0-beta1>> | <<breaking-changes-8.0.0-alpha2,8.0.0-alpha2>> | 
 <<breaking-changes-8.0.0-alpha1,8.0.0-alpha1>>
- 
+
 [float]
 [[features-8.0.0-rc2]]
 === Features

docs/developer/advanced/sharing-saved-objects.asciidoc

@@ -405,8 +405,8 @@ necessary. However, such handling of secondary objects is not considered critica
 ==== 4. What is a "legacy URL alias"?
 
 As depicted above, when an object is converted to become share-capable, if it exists in a non-Default space, its ID gets changed. To
-preserve its old ID, we also create a special object called a _legacy URL alias_ ("alias" for short); this alias retains the target object's
-old ID (_sourceId_), and it contains a pointer to the target object's new ID (_targetId_).
+preserve its old ID, we also create a special object called a <<legacy-url-aliases,_legacy URL alias_>> ("alias" for short); this alias
+retains the target object's old ID (_sourceId_), and it contains a pointer to the target object's new ID (_targetId_).
 
 Aliases are meant to be mostly invisible to end-users by design. There is no UI to manage them directly. Our vision is that aliases will be
 used as a stop-gap to help us through the 8.0 upgrade process, but we will nudge users away from relying on aliases so we can eventually
@@ -473,3 +473,7 @@ change any other data flows to use `resolve()`.
 External plugins (those not shipped with {kib}) can use this guide to convert any isolated objects to become share-capable or fully
 shareable! If you are an external plugin developer, the steps are the same, but you don't need to worry about getting anything done before a
 specific release. The only thing you need to know is that your plugin cannot convert your objects until the 8.0 release.
+
+==== 8. How will users be impacted?
+
+Refer to <<saved-object-ids,Saved Object IDs>> documentation for more details how users should expect to be impacted.

docs/developer/architecture/core/index.asciidoc

@@ -36,9 +36,3 @@ The services that core provides are:
 * <<logging-service, Logging service>>
 * <<saved-objects-service, Saved Objects service>>
 * <<ui-settings-service, UI settings service>>
-
-
-
-
-
-

docs/developer/architecture/index.asciidoc

@@ -40,6 +40,8 @@ include::core/http-service.asciidoc[leveloffset=+1]
 
 include::core/logging-service.asciidoc[leveloffset=+1]
 
+include::core/logging-configuration-migration.asciidoc[leveloffset=+1]
+
 include::core/saved-objects-service.asciidoc[leveloffset=+1]
 
 include::core/uisettings-service.asciidoc[leveloffset=+1]

docs/management/managing-saved-objects.asciidoc

@@ -126,3 +126,5 @@ WARNING: Validation is not performed for object properties. Submitting an invali
 change will render the object unusable. A more failsafe approach is to use
 *Discover* or *Dashboard* to create new objects instead of
 directly editing an existing one.
+
+include::saved-objects/saved-object-ids.asciidoc[]

docs/management/saved-objects/saved-object-ids.asciidoc

@@ -0,0 +1,87 @@
+[[saved-object-ids]]
+=== Saved Object IDs
+
+In the past, many saved object types could have the same ID in different <<xpack-spaces,spaces>>. For example, if you copied dashboard "123"
+from the one space to another space, the second dashboard would also have an ID of "123". While the saved object ID is not something
+that users would interact with directly, many aspects of {kib} rely on it, notably URLs. If you have a "deep link" URL to a saved dashboard,
+that URL includes the saved object ID.
+
+**Starting in the 8.0 release**, {kib} requires most saved objects to have _globally unique_ IDs. This is a change that we needed to make to
+support sharing saved objects to multiple spaces. Most saved objects cannot be shared to multiple spaces _yet_, but we needed to start
+enforcing globally unique object IDs first.
+
+We have made several enhancements to minimize the impact, and this document describes what you need to know about the changes and
+how it will affect you.
+
+[[saved-object-ids-impact-upon-upgrading]]
+==== Impact upon upgrading to 8.x
+
+Every time you upgrade {kib}, <<saved-object-migrations,saved objects are migrated to a new format>>. When you
+first upgrade from 7.x to 8.x, this migration process will start enforcing globally unique saved object IDs.
+
+In practical terms, **any old saved objects that exist in a custom space will have their IDs changed to a new UUID**, while saved objects in
+the Default space will be unchanged. This is how we can ensure that every saved object ID is unique. For example: if you had dashboard "123"
+in the Default space and dashboard "123" in Another space, after the upgrade you would have dashboard "123" in the Default space and
+dashboard "456" in Another space.
+
+[[saved-object-ids-impact-when-using]]
+==== Impact when using 8.x
+
+After you upgrade, or if you set up a new {kib} instance using 8.x, there are a few more things that behave differently.
+
+[[saved-object-ids-impact-when-using-legacy-urls]]
+===== Accessing saved objects using old URLs
+
+When you upgrade {kib} and saved object IDs change, the "deep link" URLs to access those saved objects will also change. To reduce the impact,
+each existing URL is preserved with a special <<legacy-url-aliases,legacy URL alias>>. This means that if you use a bookmark for
+a saved object ID that was changed, you'll be redirected to the new URL for that saved object.
+
+[[saved-object-ids-impact-when-using-import-and-copy]]
+===== Importing and copying saved objects
+
+When you <<managing-saved-objects-copy-to-space,copy a saved object to another space>>, {kib} effectively
+<<managing-saved-objects-export-objects,exports it and imports it into that space>>. In this way, copying a saved object has always behaved
+like an import. In this document when we say "import", it applies to both features.
+
+Historically, whether you imported or copied a saved object, {kib} would create _at most_ one copy of a saved object in that space. If you
+imported the saved object multiple times, {kib} would overwrite the existing object, because it used the same ID. Since saved object IDs are
+now globally unique, {kib} maintains this functionality by tracking each saved object's _origin_. When you import an object in 8.x, {kib}
+uses either the saved object ID _or_ the origin to determine its destination.
+
+If you import a saved object using the "Check for existing objects" option -- whether it was exported from 7.x or 8.x -- {kib} will
+take the following steps:
+
+1. If {kib} finds a matching saved object with the exact same ID in the target space, that will be the import destination -- you can **overwrite** that
+destination or **skip** it.
+
+2. Otherwise, if {kib} finds a matching saved object with a _different_ ID that has the same origin, that will be the import destination
+-- again, you can **overwrite** that destination or **skip** it.
+
+3. Otherwise, if a saved object with the exact same ID exists in a _different_ space, then {kib} will generate a random ID for the import
+destination, preserving the saved object's origin.
+
+4. Otherwise, {kib} creates the saved object with the given ID.
+
+For example, you have a saved object in an `export.ndjson` file, and you set up a brand new {kib} instance. You attempt to import the saved
+object using the "Check for existing objects" and "Automatically overwrite conflicts" options. The first time you import the saved object,
+{kib} will create a new object with the same ID (step 4 above). If you import it again, {kib} will find that object and overwrite it (step 1
+above). If you then create a _different_ space and import it there, {kib} will create a new object with a random ID (step 3 above). Finally,
+if you import it into the second space again, {kib} will find the second object with a matching origin and overwrite it (step 2 above).
+
+WARNING: When you import a saved object and it is created with a different ID, if 1. it contains weak links to other saved objects (such as
+a dashboard with a Markdown URL to navigate to another dashboard) and 2. the object's ID has changed (step 3 above), those weak links will
+be broken. For more information, refer to <<known-issue-123550,the known issue in the changelog>>.
+
+[[saved-object-ids-impact-when-using-apis]]
+===== Using the saved objects APIs
+
+If you are using the saved objects APIs directly, you should be aware of these changes:
+
+* When using the <<saved-objects-api-create,create>> or <<saved-objects-api-bulk-create,bulk create>> API, you may encounter
+  <<saved-objects-api-bulk-create-conflict-errors,conflict errors>> that **cannot** be overridden using the `overwrite: true`
+  option. This can occur if there is already a saved object with this ID in a _different_ space, or if there is a legacy URL alias for this
+  ID in the same space.
+* When using the <<saved-objects-api-import,import>> or <<spaces-api-copy-saved-objects,copy to space>> API, objects can potentially be
+  created with a different ID as described above.
+* When using the <<saved-objects-api-delete,delete>> API, if the saved object exists in multiple spaces, it can only be deleted by using the
+  <<saved-objects-api-delete-query-params,`force` option>>.

docs/migration/migrate_8_0.asciidoc

@@ -7,8 +7,6 @@
 This section discusses the changes that you need to be aware of when migrating
 your application to Kibana 8.0.
 
-coming[8.0.0]
-
 See also <<whats-new>> and <<release-notes>>.
 
 * <<breaking_80_index_pattern_changes>>

docs/redirects.asciidoc

@@ -386,16 +386,16 @@ This content has moved. Refer to <<managing-data-views>>.
 
 This content has moved. Refer to <<kibana-role-management>>.
 
-[role="exclude",id="logging-configuration-changes"]
-== Logging configuration changes
-
-This content has moved. Refer to <<logging-config-changes>>.
-
 [role="exclude",id="upgrade-migrations"]
 == Upgrade migrations
 
 This content has moved. Refer to <<saved-object-migrations>>.
 
+[role="exclude",id="upgrade-standard"]
+== Standard Upgrade
+
+This content has moved. Refer to {stack-ref}/upgrading-kibana.html[Upgrade Kibana].
+
 [role="exclude",id="upgrade-assistant"]
 == Upgrade Assistant
 

docs/settings/apm-settings.asciidoc

@@ -74,7 +74,7 @@ Changing these settings may disable features of the APM App.
   | Index name where Observability annotations are stored. Defaults to `observability-annotations`.
 
 | `xpack.apm.searchAggregatedTransactions` {ess-icon}
-  | experimental[] Enables Transaction histogram metrics. Defaults to `never` and aggregated transactions are not used. When set to `auto`, the UI will use metric indices over transaction indices for transactions if aggregated transactions are found. When set to `always`, additional configuration in APM Server is required.
+  | Enables Transaction histogram metrics. Defaults to `auto` so the UI will use metric indices over transaction indices for transactions if aggregated transactions are found. When set to `always`, additional configuration in APM Server is required. When set to `never` and aggregated transactions are not used.
     See {apm-guide-ref}/transaction-metrics.html[Configure transaction metrics] for more information.
 
 | `xpack.apm.metricsInterval` {ess-icon}

docs/setup/upgrade.asciidoc

@@ -1,31 +1,72 @@
 [[upgrade]]
 == Upgrade {kib}
 
-To upgrade from 7.16.0 or earlier to {version}, 
+To upgrade from 7.16.0 or earlier to {version},
 **you must first upgrade to {prev-major-last}**, which enables you to use the *Upgrade Assistant* to
 {stack-ref}/upgrading-elastic-stack.html#prepare-to-upgrade[prepare for the upgrade].
-Before you upgrade, you must resolve all critical issues identified by the *Upgrade Assistant*. 
+Before you upgrade, you must resolve all critical issues identified by the *Upgrade Assistant*.
 
-Rolling upgrades are unsupported in {kib}. To upgrade,  
+Rolling upgrades are unsupported in {kib}. To upgrade,
 you must shut down all {kib} instances, install the new software, and restart {kib}.
 Upgrading while older {kib} instances are running can cause data loss or upgrade failures.
 
 [WARNING]
 ====
-When required, {kib} automatically migrates <<saved-object-migrations, saved objects>>. 
+When required, {kib} automatically migrates <<saved-object-migrations, saved objects>>.
 In case of an upgrade failure, you can roll back to an
 earlier version of {kib}. To roll back, you **must** have a
 {ref}/snapshot-restore.html[backup snapshot] that includes the `kibana` feature
 state. By default, snapshots include the `kibana` feature state.
 ====
 
-For more information about upgrading, 
+For more information about upgrading,
 refer to {stack-ref}/upgrading-elastic-stack.html[Upgrading to Elastic {version}.]
 
-IMPORTANT: You can upgrade to pre-release versions for testing, 
+IMPORTANT: You can upgrade to pre-release versions for testing,
 but upgrading from a pre-release to the General Available version is unsupported.
 You should use pre-release versions only for testing in a temporary environment.
 
-include::upgrade/upgrade-migrations.asciidoc[leveloffset=-1]
+[float]
+=== Upgrading multiple {kib} instances
+When upgrading several {kib} instances connected to the same {es} cluster,
+ensure that all outdated instances are shut down before starting the upgrade.
 
-include::upgrade/logging-configuration-changes.asciidoc[]
+Rolling upgrades are unsupported in {kib}. However, when outdated instances are shut down, you can start all upgraded instances in parallel,
+which allows all instances to participate in the upgrade migration in parallel.
+
+For large deployments with more than 10 {kib} instances, and more than 10,000 saved objects,
+you can reduce the upgrade downtime by bringing up a single {kib} instance and waiting for it to
+complete the upgrade migration before bringing up the remaining instances.
+
+[float]
+[[preventing-migration-failures]]
+=== Preparing for migration
+
+There are extra steps you can follow to ensure you are ready for migration.
+
+[float]
+==== Ensure your {es} cluster is healthy
+Problems with your {es} cluster can prevent {kib} upgrades from succeeding. Ensure that your cluster has:
+
+ * Enough free disk space, at least twice the amount of storage taken up by the `.kibana` and `.kibana_task_manager` indices
+ * Sufficient heap size
+ * A "green" cluster status
+
+[float]
+==== Ensure that all {kib} instances are the same
+When you perform an upgrade migration of different {kib} versions, the migration can fail.
+Ensure that all {kib} instances are running the same version, configuration, and plugins.
+
+[float]
+==== Back up your data
+Be sure to have a {ref}/snapshot-restore.html[snapshot] of all your data before attempting a migration.
+If something goes wrong during migration, you can restore from the snapshot and try again.
+
+Review the <<resolve-migrations-failures,common causes of {kib} upgrade failures>> and how to prevent them.
+
+
+include::upgrade/saved-objects-migration.asciidoc[]
+
+include::upgrade/resolving-migration-failures.asciidoc[]
+
+include::upgrade/rollback-migration.asciidoc[]