diff --git a/docs/reference/ccr/auto-follow.asciidoc b/docs/reference/ccr/auto-follow.asciidoc index d83c2eb0555e3..d072dd8022b12 100644 --- a/docs/reference/ccr/auto-follow.asciidoc +++ b/docs/reference/ccr/auto-follow.asciidoc @@ -1,32 +1,82 @@ [role="xpack"] [testenv="platinum"] [[ccr-auto-follow]] -=== Automatically following indices - -In time series use cases where you want to follow new indices that are -periodically created (such as daily Beats indices), manually configuring follower -indices for each new leader index can be an operational burden. The auto-follow -functionality in {ccr} is aimed at easing this burden. With the auto-follow -functionality, you can specify that new indices in a remote cluster that have a -name that matches a pattern are automatically followed. - -==== Managing auto-follow patterns - -You can add a new auto-follow pattern configuration with the -{ref}/ccr-put-auto-follow-pattern.html[create auto-follow pattern API]. When you create -a new auto-follow pattern configuration, you are configuring a collection of -patterns against a single remote cluster. Any time a new index with a name that -matches one of the patterns in the collection is created in the remote cluster, -a follower index is configured in the local cluster. The follower index uses the -new index as its leader index. - -You can inspect all configured auto-follow pattern collections with the -{ref}/ccr-get-auto-follow-pattern.html[get auto-follow pattern API]. Auto-follow patterns -can be paused using the {ref}/ccr-pause-auto-follow-pattern.html[pause auto-follow pattern API] - and can be resumed using the {ref}/ccr-resume-auto-follow-pattern.html[resume auto-follow pattern API]. - To delete a configured auto-follow pattern collection, use the -{ref}/ccr-delete-auto-follow-pattern.html[delete auto-follow pattern API]. - -Since auto-follow functionality is handled automatically in the background on -your behalf, error reporting is done through logs on the elected master node -and through the {ref}/ccr-get-stats.html[{ccr} stats API]. +=== Manage auto-follow patterns +To replicate time series indices, you configure an auto-follow pattern so that +each new index in the series is replicated automatically. Whenever the name of +a new index on the remote cluster matches the auto-follow pattern, a +corresponding follower index is added to the local cluster. + +Auto-follow patterns are especially useful with +<>, which might continually create +new indices on the cluster containing the leader index. + +[[ccr-access-ccr-auto-follow]] +To start using {ccr} auto-follow patterns, access {kib} and go to +*Management > Stack Management*. In the side navigation, select +*Cross-Cluster Replication* and choose the *Auto-follow patterns* tab + +[[ccr-auto-follow-create]] +==== Create auto-follow patterns +When you <>, +you are configuring a collection of patterns against a single remote cluster. +When an index is created in the remote cluster with a name that matches one of +the patterns in the collection, a follower index is configured in the local +cluster. The follower index uses the new index as its leader index. + +[%collapsible] +.Use the API +==== +Use the <> to add a +new auto-follow pattern configuration. +==== + +[[ccr-auto-follow-retrieve]] +==== Retrieve auto-follow patterns +To view existing auto-follow patterns and make changes to the backing +patterns, <> on your _remote_ cluster. + +Select the auto-follow pattern that you want to view details about. From there, +you can make changes to the auto-follow pattern. You can also view your +follower indices included in the auto-follow pattern. + +[%collapsible] +.Use the API +==== +Use the <> to inspect +all configured auto-follow pattern collections. +==== + +[[ccr-auto-follow-pause]] +==== Pause and resume auto-follow patterns +To pause and resume replication of auto-follow pattern collections, +<>, select the auto-follow pattern, +and pause replication. + +To resume replication, select the pattern and choose +*Manage pattern > Resume replication*. + +[%collapsible] +.Use the API +==== +Use the <> to +pause auto-follow patterns. +Use the <> to +resume auto-follow patterns. +==== + +[[ccr-auto-follow-delete]] +==== Delete auto-follow patterns +To delete an auto-follow pattern collection, +<>, select the auto-follow pattern, +and pause replication. + +When the pattern status changes to Paused, choose +*Manage pattern > Delete pattern*. + +[%collapsible] +.Use the API +==== +Use the <> to +delete a configured auto-follow pattern collection. +==== diff --git a/docs/reference/ccr/getting-started.asciidoc b/docs/reference/ccr/getting-started.asciidoc index 5c178fd45c4b6..ef783c858c33c 100644 --- a/docs/reference/ccr/getting-started.asciidoc +++ b/docs/reference/ccr/getting-started.asciidoc @@ -1,47 +1,39 @@ [role="xpack"] [testenv="platinum"] [[ccr-getting-started]] -=== Getting started with {ccr} +=== Set up {ccr} +You can manually create follower indices to replicate specific indices on a +remote cluster, or configure auto-follow patterns to automatically create +follower indices for new time series. -This getting-started guide for {ccr} shows you how to: +After the follower index is created, the +<> process copies all of the Lucene +segment files from the remote cluster to the local cluster. -* <> -* <> in a remote cluster -* <> that replicates - a leader index -* <> +To set up {ccr}: +. <> +. <> +. <> +. Manually create a follower index or create an auto-follow pattern: + * To replicate the leader index, <> + * To automatically follow time series indices, <> -[[ccr-getting-started-before-you-begin]] -==== Before you begin -. {stack-gs}/get-started-elastic-stack.html#install-elasticsearch[Install {es}] - on your local and remote clusters. +[[ccr-getting-started-prerequisites]] +==== Prerequisites +If the Elastic {security-features} are enabled in your local and remote +clusters, you need a user with appropriate authority to complete the steps +in this tutorial. -. Obtain a license that includes the {ccr} features. See - https://www.elastic.co/subscriptions[subscriptions] and - {kibana-ref}/managing-licenses.html[License management]. +By default, you can complete the following steps as the built-in +`elastic` user. However, you must <> +for this user before proceeding. -. If the Elastic {security-features} are enabled in your local and remote - clusters, you need a user that has appropriate authority to perform the steps - in this tutorial. -+ --- -[[ccr-getting-started-security]] -The {ccr} features use cluster privileges and built-in roles to make it easier -to control which users have authority to manage {ccr}. - -By default, you can perform all of the steps in this tutorial by -using the built-in `elastic` user. However, a password must be set for this user -before the user can do anything. For information about how to set that password, -see <>. - -If you are performing these steps in a production environment, take extra care -because the `elastic` user has the `superuser` role and you could inadvertently -make significant changes. +WARNING: If you are performing these steps in a production environment, do +not use the `elastic` user. Alternatively, you can assign the appropriate privileges to a user ID of your -choice. On the remote cluster that contains the leader index, a user will need +choice. On the remote cluster that contains the leader index, a user must have the `read_ccr` cluster privilege and `monitor` and `read` privileges on the leader index. @@ -76,19 +68,31 @@ ccr_user: -------------------------------------------------- If you are managing -<> via the -cluster update settings API, you will also need a user with the `all` cluster -privilege. --- - +<> using +the cluster update settings API, you will also need a user with the `all` +cluster privilege. [[ccr-getting-started-remote-cluster]] -==== Connecting to a remote cluster +==== Connect to a remote cluster +Connect your local cluster to a +<> to begin using cross-cluster +replication. + +To configure a {kibana-ref}/working-remote-clusters.html[remote cluster], +access {kib} and go to +*Management > Stack Management*. In the side navigation, select +*Remote Clusters*. + +Add a remote cluster by specifying the IP address or host name, followed by the +transport port of the remote cluster. + +[role="screenshot"] +image::images/ccr-add-remote-cluster.png["The Add remote clusters page in {kib}"] -The {ccr} features require that you -{ref}/modules-remote-clusters.html[connect your local cluster to a remote -cluster]. In this tutorial, we will connect our local cluster to a remote -cluster with the cluster alias `leader`. +[%collapsible] +.API example +==== +Use the <> to add a remote cluster: [source,console] -------------------------------------------------- @@ -147,22 +151,19 @@ remote cluster. alias `leader` <2> This shows the number of nodes in the remote cluster the local cluster is connected to. - -Alternatively, you can {kibana-ref}/working-remote-clusters.html[manage remote clusters] from {kib}: - -. From the {kib} menu, select *Management -> Stack Management*. -. Under *Data*, select *Remote Clusters* to access the Remote Clusters -page. - -[role="screenshot"] -image::images/remote-clusters.png["The Remote Clusters page in {kib}"] - +==== [[ccr-getting-started-leader-index]] -==== Creating a leader index +==== Create a leader index +To create a leader index, access {kib} on your _remote_ cluster and go to +*Management > Dev Tools*. -In the following example, we will create a leader index in the remote cluster: +Copy the following example into the Console to create a leader index named +`server-metrics` in your remote cluster: +[%collapsible] +.Leader index example +==== [source,console] -------------------------------------------------- PUT /server-metrics @@ -201,20 +202,68 @@ PUT /server-metrics } -------------------------------------------------- // TEST[continued] +==== +[[ccr-enable-soft-deletes]] +==== Enable soft deletes on leader indices +<> must be enabled for indices that you want to +use as leader indices. Soft deletes are enabled by default on new indices +created on or after {es} 7.0.0, so +*no further action is required if your cluster is running {es} 7.0.0 or later*. -[[ccr-getting-started-follower-index]] -==== Creating a follower index +include::{es-ref-dir}/ccr/index.asciidoc[tag=ccr-existing-indices-tag] + +To enable soft deletes on indices created on versions of +{es} between 6.5.0 and 7.0.0, set <> to `true`. -Follower indices are created with the {ref}/ccr-put-follow.html[create follower -API]. When you create a follower index, you must reference the +[[ccr-getting-started-follower-index]] +==== Create a follower index +When you create a {kibana-ref}/managing-cross-cluster-replication.html#_create_specific_follower_indices[follower index], you +must reference the <> and the <> that you created in the remote cluster. +To create a follower index, access {kib} and go to +*Management > Stack Management*. In the side navigation, select +*Cross-Cluster Replication* and choose the *Follower Indices* tab. + +. Choose the remote cluster containing the index you want to replicate, which +is `leader` if you are following the tutorial. +. Enter the name of the leader index, which is `server-metrics` if you are +following the tutorial. + +image::images/ccr-add-follower-index.png["Adding a follower index named server-metrics in {kib}"] + +The follower index is initialized using the +<> +process, which transfers the existing Lucene segment files from the leader +index to the follower index. The index status changes to *Paused*. When the +remote recovery process is complete, the index following begins and the status +changes to *Active*. + +When you index documents into your leader index, the documents are replicated +in the follower index. + +[role="screenshot"] +image::images/ccr-follower-index.png["The Cross-Cluster Replication page in {kib}"] + +[%collapsible] +.API example +==== +Use the <> to create follower indices. +When you create a follower index, you must reference the +<> and the +<> that you created in the +remote cluster. + +When initiating the follower request, the response returns before the +<> process completes. To wait for the process +to complete, add the `wait_for_active_shards` parameter to your request. + [source,console] -------------------------------------------------- -PUT /server-metrics-copy/_ccr/follow?wait_for_active_shards=1 +PUT /server-metrics-follower/_ccr/follow?wait_for_active_shards=1 { "remote_cluster" : "leader", "leader_index" : "server-metrics" @@ -235,53 +284,65 @@ PUT /server-metrics-copy/_ccr/follow?wait_for_active_shards=1 ////////////////////////// -The follower index is initialized using the <> -process. The remote recovery process transfers the existing Lucene segment files -from the leader to the follower. When the remote recovery process is complete, -the index following begins. - -Now when you index documents into your leader index, you will see these -documents replicated in the follower index. You can -inspect the status of replication using the -{ref}/ccr-get-follow-stats.html[get follower stats API]. - -Alternatively, you can {kibana-ref}/managing-cross-cluster-replication.html#_create_specific_follower_indices[manage follower indices] using {kib}: - -. From the {kib} menu, click *Management -> Stack Management*. -. Under *Data*, select *Cross Cluster Replication*. -. Choose the *Follower indices* tab. - -[role="screenshot"] -image::images/ccr-follower-index.png["The Cross-Cluster Replication page in {kib}"] +Use the +<> to inspect the status of +replication ////////////////////////// [source,console] -------------------------------------------------- -POST /server-metrics-copy/_ccr/pause_follow +POST /server-metrics-follower/_ccr/pause_follow -POST /server-metrics-copy/_close +POST /server-metrics-follower/_close -POST /server-metrics-copy/_ccr/unfollow +POST /server-metrics-follower/_ccr/unfollow -------------------------------------------------- // TEST[continued] ////////////////////////// - +==== [[ccr-getting-started-auto-follow]] ==== Automatically create follower indices +Create <> to automatically follow time +series indices that are periodically created in a remote cluster (such as daily +{beats} indices). -The <> feature in {ccr} helps for time series use -cases where you want to follow new indices that are periodically created in the -remote cluster (such as daily Beats indices). Auto-following is configured using -the {ref}/ccr-put-auto-follow-pattern.html[create auto-follow pattern API]. With -an auto-follow pattern, you reference the -<> that you connected your -local cluster to. You must also specify a collection of patterns that match the +With an auto-follow pattern, you reference the +<> connected to your +local cluster. You must also specify a collection of patterns that match the indices you want to automatically follow. -For example: +// tag::ccr-create-auto-follow-pattern-tag[] +To create follower indices from an {kibana-ref}/managing-cross-cluster-replication.html#_create_follower_indices_from_an_auto_follow_pattern[auto-follow pattern], +access {kib} on your remote cluster and go to +*Management > Stack Management*. In the side navigation, select +*Cross Cluster Replication* and choose the *Auto-follow patterns* tab. + +[role="screenshot"] +image::images/auto-follow-patterns.png["The Auto-follow patterns page in {kib}"] + +* Enter a name for the auto-follow pattern. For this tutorial, enter `beats` +as the name. +* Choose the remote cluster containing the index you want to replicate, which +is `leader` if you are following the tutorial. +* Enter one or more index patterns that identify the indices you want to +replicate from the remote cluster. For this tutorial, enter +`metricbeat-*,packetbeat-*` as the index pattern. +* Enter *copy-* as the prefix to apply to the names of the follower indices so +you can more easily identify replicated indices. + +As new indices matching these patterns are +created, they are replicated to the follower indices. + +// end::ccr-create-auto-follow-pattern-tag[] + +[%collapsible] +.API example +==== +Use the <> to +configure auto-follow patterns. [source,console] -------------------------------------------------- @@ -322,14 +383,4 @@ DELETE /_ccr/auto_follow/beats // TEST[continued] ////////////////////////// - -Alternatively, you can manage auto-follow patterns from {kib}: - -. From the {kib} menu, click *Management -> Stack Management*. -. Under *Data*, select *Cross Cluster Replication*. -. Choose the *Auto-follow patterns* tab. - -[role="screenshot"] -image::images/auto-follow-patterns.png["The Auto-follow patterns page in {kib}"] - -See {kibana-ref}/managing-cross-cluster-replication.html#_create_follower_indices_from_an_auto_follow_pattern[Create follower indices from an auto-follow pattern] for more information. +==== diff --git a/docs/reference/ccr/images/auto-follow-patterns.png b/docs/reference/ccr/images/auto-follow-patterns.png index a5eb9c406bbf2..626e83445f99b 100644 Binary files a/docs/reference/ccr/images/auto-follow-patterns.png and b/docs/reference/ccr/images/auto-follow-patterns.png differ diff --git a/docs/reference/ccr/images/ccr-add-follower-index.png b/docs/reference/ccr/images/ccr-add-follower-index.png new file mode 100644 index 0000000000000..102440ffb8b6c Binary files /dev/null and b/docs/reference/ccr/images/ccr-add-follower-index.png differ diff --git a/docs/reference/ccr/images/ccr-add-remote-cluster.png b/docs/reference/ccr/images/ccr-add-remote-cluster.png new file mode 100644 index 0000000000000..a5760827823fa Binary files /dev/null and b/docs/reference/ccr/images/ccr-add-remote-cluster.png differ diff --git a/docs/reference/ccr/images/ccr-arch-bi-directional.png b/docs/reference/ccr/images/ccr-arch-bi-directional.png new file mode 100644 index 0000000000000..9c936cd9f4b42 Binary files /dev/null and b/docs/reference/ccr/images/ccr-arch-bi-directional.png differ diff --git a/docs/reference/ccr/images/ccr-arch-central-reporting.png b/docs/reference/ccr/images/ccr-arch-central-reporting.png new file mode 100644 index 0000000000000..55f50af56046d Binary files /dev/null and b/docs/reference/ccr/images/ccr-arch-central-reporting.png differ diff --git a/docs/reference/ccr/images/ccr-arch-chain-dcs.png b/docs/reference/ccr/images/ccr-arch-chain-dcs.png new file mode 100644 index 0000000000000..042a8185c7589 Binary files /dev/null and b/docs/reference/ccr/images/ccr-arch-chain-dcs.png differ diff --git a/docs/reference/ccr/images/ccr-arch-data-locality.png b/docs/reference/ccr/images/ccr-arch-data-locality.png new file mode 100644 index 0000000000000..a2b67b07284f4 Binary files /dev/null and b/docs/reference/ccr/images/ccr-arch-data-locality.png differ diff --git a/docs/reference/ccr/images/ccr-arch-disaster-recovery.png b/docs/reference/ccr/images/ccr-arch-disaster-recovery.png new file mode 100644 index 0000000000000..244c71910ff55 Binary files /dev/null and b/docs/reference/ccr/images/ccr-arch-disaster-recovery.png differ diff --git a/docs/reference/ccr/images/ccr-arch-multiple-dcs.png b/docs/reference/ccr/images/ccr-arch-multiple-dcs.png new file mode 100644 index 0000000000000..2a2fd05a00711 Binary files /dev/null and b/docs/reference/ccr/images/ccr-arch-multiple-dcs.png differ diff --git a/docs/reference/ccr/images/ccr-follower-index.png b/docs/reference/ccr/images/ccr-follower-index.png index 4825b551ed6fd..e2e825081f398 100644 Binary files a/docs/reference/ccr/images/ccr-follower-index.png and b/docs/reference/ccr/images/ccr-follower-index.png differ diff --git a/docs/reference/ccr/index.asciidoc b/docs/reference/ccr/index.asciidoc index 103462670a818..5adf860e9d0d6 100644 --- a/docs/reference/ccr/index.asciidoc +++ b/docs/reference/ccr/index.asciidoc @@ -2,28 +2,301 @@ [testenv="platinum"] [[xpack-ccr]] == {ccr-cap} +With {ccr}, you can replicate indices across clusters to: -The {ccr} (CCR) feature enables replication of indices in remote clusters to a -local cluster. This functionality can be used in some common production use -cases: +* Continue handling search requests in the event of a datacenter outage +* Prevent search volume from impacting indexing throughput +* Reduce search latency by processing search requests in geo-proximity to the +user -* Disaster recovery in case a primary cluster fails. A secondary cluster can - serve as a hot backup -* Geo-proximity so that reads can be served locally -* Prevent search load from interfering with indexing by offloading search to a secondary cluster +{ccr-cap} uses an active-passive model. You index to a _leader_ index, and the +data is replicated to one or more read-only _follower_ indices. Before you can add a follower index to a cluster, you must configure the _remote cluster_ that contains the leader index. -This guide provides an overview of {ccr}: +When the leader index receives writes, the follower indices pull changes from +the leader index on the remote cluster. You can manually create follower +indices, or configure auto-follow patterns to automatically create follower +indices for new time series indices. + +You configure {ccr} clusters in a uni-directional or bi-directional setup: + +* In a uni-directional configuration, one cluster contains only +leader indices, and the other cluster contains only follower indices. +* In a bi-directional configuration, each cluster contains both leader and +follower indices. + +In a uni-directional configuration, the cluster containing follower indices +must be running **the same or newer** version of {es} as the remote cluster. +If newer, the versions must also be compatible as outlined in the following matrix. + +[%collapsible] +[[ccr-version-compatibility]] +.Version compatibility matrix +==== +include::../modules/remote-clusters.asciidoc[tag=remote-cluster-compatibility-matrix] +==== + +[discrete] +[[ccr-multi-cluster-architectures]] +=== Multi-cluster architectures +Use {ccr} to construct several multi-cluster architectures within the Elastic +Stack: + +* <> in case a primary cluster fails, +with a secondary cluster serving as a hot backup +* <> to maintain multiple copies of the +dataset close to the application servers (and users), and reduce costly latency +* <> for minimizing network +traffic and latency in querying multiple geo-distributed {es} clusters, or for +preventing search load from interfering with indexing by offloading search to a +secondary cluster + +Watch the +https://www.elastic.co/webinars/replicate-elasticsearch-data-with-cross-cluster-replication-ccr[{ccr} webinar] to learn more about the following use cases. +Then, <> on your local machine and work +through the demo from the webinar. + +[discrete] +[[ccr-disaster-recovery]] +==== Disaster recovery and high availability +Disaster recovery provides your mission-critical applications with the +tolerance to withstand datacenter or region outages. This use case is the +most common deployment of {ccr}. You can configure clusters in different +architectures to support disaster recovery and high availability: + +* <> +* <> +* <> +* <> + +[discrete] +[[ccr-single-datacenter-recovery]] +===== Single disaster recovery datacenter +In this configuration, data is replicated from the production datacenter to the +disaster recovery datacenter. Because the follower indices replicate the leader +index, your application can use the disaster recovery datacenter if the +production datacenter is unavailable. + +image::images/ccr-arch-disaster-recovery.png[Production datacenter that replicates data to a disaster recovery datacenter] + +[discrete] +[[ccr-multiple-datacenter-recovery]] +===== Multiple disaster recovery datacenters +You can replicate data from one datacenter to multiple datacenters. This +configuration provides both disaster recovery and high availability, ensuring +that data is replicated in two datacenters if the primary datacenter is down +or unavailable. + +In the following diagram, data from Datacenter A is replicated to +Datacenter B and Datacenter C, which both have a read-only copy of the leader +index from Datacenter A. + +image::images/ccr-arch-multiple-dcs.png[Production datacenter that replicates data to two other datacenters] + +[discrete] +[[ccr-chained-replication]] +===== Chained replication +You can replicate data across multiple datacenters to form a replication +chain. In the following diagram, Datacenter A contains the leader index. +Datacenter B replicates data from Datacenter A, and Datacenter C replicates +from the follower indices in Datacenter B. The connection between these +datacenters forms a chained replication pattern. + +image::images/ccr-arch-chain-dcs.png[Three datacenters connected to form a replication chain] + +[discrete] +[[ccr-bi-directional-replication]] +===== Bi-directional replication +In a https://www.elastic.co/blog/bi-directional-replication-with-elasticsearch-cross-cluster-replication-ccr[bi-directional replication] setup, all clusters have access to view +all data, and all clusters have an index to write to without manually +implementing failover. Applications can write to the local index within each +datacenter, and read across multiple indices for a global view of all +information. + +This configuration requires no manual intervention when a cluster or datacenter +is unavailable. In the following diagram, if Datacenter A is unavailable, you can continue using Datacenter B without manual failover. When Datacenter A +comes online, replication resumes between the clusters. + +image::images/ccr-arch-bi-directional.png[Bi-directional configuration where each cluster contains both a leader index and follower indices] + +NOTE: This configuration is useful for index-only workloads, where no updates +to document values occur. In this configuration, documents indexed by {es} are +immutable. Clients are located in each datacenter alongside the {es} +cluster, and do not communicate with clusters in different datacenters. + +[discrete] +[[ccr-data-locality]] +==== Data locality +Bringing data closer to your users or application server can reduce latency +and response time. This methodology also applies when replicating data in {es}. +For example, you can replicate a product catalog or reference dataset to 20 or +more datacenters around the world to minimize the distance between the data and +the application server. + +In the following diagram, data is replicated from one datacenter to three +additional datacenters, each in their own region. The central datacenter +contains the leader index, and the additional datacenters contain follower +indices that replicate data in that particular region. This configuration +puts data closer to the application accessing it. + +image::images/ccr-arch-data-locality.png[A centralized datacenter replicated across three other datacenters, each in their own region] + +[discrete] +[[ccr-centralized-reporting]] +==== Centralized reporting +Using a centralized reporting cluster is useful when querying across a large +network is inefficient. In this configuration, you replicate data from many +smaller clusters to the centralized reporting cluster. + +For example, a large global bank might have 100 {es} clusters around the world +that are distributed across different regions for each bank branch. Using +{ccr}, the bank can replicate events from all 100 banks to a central cluster to +analyze and aggregate events locally for reporting. Rather than maintaining a +mirrored cluster, the bank can use {ccr} to replicate specific indices. + +In the following diagram, data from three datacenters in different regions is +replicated to a centralized reporting cluster. This configuration enables you +to copy data from regional hubs to a central cluster, where you can run all +reports locally. + +image::images/ccr-arch-central-reporting.png[Three clusters in different regions sending data to a centralized reporting cluster for analysis] + +[discrete] +[[ccr-replication-mechanics]] +=== Replication mechanics +Although you <> at the index level, {es} +achieves replication at the shard level. When a follower index is created, +each shard in that index pulls changes from its corresponding shard in the +leader index, which means that a follower index has the same number of +shards as its leader index. All operations on the leader are replicated by the +follower, such as operations to create, update, or delete a document. +These requests can be served from any copy of the leader shard (primary or +replica). + +When a follower shard sends a read request, the leader shard responds with +any new operations, limited by the read parameters that you establish when +configuring the follower index. If no new operations are available, the +leader shard waits up to the configured timeout for new operations. If the +timeout elapses, the leader shard responds to the follower shard that there +are no new operations. The follower shard updates shard statistics and +immediately sends another read request to the leader shard. This +communication model ensures that network connections between the remote +cluster and the local cluster are continually in use, avoiding forceful +termination by an external source such as a firewall. + +If a read request fails, the cause of the failure is inspected. If the +cause of the failure is deemed to be recoverable (such as a network +failure), the follower shard enters into a retry loop. Otherwise, the +follower shard pauses +<>. + +When a follower shard receives operations from the leader shard, it places +those operations in a write buffer. The follower shard submits bulk write +requests using operations from the write buffer. If the write buffer exceeds +its configured limits, no additional read requests are sent. This configuration +provides a back-pressure against read requests, allowing the follower shard +to resume sending read requests when the write buffer is no longer full. + +To manage how operations are replicated from the leader index, you can +configure settings when +<>. + +The follower index automatically retrieves some updates applied to the leader +index, while other updates are retrieved as needed: + +[cols="3"] +|=== +h| Update type h| Automatic h| As needed +| Alias | {yes-icon} | {no-icon} +| Mapping | {no-icon} | {yes-icon} +| Settings | {no-icon} | {yes-icon} +|=== + +For example, changing the number of replicas on the leader index is not +replicated by the follower index, so that setting might not be retrieved. + +NOTE: You cannot manually modify a follower index's mappings or aliases. + +If you apply a non-dynamic settings change to the leader index that is +needed by the follower index, the follower index closes itself, applies the +settings update, and then re-opens itself. The follower index is unavailable +for reads and cannot replicate writes during this cycle. + +[discrete] +[[ccr-remote-recovery]] +=== Initializing followers using remote recovery +When you create a follower index, you cannot use it until it is fully +initialized. The _remote recovery_ process builds a new copy of a shard on a +follower node by copying data from the primary shard in the leader cluster. + +{es} uses this remote recovery process to bootstrap a follower index using the +data from the leader index. This process provides the follower with a copy of +the current state of the leader index, even if a complete history of changes +is not available on the leader due to Lucene segment merging. + +Remote recovery is a network intensive process that transfers all of the Lucene +segment files from the leader cluster to the follower cluster. The follower +requests that a recovery session be initiated on the primary shard in the +leader cluster. The follower then requests file chunks concurrently from the +leader. By default, the process concurrently requests five 1MB file +chunks. This default behavior is designed to support leader and follower +clusters with high network latency between them. + +TIP: You can modify dynamic <> +to rate-limit the transmitted data and manage the resources consumed by remote +recoveries. + +Use the <> on the cluster containing the follower +index to obtain information about an in-progress remote recovery. Because {es} +implements remote recoveries using the +<> infrastructure, running remote +recoveries are labelled as type `snapshot` in the recovery API. + +[discrete] +[[ccr-leader-requirements]] +=== Replicating a leader requires soft deletes +{ccr-cap} works by replaying the history of individual write +operations that were performed on the shards of the leader index. {es} needs to +retain the +<> on the leader +shards so that they can be pulled by the follower shard tasks. The underlying +mechanism used to retain these operations is _soft deletes_. + +A soft delete occurs whenever an existing document is deleted or updated. By +retaining these soft deletes up to configurable limits, the history of +operations can be retained on the leader shards and made available to the +follower shard tasks as it replays the history of operations. + +The <> setting defines the +maximum time to retain a shard history retention lease before it is +considered expired. This setting determines how long the cluster containing +your leader index can be offline, which is 12 hours by default. If a shard copy +recovers after its retention lease expires, then {es} will fall back to copying +the entire index, because it can no longer replay the missing history. + +Soft deletes must be enabled for indices that you want to use as leader +indices. Soft deletes are enabled by default on new indices created on +or after {es} 7.0.0. + +// tag::ccr-existing-indices-tag[] +IMPORTANT: {ccr-cap} cannot be used on existing indices created using {es} +7.0.0 or earlier, where soft deletes are disabled. You must +<> your data into a new index with soft deletes +enabled. + +// end::ccr-existing-indices-tag[] + +[discrete] +[[ccr-learn-more]] +=== Use {ccr} +This following sections provide more information about how to configure +and use {ccr}: -* <> -* <> -* <> * <> +* <> +* <> * <> - -include::overview.asciidoc[] -include::requirements.asciidoc[] -include::auto-follow.asciidoc[] include::getting-started.asciidoc[] -include::remote-recovery.asciidoc[] +include::managing.asciidoc[] +include::auto-follow.asciidoc[] include::upgrading.asciidoc[] diff --git a/docs/reference/ccr/managing.asciidoc b/docs/reference/ccr/managing.asciidoc new file mode 100644 index 0000000000000..bb07375a17224 --- /dev/null +++ b/docs/reference/ccr/managing.asciidoc @@ -0,0 +1,164 @@ +[role="xpack"] +[testenv="platinum"] + +////////////////////////// + +[source,console] +-------------------------------------------------- +PUT /follower_index/_ccr/follow?wait_for_active_shards=1 +{ + "remote_cluster" : "remote_cluster", + "leader_index" : "leader_index" +} +-------------------------------------------------- +// TESTSETUP +// TEST[setup:remote_cluster_and_leader_index] + +[source,console] +-------------------------------------------------- +POST /follower_index/_ccr/pause_follow +-------------------------------------------------- +// TEARDOWN + +////////////////////////// + +[[ccr-managing]] +=== Manage {ccr} +Use the following information to manage {ccr} tasks, such as inspecting +replication progress, pausing and resuming replication, recreating a follower +index, and terminating replication. + +[[ccr-access-ccr]] +To start using {ccr}, access {kib} and go to +*Management > Stack Management*. In the side navigation, select +*Cross-Cluster Replication*. + +[[ccr-inspect-progress]] +==== Inspect replication statistics +To inspect the progress of replication for a follower index and view +detailed shard statistics, <> and choose the *Follower indices* tab. + +Select the name of the follower index you want to view replication details +for. The slide-out panel shows settings and replication statistics for the +follower index, including read and write operations that are managed by the +follower shard. + +To view more detailed statistics, click *View in Index Management*, and +then select the name of the follower index in Index Management. +Open the tabs for detailed statistics about the follower index. + +[%collapsible] +.API example +==== +Use the <> to inspect replication +progress at the shard level. This API provides insight into the read and writes +managed by the follower shard. The API also reports read exceptions that can be +retried and fatal exceptions that require user intervention. +==== + +[[ccr-pause-replication]] +==== Pause and resume replication +To pause and resume replication of the leader index, <> and choose the *Follower indices* tab. + +Select the follower index you want to pause and choose *Manage > Pause Replication*. The follower index status changes to Paused. + +To resume replication, select the follower index and choose +*Resume replication*. + +[%collapsible] +.API example +==== +You can pause replication with the +<> and then later resume +replication with the <>. +Using these APIs in tandem enables you to adjust the read and write parameters +on the follower shard task if your initial configuration is not suitable for +your use case. +==== + +[[ccr-recreate-follower-index]] +==== Recreate a follower index +When a document is updated or deleted, the underlying operation is retained in +the Lucene index for a period of time defined by the +<> parameter. You configure +this setting on the <>. + +When a follower index starts, it acquires a retention lease from +the leader index. This lease informs the leader that it should not allow a soft +delete to be pruned until either the follower indicates that it has received +the operation, or until the lease expires. + +If a follower index falls sufficiently behind a leader and cannot +replicate operations, {es} reports an `indices[].fatal_exception` error. To +resolve the issue, recreate the follower index. When the new follow index +starts, the <> process recopies the +Lucene segment files from the leader. + +IMPORTANT: Recreating the follower index is a destructive action. All existing +Lucene segment files are deleted on the cluster containing the follower index. + +To recreate a follower index, +<> and choose the +*Follower indices* tab. + +[role="screenshot"] +image::images/ccr-follower-index.png["The Cross-Cluster Replication page in {kib}"] + +Select the follower index and pause replication. When the follower index status +changes to Paused, reselect the follower index and choose to unfollow the +leader index. + +The follower index will be converted to a standard index and will no longer +display on the Cross-Cluster Replication page. + +In the side navigation, choose *Index Management*. Select the follower index +from the previous steps and close the follower index. + +You can then <> +to restart the replication process. + +[%collapsible] +.Use the API +==== +Use the <> to pause the replication +process. Then, close the follower index and recreate it. For example: + +[source,console] +---------------------------------------------------------------------- +POST /follower_index/_ccr/pause_follow + +POST /follower_index/_close + +PUT /follower_index/_ccr/follow?wait_for_active_shards=1 +{ + "remote_cluster" : "remote_cluster", + "leader_index" : "leader_index" +} +---------------------------------------------------------------------- +==== + +[[ccr-terminate-replication]] +==== Terminate replication +You can unfollow a leader index to terminate replication and convert the +follower index to a standard index. + +<> and choose the +*Follower indices* tab. + +Select the follower index and pause replication. When the follower index status +changes to Paused, reselect the follower index and choose to unfollow the +leader index. + +The follower index will be converted to a standard index and will no longer +display on the Cross-Cluster Replication page. + +You can then choose *Index Management*, select the follower index +from the previous steps, and close the follower index. + +[%collapsible] +.Use the API +==== +You can terminate replication with the +<>. This API converts a follower index +to a standard (non-follower) index. +==== diff --git a/docs/reference/ccr/overview.asciidoc b/docs/reference/ccr/overview.asciidoc deleted file mode 100644 index 7a68a0b2166c6..0000000000000 --- a/docs/reference/ccr/overview.asciidoc +++ /dev/null @@ -1,228 +0,0 @@ -[role="xpack"] -[testenv="platinum"] -[[ccr-overview]] -=== Overview - - -{ccr-cap} is done on an index-by-index basis. Replication is -configured at the index level. For each configured replication there is a -replication source index called the _leader index_ and a replication target -index called the _follower index_. - -Replication is active-passive. This means that while the leader index -can directly be written into, the follower index can not directly receive -writes. - -Replication is pull-based. This means that replication is driven by the -follower index. This simplifies state management on the leader index and means -that {ccr} does not interfere with indexing on the leader index. - -In {ccr}, the cluster performing this pull is known as the _local cluster_. The -cluster being replicated is known as the _remote cluster_. - -==== Prerequisites - -* {ccr-cap} requires <>. - -* The {es} version of the local cluster must be **the same as or newer** than -the remote cluster. If newer, the versions must also be compatible as outlined -in the following matrix. - -include::../modules/remote-clusters.asciidoc[tag=remote-cluster-compatibility-matrix] - -==== Configuring replication - -Replication can be configured in two ways: - -* Manually creating specific follower indices (in {kib} or by using the -{ref}/ccr-put-follow.html[create follower API]) - -* Automatically creating follower indices from auto-follow patterns (in {kib} or -by using the {ref}/ccr-put-auto-follow-pattern.html[create auto-follow pattern API]) - -For more information about managing {ccr} in {kib}, see -{kibana-ref}/working-remote-clusters.html[Working with remote clusters]. - -NOTE: You must also <>. - -When you initiate replication either manually or through an auto-follow pattern, the -follower index is created on the local cluster. Once the follower index is created, -the <> process copies all of the Lucene segment -files from the remote cluster to the local cluster. - -By default, if you initiate following manually (by using {kib} or the create follower API), -the recovery process is asynchronous in relationship to the -{ref}/ccr-put-follow.html[create follower request]. The request returns before -the <> process completes. If you would like to wait on -the process to complete, you can use the `wait_for_active_shards` parameter. - -////////////////////////// - -[source,console] --------------------------------------------------- -PUT /follower_index/_ccr/follow?wait_for_active_shards=1 -{ - "remote_cluster" : "remote_cluster", - "leader_index" : "leader_index" -} --------------------------------------------------- -// TESTSETUP -// TEST[setup:remote_cluster_and_leader_index] - -[source,console] --------------------------------------------------- -POST /follower_index/_ccr/pause_follow --------------------------------------------------- -// TEARDOWN - -////////////////////////// - - -==== The mechanics of replication - -While replication is managed at the index level, replication is performed at the -shard level. When a follower index is created, it is automatically -configured to have an identical number of shards as the leader index. A follower -shard task in the follower index pulls from the corresponding leader shard in -the leader index by sending read requests for new operations. These read -requests can be served from any copy of the leader shard (primary or replicas). - -For each read request sent by the follower shard task, if there are new -operations available on the leader shard, the leader shard responds with -operations limited by the read parameters that you established when you -configured the follower index. If there are no new operations available on the -leader shard, the leader shard waits up to a configured timeout for new -operations. If new operations occur within that timeout, the leader shard -immediately responds with those new operations. Otherwise, if the timeout -elapses, the leader shard replies that there are no new operations. The -follower shard task updates some statistics and immediately sends another read -request to the leader shard. This ensures that the network connections between -the remote cluster and the local cluster are continually being used so as to -avoid forceful termination by an external source (such as a firewall). - -If a read request fails, the cause of the failure is inspected. If the -cause of the failure is deemed to be a failure that can be recovered from (for -example, a network failure), the follower shard task enters into a retry -loop. Otherwise, the follower shard task is paused and requires user -intervention before it can be resumed with the -{ref}/ccr-post-resume-follow.html[resume follower API]. - -When operations are received by the follower shard task, they are placed in a -write buffer. The follower shard task manages this write buffer and submits -bulk write requests from this write buffer to the follower shard. The write -buffer and these write requests are managed by the write parameters that you -established when you configured the follower index. The write buffer serves as -back-pressure against read requests. If the write buffer exceeds its configured -limits, no additional read requests are sent by the follower shard task. The -follower shard task resumes sending read requests when the write buffer no -longer exceeds its configured limits. - -NOTE: The intricacies of how operations are replicated from the leader are -governed by settings that you can configure when you create the follower index -in {kib} or by using the {ref}/ccr-put-follow.html[create follower API]. - -Mapping updates applied to the leader index are automatically retrieved -as-needed by the follower index. It is not possible to manually modify the -mapping of a follower index. - -Settings updates applied to the leader index that are needed by the follower -index are automatically retrieved as-needed by the follower index. Not all -settings updates are needed by the follower index. For example, changing the -number of replicas on the leader index is not replicated by the follower index. - -Alias updates applied to the leader index are automatically retrieved by the -follower index. It is not possible to manually modify an alias of a follower -index. - -NOTE: If you apply a non-dynamic settings change to the leader index that is -needed by the follower index, the follower index will go through a cycle of -closing itself, applying the settings update, and then re-opening itself. The -follower index will be unavailable for reads and not replicating writes -during this cycle. - - -==== Inspecting the progress of replication - -You can inspect the progress of replication at the shard level with the -{ref}/ccr-get-follow-stats.html[get follower stats API]. This API gives you -insight into the read and writes managed by the follower shard task. It also -reports read exceptions that can be retried and fatal exceptions that require -user intervention. - - -==== Pausing and resuming replication - -You can pause replication with the -{ref}/ccr-post-pause-follow.html[pause follower API] and then later resume -replication with the {ref}/ccr-post-resume-follow.html[resume follower API]. -Using these APIs in tandem enables you to adjust the read and write parameters -on the follower shard task if your initial configuration is not suitable for -your use case. - - -==== Leader index retaining operations for replication - -If the follower is unable to replicate operations from a leader for a period of -time, the following process can fail due to the leader lacking a complete history -of operations necessary for replication. - -Operations replicated to the follower are identified using a sequence number -generated when the operation was initially performed. Lucene segment files are -occasionally merged in order to optimize searches and save space. When these -merges occur, it is possible for operations associated with deleted or updated -documents to be pruned during the merge. When the follower requests the sequence -number for a pruned operation, the process will fail due to the operation missing -on the leader. - -This scenario is not possible in an append-only workflow. As documents are never -deleted or updated, the underlying operation will not be pruned. - -Elasticsearch attempts to mitigate this potential issue for update workflows using -a Lucene feature called soft deletes. When a document is updated or deleted, the -underlying operation is retained in the Lucene index for a period of time. This -period of time is governed by the `index.soft_deletes.retention_lease.period` -setting which can be <>. - -When a follower initiates the index following, it acquires a retention lease from -the leader. This informs the leader that it should not allow a soft delete to be -pruned until either the follower indicates that it has received the operation or -the lease expires. It is valuable to have monitoring in place to detect a follower -replication issue prior to the lease expiring so that the problem can be remedied -before the follower falls fatally behind. - - -==== Remedying a follower that has fallen behind - -If a follower falls sufficiently behind a leader that it can no longer replicate -operations this can be detected in {kib} or by using the -{ref}/ccr-get-follow-stats.html[get follow stats API]. It will be reported as a -`indices[].fatal_exception`. - -In order to restart the follower, you must pause the following process, close the -index, and the create follower index again. For example: - -[source,console] ----------------------------------------------------------------------- -POST /follower_index/_ccr/pause_follow - -POST /follower_index/_close - -PUT /follower_index/_ccr/follow?wait_for_active_shards=1 -{ - "remote_cluster" : "remote_cluster", - "leader_index" : "leader_index" -} ----------------------------------------------------------------------- - -Re-creating the follower index is a destructive action. All of the existing Lucene -segment files are deleted on the follower cluster. The -<> process copies the Lucene segment -files from the leader again. After the follower index initializes, the -following process starts again. - - -==== Terminating replication - -You can terminate replication with the -{ref}/ccr-post-unfollow.html[unfollow API]. This API converts a follower index -to a regular (non-follower) index. diff --git a/docs/reference/ccr/remote-recovery.asciidoc b/docs/reference/ccr/remote-recovery.asciidoc deleted file mode 100644 index f96aaad3d0960..0000000000000 --- a/docs/reference/ccr/remote-recovery.asciidoc +++ /dev/null @@ -1,29 +0,0 @@ -[role="xpack"] -[testenv="platinum"] -[[remote-recovery]] -=== Remote recovery - -When you create a follower index, you cannot use it until it is fully initialized. -The _remote recovery_ process builds a new copy of a shard on a follower node by -copying data from the primary shard in the leader cluster. {es} uses this remote -recovery process to bootstrap a follower index using the data from the leader index. -This process provides the follower with a copy of the current state of the leader index, -even if a complete history of changes is not available on the leader due to Lucene -segment merging. - -Remote recovery is a network intensive process that transfers all of the Lucene -segment files from the leader cluster to the follower cluster. The follower -requests that a recovery session be initiated on the primary shard in the leader -cluster. The follower then requests file chunks concurrently from the leader. By -default, the process concurrently requests `5` large `1mb` file chunks. This default -behavior is designed to support leader and follower clusters with high network latency -between them. - -There are dynamic settings that you can use to rate-limit the transmitted data -and manage the resources consumed by remote recoveries. See -{ref}/ccr-settings.html[{ccr-cap} settings]. - -You can obtain information about an in-progress remote recovery by using the -{ref}/cat-recovery.html[recovery API] on the follower cluster. Remote recoveries -are implemented using the {ref}/modules-snapshots.html[snapshot and restore] infrastructure. This means that on-going remote recoveries are labelled as type -`snapshot` in the recovery API. diff --git a/docs/reference/ccr/requirements.asciidoc b/docs/reference/ccr/requirements.asciidoc deleted file mode 100644 index 5e3b188849aa6..0000000000000 --- a/docs/reference/ccr/requirements.asciidoc +++ /dev/null @@ -1,43 +0,0 @@ -[role="xpack"] -[testenv="platinum"] -[[ccr-requirements]] -=== Requirements for leader indices - -{ccr-cap} works by replaying the history of individual write -operations that were performed on the shards of the leader index. This means that the -history of these operations needs to be retained on the leader shards so that -they can be pulled by the follower shard tasks. The underlying mechanism used to -retain these operations is _soft deletes_. A soft delete occurs whenever an -existing document is deleted or updated. By retaining these soft deletes up to -configurable limits, the history of operations can be retained on the leader -shards and made available to the follower shard tasks as it replays the history -of operations. - -Soft deletes must be enabled for indices that you want to use as leader -indices. Soft deletes are enabled by default on new indices created on -or after {es} 7.0.0. - -IMPORTANT: This means that {ccr} can not be used on existing indices. If you have -existing data that you want to replicate from another cluster, you must -{ref}/docs-reindex.html[reindex] your data into a new index with soft deletes -enabled. - - -[[ccr-overview-soft-deletes]] -==== Soft delete settings - -`index.soft_deletes.enabled`:: - -Whether or not soft deletes are enabled on the index. Soft deletes can only be -configured at index creation and only on indices created on or after 6.5.0. The -default value is `true`. - -`index.soft_deletes.retention_lease.period`:: - -The maximum period to retain a shard history retention lease before it is considered -expired. Shard history retention leases ensure that soft deletes are retained during -merges on the Lucene index. If a soft delete is merged away before it can be replicated -to a follower the following process will fail due to incomplete history on the leader. -The default value is `12h`. - -For more information about index settings, see {ref}/index-modules.html[Index modules]. diff --git a/docs/reference/ccr/upgrading.asciidoc b/docs/reference/ccr/upgrading.asciidoc index 716822d4d9e41..e7cf6249d5eca 100644 --- a/docs/reference/ccr/upgrading.asciidoc +++ b/docs/reference/ccr/upgrading.asciidoc @@ -1,49 +1,67 @@ [role="xpack"] [testenv="platinum"] [[ccr-upgrading]] -=== Upgrading clusters +=== Upgrading clusters using {ccr} +++++ +Upgrading clusters +++++ Clusters that are actively using {ccr} require a careful approach to upgrades. -Otherwise index following may fail during a rolling upgrade, because of the -following reasons: +The following conditions could cause index following to fail during rolling +upgrades: -* If a new index setting or mapping type is replicated from an upgraded cluster - to a non-upgraded cluster then the non-upgraded cluster will reject that and - will fail index following. -* Lucene is not forwards compatible and when index following is falling back to - file based recovery then a node in a non-upgraded cluster will reject index - files from a newer Lucene version compared to what it is using. - -Rolling upgrading clusters with {ccr} is different in case of uni-directional -index following and bi-directional index following. +* Clusters that have not yet been upgraded will reject new index settings or +mapping types that are replicated from an upgraded cluster. +* Nodes in a cluster that has not been upgraded will reject index files from a +node in an upgraded cluster when index following tries to fall back to +file-based recovery. This limitation is due to Lucene not being forward +compatible. +The approach to running a rolling upgrade on clusters where {ccr} is +enabled differs based on uni-directional and bi-directional index following. +[[ccr-uni-directional-upgrade]] ==== Uni-directional index following +In a uni-directional configuration, one cluster contains only +leader indices, and the other cluster contains only follower indices that +replicate the leader indices. -In a uni-directional setup between two clusters, one cluster contains only -leader indices, and the other cluster contains only follower indices following -indices in the first cluster. - -In this setup, the cluster with follower indices should be upgraded +In this strategy, the cluster with follower indices should be upgraded first and the cluster with leader indices should be upgraded last. -If clusters are upgraded in this order then index following can continue +Upgrading the clusters in this order ensures that index following can continue during the upgrade without downtime. -Note that a chain index following setup can also be upgraded in this way. -For example if there is a cluster A that contains all leader indices, -cluster B that follows indices in cluster A and cluster C that follows -indices in cluster B. In this case the cluster C should be upgraded first, -then cluster B and finally cluster A. +You can also use this strategy to upgrade a +<>. Start by upgrading clusters at +the end of the chain and working your way back to the cluster that contains the +leader indices. + +For example, consider a configuration where Cluster A contains all leader +indices. Cluster B follows indices in Cluster A, and Cluster C follows indices +in Cluster B. +-- + Cluster A + ^--Cluster B + ^--Cluster C +-- +In this configuration, upgrade the clusters in the following order: + +. Cluster C +. Cluster B +. Cluster A + +[[ccr-bi-directional-upgrade]] ==== Bi-directional index following -In a bi-directional setup between two clusters, each cluster contains both -leader and follower indices. +In a bi-directional configuration, each cluster contains both leader and +follower indices. + +When upgrading clusters in this configuration, +<> and +<> prior to +upgrading both clusters. -When upgrading clusters in this setup, all index following needs to be paused -using the {ref}/ccr-post-pause-follow.html[pause follower API] and the - {ref}/ccr-pause-auto-follow-pattern.html[pause auto-follower API] prior to -upgrading both clusters. After both clusters have been upgraded then index -following can be resumed using the -{ref}/ccr-post-resume-follow.html[resume follower API]]. +After upgrading both clusters, resume index following and resume replication +of auto-follow patterns. diff --git a/docs/reference/glossary.asciidoc b/docs/reference/glossary.asciidoc index 96293f49cde5e..38d45032b956e 100644 --- a/docs/reference/glossary.asciidoc +++ b/docs/reference/glossary.asciidoc @@ -26,17 +26,17 @@ Also see <> and <>. [[glossary-api-key]] API key :: // tag::api-key-def[] -A unique identifier that you can use for authentication when submitting {es} requests. -When TLS is enabled, all requests must be authenticated using either basic authentication +A unique identifier that you can use for authentication when submitting {es} requests. +When TLS is enabled, all requests must be authenticated using either basic authentication (user name and password) or an API key. // end::api-key-def[] [[glossary-auto-follow-pattern]] auto-follow pattern :: // tag::auto-follow-pattern-def[] -An <> that automatically configures new indices as +An <> that automatically configures new indices as <> for <>. -For more information, see {ref}/ccr-auto-follow.html#_managing_auto_follow_patterns[Managing auto follow patterns]. +For more information, see {ref}/ccr-auto-follow.html[Managing auto follow patterns]. // end::auto-follow-pattern-def[] [[glossary-cluster]] cluster :: @@ -48,24 +48,24 @@ chosen automatically by the cluster and can be replaced if it fails. [[glossary-cold-phase]] cold phase :: // tag::cold-phase-def[] -The third possible phase in the <>. -In the cold phase, an index is no longer updated and seldom queried. +The third possible phase in the <>. +In the cold phase, an index is no longer updated and seldom queried. The information still needs to be searchable, but it’s okay if those queries are slower. // end::cold-phase-def[] [[glossary-component-template]] component template :: // tag::component-template-def[] A building block for constructing <> that specifies index -<>, <>, and <>. +<>, <>, and <>. // end::component-template-def[] [[glossary-ccr]] {ccr} (CCR):: // tag::ccr-def[] A feature that enables you to replicate indices in remote clusters to your -local cluster. For more information, see -{ref}/xpack-ccr.html[{ccr-cap}]. +local cluster. For more information, see +{ref}/xpack-ccr.html[{ccr-cap}]. // end::ccr-def[] - + [[glossary-ccs]] {ccs} (CCS):: // tag::ccs-def[] A feature that enables any node to act as a federated client across @@ -88,7 +88,7 @@ See {ref}/data-streams.html[Data streams]. [[glossary-delete-phase]] delete phase :: // tag::delete-phase-def[] -The last possible phase in the <>. +The last possible phase in the <>. In the delete phase, an index is no longer needed and can safely be deleted. // end::delete-phase-def[] @@ -132,56 +132,56 @@ how the value for a field should be analyzed. // tag::filter-def[] A filter is a non-scoring <>, meaning that it does not score documents. -It is only concerned about answering the question - "Does this document match?". -The answer is always a simple, binary yes or no. This kind of query is said to be made -in a {ref}/query-filter-context.html[filter context], -hence it is called a filter. Filters are simple checks for set inclusion or exclusion. +It is only concerned about answering the question - "Does this document match?". +The answer is always a simple, binary yes or no. This kind of query is said to be made +in a {ref}/query-filter-context.html[filter context], +hence it is called a filter. Filters are simple checks for set inclusion or exclusion. In most cases, the goal of filtering is to reduce the number of documents that have to be examined. // end::filter-def[] [[glossary-flush]] flush :: // tag::flush-def[] -Peform a Lucene commit to write index updates in the transaction log (translog) to disk. -Because a Lucene commit is a relatively expensive operation, -{es} records index and delete operations in the translog and +Peform a Lucene commit to write index updates in the transaction log (translog) to disk. +Because a Lucene commit is a relatively expensive operation, +{es} records index and delete operations in the translog and automatically flushes changes to disk in batches. -To recover from a crash, operations that have been acknowledged but not yet committed +To recover from a crash, operations that have been acknowledged but not yet committed can be replayed from the translog. Before upgrading, you can explicitly call the {ref}/indices-flush.html[Flush] API to ensure that all changes are committed to disk. // end::flush-def[] -[[glossary-follower-index]] follower index :: +[[glossary-follower-index]] follower index :: // tag::follower-index-def[] The target index for <>. A follower index exists in a local cluster and replicates a <>. // end::follower-index-def[] -[[glossary-force-merge]] force merge :: +[[glossary-force-merge]] force merge :: // tag::force-merge-def[] // tag::force-merge-def-short[] -Manually trigger a merge to reduce the number of segments in each shard of an index +Manually trigger a merge to reduce the number of segments in each shard of an index and free up the space used by deleted documents. // end::force-merge-def-short[] You should not force merge indices that are actively being written to. -Merging is normally performed automatically, but you can use force merge after -<> to reduce the shards in the old index to a single segment. +Merging is normally performed automatically, but you can use force merge after +<> to reduce the shards in the old index to a single segment. See the {ref}/indices-forcemerge.html[force merge API]. // end::force-merge-def[] -[[glossary-freeze]] freeze :: +[[glossary-freeze]] freeze :: // tag::freeze-def[] // tag::freeze-def-short[] -Make an index read-only and minimize its memory footprint. +Make an index read-only and minimize its memory footprint. // end::freeze-def-short[] Frozen indices can be searched without incurring the overhead of of re-opening a closed index, -but searches are throttled and might be slower. -You can freeze indices to reduce the overhead of keeping older indices searchable +but searches are throttled and might be slower. +You can freeze indices to reduce the overhead of keeping older indices searchable before you are ready to archive or delete them. See the {ref}/freeze-index-api.html[freeze API]. // end::freeze-def[] -[[glossary-frozen-index]] frozen index :: +[[glossary-frozen-index]] frozen index :: // tag::frozen-index-def[] An index reduced to a low overhead state that still enables occasional searches. Frozen indices use a memory-efficient shard implementation and throttle searches to conserve resources. @@ -190,8 +190,8 @@ Searching a frozen index is lower overhead than re-opening a closed index to ena [[glossary-hot-phase]] hot phase :: // tag::hot-phase-def[] -The first possible phase in the <>. -In the hot phase, an index is actively updated and queried. +The first possible phase in the <>. +In the hot phase, an index is actively updated and queried. // end::hot-phase-def[] [[glossary-id]] id :: @@ -206,8 +206,8 @@ then it will be auto-generated. (also see <>) -- // tag::index-def[] // tag::index-def-short[] -An optimized collection of JSON documents. Each document is a collection of fields, -the key-value pairs that contain your data. +An optimized collection of JSON documents. Each document is a collection of fields, +the key-value pairs that contain your data. // end::index-def-short[] An index is a logical namespace that maps to one or more @@ -232,7 +232,7 @@ See {ref}/indices-add-alias.html[Add index alias]. [[glossary-index-lifecycle]] index lifecycle :: // tag::index-lifecycle-def[] -The four phases an index can transition through: +The four phases an index can transition through: <>, <>, <>, and <>. For more information, see {ref}/ilm-policy-definition.html[Index lifecycle]. @@ -240,17 +240,17 @@ For more information, see {ref}/ilm-policy-definition.html[Index lifecycle]. [[glossary-index-lifecycle-policy]] index lifecycle policy :: // tag::index-lifecycle-policy-def[] -Specifies how an index moves between phases in the index lifecycle and -what actions to perform during each phase. +Specifies how an index moves between phases in the index lifecycle and +what actions to perform during each phase. // end::index-lifecycle-policy-def[] [[glossary-index-pattern]] index pattern :: // tag::index-pattern-def[] A string that can contain the `*` wildcard to match multiple index names. -In most cases, the index parameter in an {es} request can be the name of a specific index, +In most cases, the index parameter in an {es} request can be the name of a specific index, a list of index names, or an index pattern. For example, if you have the indices `datastream-000001`, `datastream-000002`, and `datastream-000003`, -to search across all three you could use the `datastream-*` index pattern. +to search across all three you could use the `datastream-*` index pattern. // end::index-pattern-def[] [[glossary-index-template]] index template :: @@ -258,18 +258,18 @@ to search across all three you could use the `datastream-*` index pattern. -- // tag::index-template-def[] // tag::index-template-def-short[] -Defines settings and mappings to apply to new indexes that match a simple naming pattern, such as _logs-*_. +Defines settings and mappings to apply to new indexes that match a simple naming pattern, such as _logs-*_. // end::index-template-def-short[] An index template can also attach a lifecycle policy to the new index. -Index templates are used to automatically configure indices created during <>. +Index templates are used to automatically configure indices created during <>. // end::index-template-def[] -- -[[glossary-leader-index]] leader index :: +[[glossary-leader-index]] leader index :: // tag::leader-index-def[] The source index for <>. A leader index exists -on a remote cluster and is replicated to +on a remote cluster and is replicated to <>. // end::leader-index-def[] @@ -376,8 +376,8 @@ or upgrade {es} between incompatible versions. [[glossary-remote-cluster]] remote cluster :: // tag::remote-cluster-def[] -A separate cluster, often in a different data center or locale, that contains indices that -can be replicated or searched by the <>. +A separate cluster, often in a different data center or locale, that contains indices that +can be replicated or searched by the <>. The connection to a remote cluster is unidirectional. // end::remote-cluster-def[] @@ -416,21 +416,21 @@ See the {ref}/indices-rollover-index.html[rollover index API]. [[glossary-rollup]] rollup :: // tag::rollup-def[] -Summarize high-granularity data into a more compressed format to +Summarize high-granularity data into a more compressed format to maintain access to historical data in a cost-effective way. // end::rollup-def[] [[glossary-rollup-index]] rollup index :: // tag::rollup-index-def[] -A special type of index for storing historical data at reduced granularity. +A special type of index for storing historical data at reduced granularity. Documents are summarized and indexed into a rollup index by a <>. // end::rollup-index-def[] [[glossary-rollup-job]] rollup job :: // tag::rollup-job-def[] -A background task that runs continuously to summarize documents in an index and -index the summaries into a separate rollup index. -The job configuration controls what information is rolled up and how often. +A background task that runs continuously to summarize documents in an index and +index the summaries into a separate rollup index. +The job configuration controls what information is rolled up and how often. // end::rollup-job-def[] [[glossary-routing]] routing :: @@ -479,7 +479,7 @@ Reduce the number of primary shards in an index. // end::shrink-def-short[] You can shrink an index to reduce its overhead when the request volume drops. -For example, you might opt to shrink an index once it is no longer the write index. +For example, you might opt to shrink an index once it is no longer the write index. See the {ref}/indices-shrink-index.html[shrink index API]. // end::shrink-def[] -- @@ -493,13 +493,13 @@ indices you specify. [[glossary-snapshot-lifecycle-policy]] snapshot lifecycle policy :: // tag::snapshot-lifecycle-policy-def[] -Specifies how frequently to perform automatic backups of a cluster and -how long to retain the resulting snapshots. +Specifies how frequently to perform automatic backups of a cluster and +how long to retain the resulting snapshots. // end::snapshot-lifecycle-policy-def[] [[glossary-snapshot-repository]] snapshot repository :: // tag::snapshot-repository-def[] -Specifies where snapshots are to be stored. +Specifies where snapshots are to be stored. Snapshots can be written to a shared filesystem or to a remote repository. // end::snapshot-repository-def[] @@ -556,6 +556,6 @@ See {ref}/removal-of-types.html[Removal of mapping types]. [[glossary-warm-phase]] warm phase :: // tag::warm-phase-def[] -The second possible phase in the <>. +The second possible phase in the <>. In the warm phase, an index is generally optimized for search and no longer updated. // end::warm-phase-def[] diff --git a/docs/reference/index-modules.asciidoc b/docs/reference/index-modules.asciidoc index 26d9075f889a9..6af6118663b4c 100644 --- a/docs/reference/index-modules.asciidoc +++ b/docs/reference/index-modules.asciidoc @@ -95,6 +95,25 @@ indices. than the `index.number_of_shards` unless the `index.number_of_shards` value is also 1. See <> for more details about how this setting is used. +[[ccr-index-soft-deletes]] +// tag::ccr-index-soft-deletes-tag[] +`index.soft_deletes.enabled`:: +deprecated:[7.6.0, Creating indices with soft-deletes disabled is deprecated and will be removed in future Elasticsearch versions.] +Indicates whether soft deletes are enabled on the index. Soft deletes can only +be configured at index creation and only on indices created on or after +{es} 6.5.0. Defaults to `true`. +// end::ccr-index-soft-deletes-tag[] + +[[ccr-index-soft-deletes-retention-period]] +//tag::ccr-index-soft-deletes-retention-tag[] +`index.soft_deletes.retention_lease.period`:: +The maximum period to retain a shard history retention lease before it is +considered expired. Shard history retention leases ensure that soft deletes are +retained during merges on the Lucene index. If a soft delete is merged away +before it can be replicated to a follower the following process will fail due +to incomplete history on the leader. Defaults to `12h`. +//end::ccr-index-soft-deletes-retention-tag[] + [[load-fixed-bitset-filters-eagerly]] `index.load_fixed_bitset_filters_eagerly`:: Indicates whether <> are pre-loaded for diff --git a/docs/reference/index-modules/history-retention.asciidoc b/docs/reference/index-modules/history-retention.asciidoc index ecee0bd2dff98..0685c1a5c5d13 100644 --- a/docs/reference/index-modules/history-retention.asciidoc +++ b/docs/reference/index-modules/history-retention.asciidoc @@ -52,16 +52,6 @@ reasonable recovery scenarios. [discrete] === History retention settings -`index.soft_deletes.enabled`:: +include::{es-ref-dir}/index-modules.asciidoc[tag=ccr-index-soft-deletes-tag] - deprecated:[7.6.0, Creating indices with soft-deletes disabled is deprecated and will be removed in future Elasticsearch versions.] - Whether or not soft deletes are enabled on the index. Soft deletes can only be - configured at index creation and only on indices created on or after 6.5.0. - The default value is `true`. - - -`index.soft_deletes.retention_lease.period`:: - - The maximum length of time to retain a shard history retention lease before - it expires and the history that it retains can be discarded. The default - value is `12h`. +include::{es-ref-dir}/index-modules.asciidoc[tag=ccr-index-soft-deletes-retention-tag] diff --git a/docs/reference/modules/remote-clusters.asciidoc b/docs/reference/modules/remote-clusters.asciidoc index 7e44ffa6fd6c1..78a8cfd3a2a3e 100644 --- a/docs/reference/modules/remote-clusters.asciidoc +++ b/docs/reference/modules/remote-clusters.asciidoc @@ -46,14 +46,15 @@ communicate with 6.7. The matrix below summarizes compatibility as described abo // tag::remote-cluster-compatibility-matrix[] [cols="^,^,^,^,^,^,^,^"] |==== -| Compatibility | 5.0->5.5 | 5.6 | 6.0->6.6 | 6.7 | 6.8 | 7.0 | 7.1->7.x -| 5.0->5.5 | Yes | Yes | No | No | No | No | No -| 5.6 | Yes | Yes | Yes | Yes | Yes | No | No -| 6.0->6.6 | No | Yes | Yes | Yes | Yes | No | No -| 6.7 | No | Yes | Yes | Yes | Yes | Yes | No -| 6.8 | No | Yes | Yes | Yes | Yes | Yes | Yes -| 7.0 | No | No | No | Yes | Yes | Yes | Yes -| 7.1->7.x | No | No | No | No | Yes | Yes | Yes +| 7+^h| Local cluster +h| Remote cluster | 5.0->5.5 | 5.6 | 6.0->6.6 | 6.7 | 6.8 | 7.0 | 7.1->7.x +| 5.0->5.5 | {yes-icon} | {yes-icon} | {no-icon} | {no-icon} | {no-icon} | {no-icon} | {no-icon} +| 5.6 | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {no-icon} | {no-icon} +| 6.0->6.6 | {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {no-icon} | {no-icon} +| 6.7 | {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {no-icon} +| 6.8 | {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} +| 7.0 | {no-icon} | {no-icon} | {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} +| 7.1->7.x | {no-icon} | {no-icon} | {no-icon} | {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} |==== // end::remote-cluster-compatibility-matrix[] diff --git a/docs/reference/redirects.asciidoc b/docs/reference/redirects.asciidoc index c3d0fc6d6c599..4c4be1d124cab 100644 --- a/docs/reference/redirects.asciidoc +++ b/docs/reference/redirects.asciidoc @@ -3,6 +3,31 @@ The following pages have moved or been deleted. +[role="exclude",id="ccr-remedy-follower-index"] +=== Leader index retaining operations for replication + +See <>. + +[role="exclude",id="ccr-leader-not-replicating"] +=== Remedying a follower that has fallen behind + +See <>. + +[role="exclude",id="remote-reovery"] +=== Remote recovery process + +See <>. + +[role="exclude",id="ccr-requirements"] +=== Leader index requirements + +See <>. + +[role="exclude",id="ccr-overview"] +=== Cross-cluster replication overview + +See <>. + [role="exclude",id="indices-upgrade"] === Upgrade API @@ -639,66 +664,66 @@ See: [role="exclude",id="testing"] === Testing -This page was deleted. -Information about the Java testing framework was removed -({es-issue}55257[#55257]) from the {es} Reference -because it was out of date and erroneously implied that it should be used by application developers. -There is an issue ({es-issue}55258[#55258]) -for providing general testing guidance for applications that communicate with {es}. +This page was deleted. +Information about the Java testing framework was removed +({es-issue}55257[#55257]) from the {es} Reference +because it was out of date and erroneously implied that it should be used by application developers. +There is an issue ({es-issue}55258[#55258]) +for providing general testing guidance for applications that communicate with {es}. [role="exclude",id="testing-framework"] === Java testing framework This page was deleted. -Information about the Java testing framework was removed -({es-issue}55257[55257]) from the {es} Reference -because it was out of date and erroneously implied that it should be used by application developers. -There is an issue ({es-issue}[#55258]) -for providing general testing guidance for applications that communicate with {es}. +Information about the Java testing framework was removed +({es-issue}55257[55257]) from the {es} Reference +because it was out of date and erroneously implied that it should be used by application developers. +There is an issue ({es-issue}[#55258]) +for providing general testing guidance for applications that communicate with {es}. [role="exclude",id="why-randomized-testing"] === Why randomized testing? This page was deleted. -Information about the Java testing framework was removed -({es-issue}55257[55257]) from the {es} Reference -because it was out of date and erroneously implied that it should be used by application developers. -There is an issue ({es-issue}[#55258]) -for providing general testing guidance for applications that communicate with {es}. +Information about the Java testing framework was removed +({es-issue}55257[55257]) from the {es} Reference +because it was out of date and erroneously implied that it should be used by application developers. +There is an issue ({es-issue}[#55258]) +for providing general testing guidance for applications that communicate with {es}. [role="exclude",id="using-elasticsearch-test-classes"] === Using the {es} test classes This page was deleted. -Information about the Java testing framework was removed -({es-issue}55257[55257]) from the {es} Reference -because it was out of date and erroneously implied that it should be used by application developers. -There is an issue ({es-issue}55258[#55258]) -for providing general testing guidance for applications that communicate with {es}. +Information about the Java testing framework was removed +({es-issue}55257[55257]) from the {es} Reference +because it was out of date and erroneously implied that it should be used by application developers. +There is an issue ({es-issue}55258[#55258]) +for providing general testing guidance for applications that communicate with {es}. [role="exclude",id="unit-tests"] === Unit tests This page was deleted. -Information about the Java testing framework was removed -({es-issue}55257[55257]) from the {es} Reference -because it was out of date and erroneously implied that it should be used by application developers. -There is an issue ({es-issue}55258[#55258]) -for providing general testing guidance for applications that communicate with {es}. +Information about the Java testing framework was removed +({es-issue}55257[55257]) from the {es} Reference +because it was out of date and erroneously implied that it should be used by application developers. +There is an issue ({es-issue}55258[#55258]) +for providing general testing guidance for applications that communicate with {es}. [role="exclude",id="integration-tests"] === Integration tests This page was deleted. -Information about the Java testing framework was removed -({es-issue}55257[55257]) from the {es} Reference -because it was out of date and erroneously implied that it should be used by application developers. -There is an issue ({es-issue}55258[#55258]) -for providing general testing guidance for applications that communicate with {es}. +Information about the Java testing framework was removed +({es-issue}55257[55257]) from the {es} Reference +because it was out of date and erroneously implied that it should be used by application developers. +There is an issue ({es-issue}55258[#55258]) +for providing general testing guidance for applications that communicate with {es}. [role="exclude",id="number-of-shards"] @@ -954,12 +979,12 @@ The `xpack.sql.enabled` setting has been deprecated. SQL access is always enable [role="exclude",id="indices-templates"] === Index templates [[getting]] -See <>. +See <>. [role="exclude",id="run-a-search"] === Run a search -See <>. +See <>. [role="exclude",id="how-highlighters-work-internally"] === How highlighters work internally diff --git a/docs/reference/settings/ccr-settings.asciidoc b/docs/reference/settings/ccr-settings.asciidoc index 5622af8f522e3..8124b7c367231 100644 --- a/docs/reference/settings/ccr-settings.asciidoc +++ b/docs/reference/settings/ccr-settings.asciidoc @@ -10,7 +10,7 @@ These {ccr} settings can be dynamically updated on a live cluster with the ==== Remote recovery settings The following setting can be used to rate-limit the data transmitted during -<>: +<>: `ccr.indices.recovery.max_bytes_per_sec` (<>):: Limits the total inbound and outbound remote recovery traffic on each node.