diff --git a/deploy-manage/remote-clusters.md b/deploy-manage/remote-clusters.md index d4ca57e75d..a8fc77f594 100644 --- a/deploy-manage/remote-clusters.md +++ b/deploy-manage/remote-clusters.md @@ -1,9 +1,40 @@ -# Remote clusters +--- +applies_to: + deployment: + ece: ga + eck: ga + ess: ga + self: ga + serverless: unavailable +--- -% What needs to be done: Write from scratch +# Remote clusters [remote-clusters] -% GitHub issue: https://github.com/elastic/docs-projects/issues/345 +By setting up **remote clusters**, you can connect an {{es}} cluster to other {{es}} clusters. Remote clusters can be located in different data centers, geographic regions, and run on a different type of environment: {{ech}}, {{ece}}, {{eck}}, or self-managed. -% Scope notes: "Landing page for cross cluster comms, used by CCS and CCR. -We will cover here the raw configuration at Elasticsearch level and the docs to enable remote clusters in ESS / ECE / ECK. -We can include links to the use cases of remote clusters, such as CCR and CCS." +Remote clusters are especially useful in two cases: + +- **Cross-cluster replication** + With [cross-cluster replication](/deploy-manage/tools/cross-cluster-replication.md), or CCR, you ingest data to an index on a remote cluster. This leader index is replicated to one or more read-only follower indices on your local cluster. Creating a multi-cluster architecture with cross-cluster replication enables you to configure disaster recovery, bring data closer to your users, or establish a centralized reporting cluster to process reports locally. + +- **Cross-cluster search** + [Cross-cluster search](/solutions/search/cross-cluster-search.md), or CCS, enables you to run a search request against one or more remote clusters. This capability provides each region with a global view of all clusters, allowing you to send a search request from a local cluster and return results from all connected remote clusters. For full {{ccs}} capabilities, the local and remote cluster must be on the same [subscription level](https://www.elastic.co/subscriptions). + +::::{note} about terminology +In the case of remote clusters, the {{es}} cluster or deployment initiating the connection and requests is often referred to as the **local cluster**, while the {{es}} cluster or deployment receiving the requests is referred to as the **remote cluster**. +:::: + +## Setup + +Depending on the environment the local and remote clusters are deployed on and the security model you wish to use, the exact details needed to add a remote cluster vary but generally follow the same path: + +1. **Configure trust between clusters.** In the settings of the local deployment or cluster, configure the trust security model that your remote connections will use to access the remote cluster. This step involves specifying API keys or certificates retrieved from the remote clusters. + +2. **Establish the connection.** In {{kib}} on the local cluster, finalize the connection by specifying each remote cluster's details. + +Find the instructions with details on the supported security models and available connection modes for your specific scenario: + +- [Remote clusters with {{ech}}](remote-clusters/ec-enable-ccs.md) +- [Remote clusters with {{ece}}](remote-clusters/ece-enable-ccs.md) +- [Remote clusters with {{eck}}](remote-clusters/eck-remote-clusters.md) +- [Remote clusters with self-managed installations](remote-clusters/remote-clusters-self-managed.md) \ No newline at end of file diff --git a/deploy-manage/remote-clusters/_snippets/remote-cluster-certificate-compatibility.md b/deploy-manage/remote-clusters/_snippets/remote-cluster-certificate-compatibility.md new file mode 100644 index 0000000000..50cf4b5299 --- /dev/null +++ b/deploy-manage/remote-clusters/_snippets/remote-cluster-certificate-compatibility.md @@ -0,0 +1,27 @@ +:::::{dropdown} Version compatibility table + +* Any node can communicate with another node on the same major version. For example, 9.0 can talk to any 9.x node. +* Version compatibility is symmetric, meaning that if 7.16 can communicate with 8.0, 8.0 can also communicate with 7.16. The following table depicts version compatibility between local and remote nodes. + +| | | +| --- | --- | +| | Local cluster | +| Remote cluster | 5.0–5.5 | 5.6 | 6.0–6.6 | 6.7 | 6.8 | 7.0 | 7.1–7.16 | 7.17 | 8.0–9.0 | +| 5.0–5.5 | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | +| 5.6 | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | +| 6.0–6.6 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | +| 6.7 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | +| 6.8 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | +| 7.0 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | +| 7.1–7.16 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | +| 7.17 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | +| 8.0–9.0 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | + + +::::{important} +Elastic only supports {{ccs}} on a subset of these configurations. See [Supported {{ccs}} configurations](../../../solutions/search/cross-cluster-search.md#ccs-supported-configurations). +:::: + +::::: + + diff --git a/deploy-manage/remote-clusters/ec-edit-remove-trusted-environment.md b/deploy-manage/remote-clusters/ec-edit-remove-trusted-environment.md index 88b0850948..a91d17c715 100644 --- a/deploy-manage/remote-clusters/ec-edit-remove-trusted-environment.md +++ b/deploy-manage/remote-clusters/ec-edit-remove-trusted-environment.md @@ -1,4 +1,7 @@ --- +applies_to: + deployment: + ess: ga mapped_pages: - https://www.elastic.co/guide/en/cloud/current/ec-edit-remove-trusted-environment.html --- @@ -12,7 +15,7 @@ From a deployment’s **Security** page, you can manage trusted environments tha * You want to remove or update the access level granted by a cross-cluster API key. -## Remove a trusted environment [ec_remove_a_trusted_environment] +## Remove a certificate-based trusted environment [ec_remove_a_trusted_environment] By removing a trusted environment, this deployment will no longer be able to establish remote connections using certificate trust to clusters of that environment. The remote environment will also no longer be able to connect to this deployment using certificate trust. @@ -25,11 +28,11 @@ With this method, you can only remove trusted environments relying exclusively o 2. In the list of trusted environments, locate the one you want to remove. 3. Remove it using the corresponding `delete` icon. - :::{image} ../../images/cloud-delete-trust-environment.png - :alt: button for deleting a trusted environment - ::: + :::{image} ../../images/cloud-delete-trust-environment.png + :alt: button for deleting a trusted environment + ::: -4. In Kibana, go to **Stack Management** > **Remote Clusters**. +4. In {{kib}}, go to **Stack Management** > **Remote Clusters**. 5. In the list of existing remote clusters, delete the ones corresponding to the trusted environment you removed earlier. @@ -39,14 +42,14 @@ With this method, you can only remove trusted environments relying exclusively o 2. In the list of trusted environments, locate the one you want to edit. 3. Open its details by selecting the `Edit` icon. - :::{image} ../../images/cloud-edit-trust-environment.png - :alt: button for editing a trusted environment - ::: + :::{image} ../../images/cloud-edit-trust-environment.png + :alt: button for editing a trusted environment + ::: 4. Edit the trust configuration for that environment: - * From the **Trust level** tab, you can add or remove trusted deployments. - * From the **Environment settings** tab, you can manage the certificates and the label of the environment. + * From the **Trust level** tab, you can add or remove trusted deployments. + * From the **Environment settings** tab, you can manage the certificates and the label of the environment. 5. Save your changes. @@ -56,11 +59,11 @@ With this method, you can only remove trusted environments relying exclusively o This section describes the steps to change the API key used for an existing remote connection. For example, if the previous key expired and you need to rotate it with a new one. ::::{note} -If you need to update the permissions granted by a cross-cluster API key for a remote connection, you only need to update the privileges granted by the API key directly in Kibana. +If you need to update the permissions granted by a cross-cluster API key for a remote connection, you only need to update the privileges granted by the API key directly in {{kib}}. :::: -1. On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key with the appropriate permissions. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. +1. On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key with the appropriate permissions. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. 2. Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next steps. 3. Go to the **Security** page of the local deployment and locate the **Remote connections** section. 4. Locate the API key currently used for connecting to the remote cluster, copy its current alias, and delete it. @@ -68,16 +71,14 @@ If you need to update the permissions granted by a cross-cluster API key for a r * For the **Setting name**, enter the same alias that was used for the previous key. - ::::{note} - If you use a different alias, you also need to re-create the remote cluster in Kibana with a **Name** that matches the new alias. - :::: + ::::{note} + If you use a different alias, you also need to re-create the remote cluster in {{kib}} with a **Name** that matches the new alias. + :::: - * For the **Secret**, paste the encoded cross-cluster API key. + * For the **Secret**, paste the encoded cross-cluster API key, then click **Add** to save the API key to the keystore. - 1. Click **Add** to save the API key to the keystore. +6. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
-6. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
- - ::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. - :::: + ::::{note} + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + :::: diff --git a/deploy-manage/remote-clusters/ec-enable-ccs-for-eck.md b/deploy-manage/remote-clusters/ec-enable-ccs-for-eck.md index f3207d4521..25ffa5a6e1 100644 --- a/deploy-manage/remote-clusters/ec-enable-ccs-for-eck.md +++ b/deploy-manage/remote-clusters/ec-enable-ccs-for-eck.md @@ -1,11 +1,16 @@ --- +applies_to: + deployment: + ess: ga + eck: ga +navigation_title: With {{eck}} mapped_pages: - https://www.elastic.co/guide/en/cloud/current/ec-enable-ccs-for-eck.html --- -# Enabling CCS/R between Elasticsearch Service and ECK [ec-enable-ccs-for-eck] +# Remote clusters between {{ech}} and ECK [ec-enable-ccs-for-eck] -These steps describe how to configure remote clusters between an {{es}} cluster in Elasticsearch Service and an {{es}} cluster running within [Elastic Cloud on Kubernetes (ECK)](/deploy-manage/deploy/cloud-on-k8s.md). Once that’s done, you’ll be able to [run CCS queries from {{es}}](/solutions/search/cross-cluster-search.md) or [set up CCR](/deploy-manage/tools/cross-cluster-replication/set-up-cross-cluster-replication.md). +These steps describe how to configure remote clusters between an {{es}} cluster in {{ech}} and an {{es}} cluster running within [{{eck}} (ECK)](/deploy-manage/deploy/cloud-on-k8s.md). Once that’s done, you’ll be able to [run CCS queries from {{es}}](/solutions/search/cross-cluster-search.md) or [set up CCR](/deploy-manage/tools/cross-cluster-replication/set-up-cross-cluster-replication.md). ## Establish trust between two clusters [ec_establish_trust_between_two_clusters] @@ -13,7 +18,7 @@ These steps describe how to configure remote clusters between an {{es}} cluster The first step is to establish trust between the two clusters. -### Establish trust in the Elasticsearch Service cluster [ec_establish_trust_in_the_elasticsearch_service_cluster] +### Establish trust in the {{ech}} cluster [ec_establish_trust_in_the_elasticsearch_service_cluster] 1. Save the ECK CA certificate to a file. For a cluster named `quickstart`, run: @@ -22,7 +27,7 @@ The first step is to establish trust between the two clusters. ``` -1. Update the trust settings for the Elasticsearch Service deployment. Follow the steps provided in [Access clusters of a self-managed environment](ec-remote-cluster-self-managed.md), and specifically the first three steps in **Specify the deployments trusted to be used as remote clusters** using TLS certificate as security model. +1. Update the trust settings for the {{ech}} deployment. Follow the steps provided in [Access clusters of a self-managed environment](ec-remote-cluster-self-managed.md), and specifically the first three steps in **Specify the deployments trusted to be used as remote clusters** using TLS certificate as security model. * Use the certificate file saved in the first step. * Select the {{ecloud}} pattern and enter `default.es.local` for the `Scope ID`. @@ -32,7 +37,7 @@ The first step is to establish trust between the two clusters. ### Establish trust in the ECK cluster [ec_establish_trust_in_the_eck_cluster] -1. Upload the Elasticsearch Service certificate (that you downloaded in the last step of the previous section) as a Kubernetes secret. +1. Upload the {{ech}} certificate (that you downloaded in the last step of the previous section) as a Kubernetes secret. ```sh kubectl create secret generic ce-aws-cert --from-file= @@ -73,16 +78,16 @@ The first step is to establish trust between the two clusters. -## Setup CCS/R [ec_setup_ccsr] +## Set up CCS/R [ec_setup_ccsr] -Now that trust has been established, you can set up CCS/R from the ECK cluster to the Elasticsearch Service cluster or from the Elasticsearch Service cluster to the ECK cluster. +Now that trust has been established, you can set up CCS/R from the ECK cluster to the {{ech}} cluster or from the {{ech}} cluster to the ECK cluster. -### ECK Cluster to Elasticsearch Service cluster [ec_eck_cluster_to_elasticsearch_service_cluster] +### ECK Cluster to {{ech}} cluster [ec_eck_cluster_to_elasticsearch_service_cluster] Configure the ECK cluster [using certificate based authentication](ec-remote-cluster-self-managed.md). -### Elasticsearch Service cluster to ECK Cluster [ec_elasticsearch_service_cluster_to_eck_cluster] +### {{ech}} cluster to ECK Cluster [ec_elasticsearch_service_cluster_to_eck_cluster] Follow the steps outlined in the [ECK documentation](/deploy-manage/remote-clusters/eck-remote-clusters.md#k8s_configure_the_remote_cluster_connection_through_the_elasticsearch_rest_api). diff --git a/deploy-manage/remote-clusters/ec-enable-ccs.md b/deploy-manage/remote-clusters/ec-enable-ccs.md index ceb1506f92..94c1335cdd 100644 --- a/deploy-manage/remote-clusters/ec-enable-ccs.md +++ b/deploy-manage/remote-clusters/ec-enable-ccs.md @@ -1,57 +1,59 @@ --- +applies_to: + deployment: + ess: ga +navigation_title: Elastic Cloud Hosted mapped_pages: - https://www.elastic.co/guide/en/cloud/current/ec-enable-ccs.html --- -# Enable cross-cluster search and cross-cluster replication [ec-enable-ccs] +# Remote clusters with {{ech}} [ec-enable-ccs] -[Cross-cluster search (CCS)](/solutions/search/cross-cluster-search.md) allows you to configure multiple remote clusters across different locations and to enable federated search queries across all of the configured remote clusters. +You can configure an {{ech}} deployment to remotely access or (be accessed by) a cluster from: -[Cross-cluster replication (CCR)](/deploy-manage/tools/cross-cluster-replication.md) allows you to replicate indices across multiple remote clusters regardless of where they’re located. This provides tremendous benefit in scenarios of disaster recovery or data locality. - -These remote clusters could be: - -* Another {{es}} cluster of your {{ecloud}} organization across any region or cloud provider (AWS, GCP, Azure…​) -* An {{es}} cluster of another {{ecloud}} organization -* An {{es}} cluster in an {{ece}} installation -* Any other self-managed {{es}} cluster +* Another {{ech}} deployment of your {{ecloud}} organization, across any region or cloud provider (AWS, GCP, Azure…​) +* An {{ech}} deployment of another {{ecloud}} organization +* A deployment in an {{ece}} installation +* A deployment in an {{eck}} installation +* A self-managed installation. ## Prerequisites [ec-ccs-ccr-prerequisites] To use CCS or CCR, your deployments must meet the following criteria: -* Local and remote clusters must be in compatible versions. Review the [{{es}} version compatibility](/deploy-manage/remote-clusters/remote-clusters-cert.md#remote-clusters-prerequisites-cert) table. +* The local and remote clusters must run on compatible versions of {{es}}. Review the version compatibility table. + + :::{include} _snippets/remote-cluster-certificate-compatibility.md + ::: + +* If your deployment was created before February 2021, the **Remote clusters** page in {{kib}} must be enabled manually from the **Security** page of your deployment, by selecting **Enable CCR** under **Trust management**. + +## Set up remote clusters with {{ech}} The steps, information, and authentication method required to configure CCS and CCR can vary depending on where the clusters you want to use as remote are hosted. -* Connect remotely to other clusters from your Elasticsearch Service deployments +* Connect remotely to other clusters from your {{ech}} deployments - * [Access other deployments of the same Elasticsearch Service organization](ec-remote-cluster-same-ess.md) - * [Access deployments of a different Elasticsearch Service organization](ec-remote-cluster-other-ess.md) + * [Access other deployments of the same {{ecloud}} organization](ec-remote-cluster-same-ess.md) + * [Access deployments of a different {{ecloud}} organization](ec-remote-cluster-other-ess.md) * [Access deployments of an {{ECE}} environment](ec-remote-cluster-ece.md) * [Access clusters of a self-managed environment](ec-remote-cluster-self-managed.md) * [Access deployments of an ECK environment](ec-enable-ccs-for-eck.md) -* Use clusters from your Elasticsearch Service deployments as remote - - * [From another deployment of your Elasticsearch Service organization](ec-remote-cluster-same-ess.md) - * [From a deployment of another Elasticsearch Service organization](ec-remote-cluster-other-ess.md) - * [From an ECE deployment](/deploy-manage/remote-clusters/ece-enable-ccs.md) - * [From a self-managed cluster](/deploy-manage/remote-clusters/remote-clusters-self-managed.md) - - - -## Enable CCR and the Remote Clusters UI in Kibana [ec-enable-ccr] +* Use clusters from your {{ech}} deployments as remote -If your deployment was created before February 2021, CCR won’t be enabled by default and you won’t find the Remote Clusters UI in Kibana even though your deployment meets all the [criteria](#ec-ccs-ccr-prerequisites). + * [From another deployment of your {{ecloud}} organization](ec-remote-cluster-same-ess.md) + * [From a deployment of another {{ecloud}} organization](ec-remote-cluster-other-ess.md) + * [From an ECE deployment](ece-remote-cluster-ece-ess.md) + * [From a self-managed cluster](remote-clusters-self-managed.md) + * [From an ECK environment](ec-enable-ccs-for-eck.md) -To enable these features, go to the **Security** page of your deployment and under **Trust management** select **Enable CCR**. ## Remote clusters and traffic filtering [ec-ccs-ccr-traffic-filtering] ::::{note} -Traffic filtering isn’t supported for cross-cluster operations initiated from an {{ece}} environment to a remote {{ess}} deployment. +Traffic filtering isn’t supported for cross-cluster operations initiated from an {{ece}} environment to a remote {{ech}} deployment. :::: @@ -60,8 +62,8 @@ For remote clusters configured using TLS certificate authentication, [traffic fi Traffic filtering for remote clusters supports 2 methods: * [Filtering by IP addresses and Classless Inter-Domain Routing (CIDR) masks](../security/ip-traffic-filtering.md) -* Filtering by Organization or Elasticsearch cluster ID with a Remote cluster type filter. You can configure this type of filter from the **Features** > **Traffic filters** page of your organization or using the [Elasticsearch Service API](https://www.elastic.co/docs/api/doc/cloud) and apply it from each deployment’s **Security** page. +* Filtering by Organization or {{es}} cluster ID with a Remote cluster type filter. You can configure this type of filter from the **Features** > **Traffic filters** page of your organization or using the [{{ecloud}} RESTful API](https://www.elastic.co/docs/api/doc/cloud) and apply it from each deployment’s **Security** page. ::::{note} -When setting up traffic filters for a remote connection to an {{ece}} environment, you also need to upload the region’s TLS certificate of the local cluster to the {{ece}} environment’s proxy. You can find that region’s TLS certificate in the Security page of any deployment of the environment initiating the remote connection. +When setting up traffic filters for a remote connection to an {{ece}} environment, you also need to upload the region’s TLS certificate of the local cluster to the {{ece}} environment’s proxy. You can find that region’s TLS certificate in the **Security** page of any deployment of the environment initiating the remote connection. :::: diff --git a/deploy-manage/remote-clusters/ec-migrate-ccs.md b/deploy-manage/remote-clusters/ec-migrate-ccs.md index a4dbf6183c..ac7a923299 100644 --- a/deploy-manage/remote-clusters/ec-migrate-ccs.md +++ b/deploy-manage/remote-clusters/ec-migrate-ccs.md @@ -1,11 +1,15 @@ --- +applies_to: + deployment: + ess: ga mapped_pages: - https://www.elastic.co/guide/en/cloud/current/ec-migrate-ccs.html +navigation_title: "Migrate the CCS deployment template" --- # Migrate the cross-cluster search deployment template [ec-migrate-ccs] -The cross-cluster search deployment template is now deprecated and has been removed from the Elasticsearch Service user console. You no longer need to use the dedicated cross-cluster template to search across deployments. Instead, you can now use any template to [configure remote clusters](ec-enable-ccs.md) and search across them. Existing deployments created using this template are not affected, but they are required to migrate to another template before upgrading to version 8.x. +The cross-cluster search deployment template is now deprecated and has been removed from the {{ecloud}} Console. You no longer need to use the dedicated cross-cluster template to search across deployments. Instead, you can now use any template to [configure remote clusters](ec-enable-ccs.md) and search across them. Existing deployments created using this template are not affected, but they are required to migrate to another template before upgrading to {{stack}} 8.x. There are two different approaches to do this migration: @@ -17,14 +21,14 @@ There are two different approaches to do this migration: You can use a PUT request to update your deployment, changing both the deployment template ID and the instances required by the new template. -1. First, choose the new template you want to use and obtain its ID. This template ID can be obtained from the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body) **Create Deployment** page by selecting **Equivalent API request** and inspecting the result for the field `deployment_template`. For example, we are going to use the "Storage optimized" deployment template, and in our GCP region the id is `gcp-storage-optimized-v5`. +1. First, choose the new template you want to use and obtain its ID. This template ID can be obtained from the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body) **Create Deployment** page by selecting **Equivalent API request** and inspecting the result for the field `deployment_template`. For example, we are going to use the "Storage optimized" deployment template, and in our GCP region the id is `gcp-storage-optimized-v5`. - You can also find the template in the [list of templates available for each region](asciidocalypse://docs/cloud/docs/reference/cloud/cloud-hosted/ec-regions-templates-instances.md). + You can also find the template in the [list of templates available for each region](asciidocalypse://docs/cloud/docs/reference/cloud/cloud-hosted/ec-regions-templates-instances.md). - :::{image} ../../images/cloud-ec-migrate-deployment-template(2).png - :alt: Deployment Template ID - :class: screenshot - ::: + :::{image} ../../images/cloud-ec-migrate-deployment-template(2).png + :alt: Deployment Template ID + :class: screenshot + ::: 2. Make a request to update your deployment with two changes: @@ -32,224 +36,224 @@ You can use a PUT request to update your deployment, changing both the deploymen 2. Change the cluster topology to match the new template that your deployment will migrate to. -```sh -curl -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC_API_KEY" https://api.elastic-cloud.com/api/v1/deployments/$DEPLOYMENT_ID -d "{ - "resources": { - "integrations_server": [ - { - "elasticsearch_cluster_ref_id": "main-elasticsearch", - "region": "gcp-us-central1", - "plan": { - "cluster_topology": [ - { - "instance_configuration_id": "gcp.integrationsserver.n2.68x32x45.2", - "zone_count": 1, - "size": { - "resource": "memory", - "value": 1024 - } - } - ], - "integrations_server": { - "version": "8.7.1" - } - }, - "ref_id": "main-integrations_server" - } - ], - "elasticsearch": [ - { - "region": "gcp-us-central1", - "settings": { - "dedicated_masters_threshold": 6 - }, - "plan": { - "cluster_topology": [ - { - "zone_count": 2, - "elasticsearch": { - "node_attributes": { - "data": "hot" + ```sh + curl -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC_API_KEY" https://api.elastic-cloud.com/api/v1/deployments/$DEPLOYMENT_ID -d "{ + "resources": { + "integrations_server": [ + { + "elasticsearch_cluster_ref_id": "main-elasticsearch", + "region": "gcp-us-central1", + "plan": { + "cluster_topology": [ + { + "instance_configuration_id": "gcp.integrationsserver.n2.68x32x45.2", + "zone_count": 1, + "size": { + "resource": "memory", + "value": 1024 + } } - }, - "instance_configuration_id": "gcp.es.datahot.n2.68x10x45", - "node_roles": [ - "master", - "ingest", - "transform", - "data_hot", - "remote_cluster_client", - "data_content" ], - "id": "hot_content", - "size": { - "resource": "memory", - "value": 8192 + "integrations_server": { + "version": "8.7.1" } }, - { - "zone_count": 2, - "elasticsearch": { - "node_attributes": { - "data": "warm" - } - }, - "instance_configuration_id": "gcp.es.datawarm.n2.68x10x190", - "node_roles": [ - "data_warm", - "remote_cluster_client" - ], - "id": "warm", - "size": { - "resource": "memory", - "value": 0 - } + "ref_id": "main-integrations_server" + } + ], + "elasticsearch": [ + { + "region": "gcp-us-central1", + "settings": { + "dedicated_masters_threshold": 6 }, - { - "zone_count": 1, - "elasticsearch": { - "node_attributes": { - "data": "cold" + "plan": { + "cluster_topology": [ + { + "zone_count": 2, + "elasticsearch": { + "node_attributes": { + "data": "hot" + } + }, + "instance_configuration_id": "gcp.es.datahot.n2.68x10x45", + "node_roles": [ + "master", + "ingest", + "transform", + "data_hot", + "remote_cluster_client", + "data_content" + ], + "id": "hot_content", + "size": { + "resource": "memory", + "value": 8192 + } + }, + { + "zone_count": 2, + "elasticsearch": { + "node_attributes": { + "data": "warm" + } + }, + "instance_configuration_id": "gcp.es.datawarm.n2.68x10x190", + "node_roles": [ + "data_warm", + "remote_cluster_client" + ], + "id": "warm", + "size": { + "resource": "memory", + "value": 0 + } + }, + { + "zone_count": 1, + "elasticsearch": { + "node_attributes": { + "data": "cold" + } + }, + "instance_configuration_id": "gcp.es.datacold.n2.68x10x190", + "node_roles": [ + "data_cold", + "remote_cluster_client" + ], + "id": "cold", + "size": { + "resource": "memory", + "value": 0 + } + }, + { + "zone_count": 1, + "elasticsearch": { + "node_attributes": { + "data": "frozen" + } + }, + "instance_configuration_id": "gcp.es.datafrozen.n2.68x10x95", + "node_roles": [ + "data_frozen" + ], + "id": "frozen", + "size": { + "resource": "memory", + "value": 0 + } + }, + { + "zone_count": 3, + "instance_configuration_id": "gcp.es.master.n2.68x32x45.2", + "node_roles": [ + "master", + "remote_cluster_client" + ], + "id": "master", + "size": { + "resource": "memory", + "value": 0 + } + }, + { + "zone_count": 2, + "instance_configuration_id": "gcp.es.coordinating.n2.68x16x45.2", + "node_roles": [ + "ingest", + "remote_cluster_client" + ], + "id": "coordinating", + "size": { + "resource": "memory", + "value": 0 + } + }, + { + "zone_count": 1, + "instance_configuration_id": "gcp.es.ml.n2.68x32x45", + "node_roles": [ + "ml", + "remote_cluster_client" + ], + "id": "ml", + "size": { + "resource": "memory", + "value": 0 + } } - }, - "instance_configuration_id": "gcp.es.datacold.n2.68x10x190", - "node_roles": [ - "data_cold", - "remote_cluster_client" ], - "id": "cold", - "size": { - "resource": "memory", - "value": 0 - } - }, - { - "zone_count": 1, "elasticsearch": { - "node_attributes": { - "data": "frozen" - } + "version": "8.7.1", + "enabled_built_in_plugins": [] }, - "instance_configuration_id": "gcp.es.datafrozen.n2.68x10x95", - "node_roles": [ - "data_frozen" - ], - "id": "frozen", - "size": { - "resource": "memory", - "value": 0 + "deployment_template": { + "id": "gcp-storage-optimized-v5" } }, - { - "zone_count": 3, - "instance_configuration_id": "gcp.es.master.n2.68x32x45.2", - "node_roles": [ - "master", - "remote_cluster_client" + "ref_id": "main-elasticsearch" + } + ], + "enterprise_search": [ + { + "elasticsearch_cluster_ref_id": "main-elasticsearch", + "region": "gcp-us-central1", + "plan": { + "cluster_topology": [ + { + "node_type": { + "connector": true, + "appserver": true, + "worker": true + }, + "instance_configuration_id": "gcp.enterprisesearch.n2.68x32x45", + "zone_count": 1, + "size": { + "resource": "memory", + "value": 2048 + } + } ], - "id": "master", - "size": { - "resource": "memory", - "value": 0 + "enterprise_search": { + "version": "8.7.1" } }, - { - "zone_count": 2, - "instance_configuration_id": "gcp.es.coordinating.n2.68x16x45.2", - "node_roles": [ - "ingest", - "remote_cluster_client" + "ref_id": "main-enterprise_search" + } + ], + "kibana": [ + { + "elasticsearch_cluster_ref_id": "main-elasticsearch", + "region": "gcp-us-central1", + "plan": { + "cluster_topology": [ + { + "instance_configuration_id": "gcp.kibana.n2.68x32x45", + "zone_count": 1, + "size": { + "resource": "memory", + "value": 1024 + } + } ], - "id": "coordinating", - "size": { - "resource": "memory", - "value": 0 + "kibana": { + "version": "8.7.1" } }, - { - "zone_count": 1, - "instance_configuration_id": "gcp.es.ml.n2.68x32x45", - "node_roles": [ - "ml", - "remote_cluster_client" - ], - "id": "ml", - "size": { - "resource": "memory", - "value": 0 - } - } - ], - "elasticsearch": { - "version": "8.7.1", - "enabled_built_in_plugins": [] - }, - "deployment_template": { - "id": "gcp-storage-optimized-v5" - } - }, - "ref_id": "main-elasticsearch" - } - ], - "enterprise_search": [ - { - "elasticsearch_cluster_ref_id": "main-elasticsearch", - "region": "gcp-us-central1", - "plan": { - "cluster_topology": [ - { - "node_type": { - "connector": true, - "appserver": true, - "worker": true - }, - "instance_configuration_id": "gcp.enterprisesearch.n2.68x32x45", - "zone_count": 1, - "size": { - "resource": "memory", - "value": 2048 - } - } - ], - "enterprise_search": { - "version": "8.7.1" - } - }, - "ref_id": "main-enterprise_search" - } - ], - "kibana": [ - { - "elasticsearch_cluster_ref_id": "main-elasticsearch", - "region": "gcp-us-central1", - "plan": { - "cluster_topology": [ - { - "instance_configuration_id": "gcp.kibana.n2.68x32x45", - "zone_count": 1, - "size": { - "resource": "memory", - "value": 1024 - } - } - ], - "kibana": { - "version": "8.7.1" + "ref_id": "main-kibana" } - }, - "ref_id": "main-kibana" + ] + }, + "settings": { + "autoscaling_enabled": false + }, + "name": "My deployment", + "metadata": { + "system_owned": false } - ] - }, - "settings": { - "autoscaling_enabled": false - }, - "name": "My deployment", - "metadata": { - "system_owned": false - } -}" -``` + }" + ``` `DEPLOYMENT_ID` : The ID of your deployment, as shown in the Cloud UI or obtained through the API. @@ -262,7 +266,7 @@ Note that the `ref_id` and version numbers for your resources may not be the sam ## Use a snapshot to migrate deployments that use the cross-cluster search deployment template [ec-migrate-ccs-deployment-using-snapshot] -You can make this change in the user [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). The only drawback of this method is that it changes the URL used to access the {{es}} cluster and Kibana. +You can make this change in the user [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). The only drawback of this method is that it changes the URL used to access the {{es}} cluster and {{kib}}. 1. From the deployment menu, open the **Snapshots** page and click **Take Snapshot now**. Wait for the snapshot to finish. 2. From the main **Deployments** page, click **Create deployment**. Next to **Settings** toggle on **Restore snapshot data**, and then select your deployment and the snapshot that you created. diff --git a/deploy-manage/remote-clusters/ec-remote-cluster-ece.md b/deploy-manage/remote-clusters/ec-remote-cluster-ece.md index 31851cdbff..19ad49c5fa 100644 --- a/deploy-manage/remote-clusters/ec-remote-cluster-ece.md +++ b/deploy-manage/remote-clusters/ec-remote-cluster-ece.md @@ -1,9 +1,14 @@ --- +applies_to: + deployment: + ess: ga + ece: ga +navigation_title: With {{ece}} mapped_pages: - https://www.elastic.co/guide/en/cloud/current/ec-remote-cluster-ece.html --- -# Access deployments of an Elastic Cloud Enterprise environment [ec-remote-cluster-ece] +# Access deployments of an {{ece}} environment [ec-remote-cluster-ece] This section explains how to configure a deployment to connect remotely to clusters belonging to an {{ECE}} (ECE) environment. @@ -12,71 +17,13 @@ This section explains how to configure a deployment to connect remotely to clust Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps. API key -: For deployments based on {{stack}} version 8.10 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. +: For deployments based on {{stack}} 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. -TLS certificate +TLS certificate (deprecated in {{stack}} 9.0.0) : This model uses mutual TLS authentication for cross-cluster operations. User authentication is performed on the local cluster and a user’s role names are passed to the remote cluster. A superuser on the local deployment gains total read access to the remote deployment, so it is only suitable for deployments that are in the same security domain. :::::::{tab-set} -::::::{tab-item} TLS certificate -#### Configuring trust with clusters of an {{ece}} environment [ec-trust-ece] - -A deployment can be configured to trust all or specific deployments in a remote ECE environment: - -1. Access the **Security** page of the deployment you want to use for cross-cluster operations. -2. Select **Remote Connections > Add trusted environment** and choose **{{ece}}**. Then click **Next**. -3. Select **Certificates** as authentication mechanism and click **Next**. -4. Enter the environment ID of the ECE environment. You can find it under Platform > Trust Management in your ECE administration UI. -5. Upload the Certificate Authority of the ECE environment. You can download it from Platform > Trust Management in your ECE administration UI. -6. Choose one of following options to configure the level of trust with the ECE environment: - - * All deployments - This deployment trusts all deployments in the ECE environment, including new deployments when they are created. - * Specific deployments - Specify which of the existing deployments you want to trust in the ECE environment. The full Elasticsearch cluster ID must be entered for each remote cluster. The Elasticsearch `Cluster ID` can be found in the deployment overview page under **Applications**. - -7. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s Security page. -8. Select **Create trust** to complete the configuration. -9. Configure the corresponding deployments of the ECE environment to [trust this deployment](/deploy-manage/remote-clusters/ece-enable-ccs.md). You will only be able to connect 2 deployments successfully when both of them trust each other. - -Note that the environment ID and cluster IDs must be entered fully and correctly. For security reasons, no verification of the IDs is possible. If cross-environment trust does not appear to be working, double-checking the IDs is a good place to start. - -::::{dropdown} **Using the API** -You can update a deployment using the appropriate trust settings for the {{es}} payload. - -In order to trust a deployment with cluster id `cf659f7fe6164d9691b284ae36811be1` (NOTE: use the {{es}} cluster ID, not the deployment ID) in an ECE environment with environment ID `1053523734`, you need to update the trust settings with an additional direct trust relationship like this: - -```json -{ - "trust":{ - "accounts":[ - { - "account_id":"ec38dd0aa45f4a69909ca5c81c27138a", - "trust_all":true - } - ], - "direct": [ - { - "type" : "ECE", - "name" : "My ECE environment", - "scope_id" : "1053523734", - "certificates" : [ - { - "pem" : "-----BEGIN CERTIFICATE-----\nMIIDTzCCA...H0=\n-----END CERTIFICATE-----" - } - ], - "trust_all":false, - "trust_allowlist":[ - "cf659f7fe6164d9691b284ae36811be1" - ] - } - ] - } -} -``` - -:::: -:::::: - ::::::{tab-item} API key API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. @@ -84,33 +31,31 @@ All cross-cluster requests from the local cluster are bound by the API key’s p On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). -#### Prerequisites and limitations [ec_prerequisites_and_limitations_3] +### Prerequisites and limitations [ec_prerequisites_and_limitations_3] -* The local and remote deployments must be on version 8.12 or later. +* The local and remote deployments must be on {{stack}} 8.14 or later. * API key authentication can’t be used in combination with traffic filters. * Contrary to the certificate security model, the API key security model does not require that both local and remote clusters trust each other. -#### Create a cross-cluster API key on the remote deployment [ec_create_a_cross_cluster_api_key_on_the_remote_deployment_3] +### Create a cross-cluster API key on the remote deployment [ec_create_a_cross_cluster_api_key_on_the_remote_deployment_3] -* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. +* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. * Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. -#### Configure the local deployment [ec_configure_the_local_deployment] +### Configure the local deployment [ec_configure_the_local_deployment] The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. The steps to follow depend on whether the Certificate Authority (CA) of the remote ECE environment’s proxy or load balancing infrastructure is public or private. -**The CA is public** - -::::{dropdown} -1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page in the Elasticsearch Service card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. +::::{dropdown} The CA is public +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). +2. On the home page, find your hosted deployment and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. @@ -124,10 +69,10 @@ The steps to follow depend on whether the Certificate Authority (CA) of the remo 2. Click **Add** to save the API key to the keystore. -5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
+5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. :::: @@ -136,11 +81,9 @@ If you later need to update the remote connection with different permissions, yo :::: -**The CA is private** - -::::{dropdown} -1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page in the Elasticsearch Service card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. +::::{dropdown} The CA is private +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). +2. On the home page, find your hosted deployment and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. @@ -168,12 +111,12 @@ If you later need to update the remote connection with different permissions, yo :alt: Certificate to copy from the chain ::: -8. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s Security page. +8. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s **Security** page. 9. Select **Create trust** to complete the configuration. -10. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
+10. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. :::: @@ -182,56 +125,114 @@ If you later need to update the remote connection with different permissions, yo :::: :::::: +::::::{tab-item} TLS certificate (deprecated) +### Configuring trust with clusters of an {{ece}} environment [ec-trust-ece] + +A deployment can be configured to trust all or specific deployments in a remote ECE environment: + +1. Access the **Security** page of the deployment you want to use for cross-cluster operations. +2. Select **Remote Connections > Add trusted environment** and choose **{{ece}}**. Then click **Next**. +3. Select **Certificates** as authentication mechanism and click **Next**. +4. Enter the environment ID of the ECE environment. You can find it under Platform > Trust Management in your ECE administration UI. +5. Upload the Certificate Authority of the ECE environment. You can download it from Platform > Trust Management in your ECE administration UI. +6. Choose one of following options to configure the level of trust with the ECE environment: + + * All deployments - This deployment trusts all deployments in the ECE environment, including new deployments when they are created. + * Specific deployments - Specify which of the existing deployments you want to trust in the ECE environment. The full {{es}} cluster ID must be entered for each remote cluster. The {{es}} `Cluster ID` can be found in the deployment overview page under **Applications**. + +7. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s **Security** page. +8. Select **Create trust** to complete the configuration. +9. Configure the corresponding deployments of the ECE environment to [trust this deployment](/deploy-manage/remote-clusters/ece-enable-ccs.md). You will only be able to connect 2 deployments successfully when both of them trust each other. + +Note that the environment ID and cluster IDs must be entered fully and correctly. For security reasons, no verification of the IDs is possible. If cross-environment trust does not appear to be working, double-checking the IDs is a good place to start. + +::::{dropdown} Using the API +You can update a deployment using the appropriate trust settings for the {{es}} payload. + +In order to trust a deployment with cluster id `cf659f7fe6164d9691b284ae36811be1` (NOTE: use the {{es}} cluster ID, not the deployment ID) in an ECE environment with environment ID `1053523734`, you need to update the trust settings with an additional direct trust relationship like this: + +```json +{ + "trust":{ + "accounts":[ + { + "account_id":"ec38dd0aa45f4a69909ca5c81c27138a", + "trust_all":true + } + ], + "direct": [ + { + "type" : "ECE", + "name" : "My ECE environment", + "scope_id" : "1053523734", + "certificates" : [ + { + "pem" : "-----BEGIN CERTIFICATE-----\nMIIDTzCCA...H0=\n-----END CERTIFICATE-----" + } + ], + "trust_all":false, + "trust_allowlist":[ + "cf659f7fe6164d9691b284ae36811be1" + ] + } + ] + } +} +``` + +:::: +:::::: + ::::::: You can now connect remotely to the trusted clusters. ## Connect to the remote cluster [ec_connect_to_the_remote_cluster_3] -On the local cluster, add the remote cluster using Kibana or the {{es}} API. +On the local cluster, add the remote cluster using {{kib}} or the {{es}} API. -### Using Kibana [ec_using_kibana_3] +### Using {{kib}} [ec_using_kibana_3] 1. Open the {{kib}} main menu, and select **Stack Management > Data > Remote Clusters > Add a remote cluster**. 2. Enable **Manually enter proxy address and server name**. 3. Fill in the following fields: * **Name**: This *cluster alias* is a unique identifier that represents the connection to the remote cluster and is used to distinguish between local and remote indices. - * **Proxy address**: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote.
+ * **Proxy address**: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote.
- ::::{tip} - If you’re using API keys as security model, change the port into `9443`. - :::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: - * **Server name**: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. + * **Server name**: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. - :::{image} ../../images/cloud-ce-copy-remote-cluster-parameters.png - :alt: Remote Cluster Parameters in Deployment - :class: screenshot - ::: + :::{image} ../../images/cloud-ce-copy-remote-cluster-parameters.png + :alt: Remote Cluster Parameters in Deployment + :class: screenshot + ::: - ::::{note} - If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). - :::: + ::::{note} + If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). + :::: 4. Click **Next**. 5. Click **Add remote cluster** (you have already established trust in a previous step). -### Using the Elasticsearch API [ec_using_the_elasticsearch_api_3] +### Using the {{es}} API [ec_using_the_elasticsearch_api_3] To configure a deployment as a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields: * `mode`: `proxy` -* `proxy_address`: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9400` using a semicolon. +* `proxy_address`: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9400` using a semicolon. -::::{tip} -If you’re using API keys as security model, change the port into `9443`. -:::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: -* `server_name`: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. +* `server_name`: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. This is an example of the API call to `_cluster/settings`: @@ -252,45 +253,11 @@ PUT /_cluster/settings } ``` -:::::{dropdown} **Stack Version above 6.7.0 and below 7.6.0** -::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model. -:::: - - -When the cluster to be configured as a remote is above 6.7.0 and below 7.6.0, the remote cluster must be configured using the [sniff mode](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) with the proxy field. For each remote cluster you need to pass the following fields: -* **Proxy**: This value can be found on the **Security** page of the deployment you want to use as a remote under the name `Proxy Address`. Also, using the API, this can be obtained from the elasticsearch resource info, concatenating the fields `metadata.endpoint` and `metadata.ports.transport_passthrough` using a semicolon. -* **Seeds**: This field is an array that must contain only one value, which is the `server name` that can be found on the **Security** page of the {{es}} deployment you want to use as a remote concatenated with `:1`. Also, using the API, this can be obtained from the {{es}} resource info, concatenating the fields `metadata.endpoint` and `1` with a semicolon. -* **Mode**: sniff (or empty, since sniff is the default value) - -This is an example of the API call to `_cluster/settings`: - -```json -{ - "persistent": { - "cluster": { - "remote": { - "my-remote-cluster-1": { - "seeds": [ - "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:1" - ], - "proxy": "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:9400" - } - } - } - } -} -``` - -::::: - - - -### Using the Elasticsearch Service RESTful API [ec_using_the_elasticsearch_service_restful_api_3] +### Using the {{ecloud}} RESTful API [ec_using_the_elasticsearch_service_restful_api_3] ::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same organization (for other scenarios,the Elasticsearch API should be used instead): +This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same organization. For other scenarios, the [{{es}} API](#ec_using_the_elasticsearch_api_3) should be used instead. :::: @@ -314,7 +281,7 @@ curl -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC_AP `REF_ID_REMOTE` : The unique ID of the {{es}} resources inside your remote deployment (you can obtain these values through the API). -Note the following when using the Elasticsearch Service RESTful API: +Note the following when using the {{ecloud}} RESTful API: 1. A cluster alias must contain only letters, numbers, dashes (-), or underscores (_). 2. To learn about skipping disconnected clusters, refer to the [{{es}} documentation](/solutions/search/cross-cluster-search.md#skip-unavailable-clusters). @@ -327,11 +294,9 @@ curl -X GET -H "Authorization: ApiKey $EC_API_KEY" https://api.elastic-cloud.com ``` ::::{note} -The response will include just the remote clusters from the same organization in Elasticsearch Service. In order to obtain the whole list of remote clusters, use Kibana or the Elasticsearch API [Elasticsearch API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. +The response will include just the remote clusters from the same {{ecloud}} organization. In order to obtain the whole list of remote clusters, use {{kib}} or the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. :::: - - ## Configure roles and users [ec_configure_roles_and_users_3] To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). diff --git a/deploy-manage/remote-clusters/ec-remote-cluster-other-ess.md b/deploy-manage/remote-clusters/ec-remote-cluster-other-ess.md index 4892d367ac..9604bf2ae7 100644 --- a/deploy-manage/remote-clusters/ec-remote-cluster-other-ess.md +++ b/deploy-manage/remote-clusters/ec-remote-cluster-other-ess.md @@ -1,71 +1,28 @@ --- +applies_to: + deployment: + ess: ga +navigation_title: With a different {{ecloud}} organization mapped_pages: - https://www.elastic.co/guide/en/cloud/current/ec-remote-cluster-other-ess.html --- -# Access deployments of another Elasticsearch Service organization [ec-remote-cluster-other-ess] +# Access deployments of another {{ecloud}} organization [ec-remote-cluster-other-ess] -This section explains how to configure a deployment to connect remotely to clusters belonging to a different Elasticsearch Service organization. +This section explains how to configure a deployment to connect remotely to clusters belonging to a different {{ecloud}} organization. ## Allow the remote connection [ec_allow_the_remote_connection_2] Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps. API key -: For deployments based on {{stack}} version 8.10 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. +: For deployments based on {{stack}} 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. -TLS certificate +TLS certificate (deprecated in {{stack}} 9.0.0) : This model uses mutual TLS authentication for cross-cluster operations. User authentication is performed on the local cluster and a user’s role names are passed to the remote cluster. A superuser on the local deployment gains total read access to the remote deployment, so it is only suitable for deployments that are in the same security domain. :::::::{tab-set} -::::::{tab-item} TLS certificate -#### Specify the deployments trusted to be used as remote clusters [ec-trust-other-organization] - -A deployment can be configured to trust all or specific deployments in another Elasticsearch Service [organizations](../users-roles/cloud-organization.md). To add cross-organization trust: - -1. From the **Security** menu, select **Remote Connections > Add trusted environment** and select **{{ecloud}}**. Then click **Next**. -2. Select **Certificates** as authentication mechanism and click **Next**. -3. Enter the ID of the deployment’s organization which you want to establish trust with. You can find that ID on the Organization page. It is usually made of 10 digits. -4. Choose one of following options to configure the level of trust with the other organization: - - * All deployments - This deployment trusts all deployments in the other organization, including new deployments when they are created. - * Specific deployments - Specify which of the existing deployments you want to trust in the other organization. The full Elasticsearch cluster ID must be entered for each remote cluster. The Elasticsearch `Cluster ID` can be found in the deployment overview page under **Applications**. - -5. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s Security page. -6. Select **Create trust** to complete the configuration. -7. Repeat these steps from each of the deployments you want to use for CCS or CCR in both organizations. You will only be able to connect 2 deployments successfully when both of them trust each other. - -Note that the organization ID and cluster IDs must be entered fully and correctly. For security reasons, no verification of the IDs is possible. If cross-organization trust does not appear to be working, double-checking the IDs is a good place to start. - -::::{dropdown} **Using the API** -You can update a deployment using the appropriate trust settings for the {{es}} payload. - -In order to trust a deployment with cluster id `cf659f7fe6164d9691b284ae36811be1` (NOTE: use the {{es}} cluster ID, not the deployment ID) in another organization with Organization ID `1053523734`, you need to update the trust settings with an additional account like this: - -```json -{ - "trust":{ - "accounts":[ - { - "account_id":"ec38dd0aa45f4a69909ca5c81c27138a", - "trust_all":true - }, - { - "account_id":"1053523734", - "trust_all":false, - "trust_allowlist":[ - "cf659f7fe6164d9691b284ae36811be1" - ] - } - ] - } -} -``` - -:::: -:::::: - ::::::{tab-item} API key API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. @@ -73,28 +30,28 @@ All cross-cluster requests from the local cluster are bound by the API key’s p On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). -#### Prerequisites and limitations [ec_prerequisites_and_limitations_2] +### Prerequisites and limitations [ec_prerequisites_and_limitations_2] -* The local and remote deployments must be on version 8.12 or later. +* The local and remote deployments must be on {{stack}} 8.14 or later. * API key authentication can’t be used in combination with traffic filters. * Contrary to the certificate security model, the API key security model does not require that both local and remote clusters trust each other. -#### Create a cross-cluster API key on the remote deployment [ec_create_a_cross_cluster_api_key_on_the_remote_deployment_2] +### Create a cross-cluster API key on the remote deployment [ec_create_a_cross_cluster_api_key_on_the_remote_deployment_2] -* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. +* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. * Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. -#### Add the cross-cluster API key to the keystore of the local deployment [ec_add_the_cross_cluster_api_key_to_the_keystore_of_the_local_deployment_2] +### Add the cross-cluster API key to the keystore of the local deployment [ec_add_the_cross_cluster_api_key_to_the_keystore_of_the_local_deployment_2] The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. -1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page in the Elasticsearch Service card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). +2. On the home page, find your hosted deployment and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. @@ -108,66 +65,112 @@ The API key created previously will be used by the local deployment to authentic 2. Click **Add** to save the API key to the keystore. -5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
+5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. :::: If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ec-edit-remove-trusted-environment.md#ec-edit-remove-trusted-environment-api-key). :::::: +::::::{tab-item} TLS certificate (deprecated) +### Specify the deployments trusted to be used as remote clusters [ec-trust-other-organization] + +A deployment can be configured to trust all or specific deployments in another {{ech}} [organization](../users-roles/cloud-organization.md). To add cross-organization trust: + +1. From the **Security** menu, select **Remote Connections > Add trusted environment** and select **{{ecloud}}**. Then click **Next**. +2. Select **Certificates** as authentication mechanism and click **Next**. +3. Enter the ID of the deployment’s organization which you want to establish trust with. You can find that ID on the Organization page. It is usually made of 10 digits. +4. Choose one of following options to configure the level of trust with the other organization: + + * All deployments - This deployment trusts all deployments in the other organization, including new deployments when they are created. + * Specific deployments - Specify which of the existing deployments you want to trust in the other organization. The full {{es}} cluster ID must be entered for each remote cluster. The {{es}} `Cluster ID` can be found in the deployment overview page under **Applications**. + +5. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s **Security** page. +6. Select **Create trust** to complete the configuration. +7. Repeat these steps from each of the deployments you want to use for CCS or CCR in both organizations. You will only be able to connect 2 deployments successfully when both of them trust each other. + +Note that the organization ID and cluster IDs must be entered fully and correctly. For security reasons, no verification of the IDs is possible. If cross-organization trust does not appear to be working, double-checking the IDs is a good place to start. + +::::{dropdown} Using the API +You can update a deployment using the appropriate trust settings for the {{es}} payload. + +In order to trust a deployment with cluster id `cf659f7fe6164d9691b284ae36811be1` (NOTE: use the {{es}} cluster ID, not the deployment ID) in another organization with Organization ID `1053523734`, you need to update the trust settings with an additional account like this: + +```json +{ + "trust":{ + "accounts":[ + { + "account_id":"ec38dd0aa45f4a69909ca5c81c27138a", + "trust_all":true + }, + { + "account_id":"1053523734", + "trust_all":false, + "trust_allowlist":[ + "cf659f7fe6164d9691b284ae36811be1" + ] + } + ] + } +} +``` + +:::: +:::::: ::::::: You can now connect remotely to the trusted clusters. ## Connect to the remote cluster [ec_connect_to_the_remote_cluster_2] -On the local cluster, add the remote cluster using Kibana or the {{es}} API. +On the local cluster, add the remote cluster using {{kib}} or the {{es}} API. -### Using Kibana [ec_using_kibana_2] +### Using {{kib}} [ec_using_kibana_2] 1. Open the {{kib}} main menu, and select **Stack Management > Data > Remote Clusters > Add a remote cluster**. 2. Enable **Manually enter proxy address and server name**. 3. Fill in the following fields: * **Name**: This *cluster alias* is a unique identifier that represents the connection to the remote cluster and is used to distinguish between local and remote indices. - * **Proxy address**: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote.
+ * **Proxy address**: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote.
- ::::{tip} - If you’re using API keys as security model, change the port into `9443`. - :::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: - * **Server name**: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. + * **Server name**: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. - :::{image} ../../images/cloud-ce-copy-remote-cluster-parameters.png - :alt: Remote Cluster Parameters in Deployment - :class: screenshot - ::: + :::{image} ../../images/cloud-ce-copy-remote-cluster-parameters.png + :alt: Remote Cluster Parameters in Deployment + :class: screenshot + ::: - ::::{note} - If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). - :::: + ::::{note} + If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). + :::: 4. Click **Next**. 5. Click **Add remote cluster** (you have already established trust in a previous step). -### Using the Elasticsearch API [ec_using_the_elasticsearch_api_2] +### Using the {{es}} API [ec_using_the_elasticsearch_api_2] To configure a deployment as a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields: * `mode`: `proxy` -* `proxy_address`: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9400` using a semicolon. +* `proxy_address`: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9400` using a semicolon. -::::{tip} -If you’re using API keys as security model, change the port into `9443`. -:::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: -* `server_name`: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. +* `server_name`: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. This is an example of the API call to `_cluster/settings`: @@ -188,45 +191,10 @@ PUT /_cluster/settings } ``` -:::::{dropdown} **Stack Version above 6.7.0 and below 7.6.0** -::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model. -:::: - - -When the cluster to be configured as a remote is above 6.7.0 and below 7.6.0, the remote cluster must be configured using the [sniff mode](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) with the proxy field. For each remote cluster you need to pass the following fields: - -* **Proxy**: This value can be found on the **Security** page of the deployment you want to use as a remote under the name `Proxy Address`. Also, using the API, this can be obtained from the elasticsearch resource info, concatenating the fields `metadata.endpoint` and `metadata.ports.transport_passthrough` using a semicolon. -* **Seeds**: This field is an array that must contain only one value, which is the `server name` that can be found on the **Security** page of the {{es}} deployment you want to use as a remote concatenated with `:1`. Also, using the API, this can be obtained from the {{es}} resource info, concatenating the fields `metadata.endpoint` and `1` with a semicolon. -* **Mode**: sniff (or empty, since sniff is the default value) - -This is an example of the API call to `_cluster/settings`: - -```json -{ - "persistent": { - "cluster": { - "remote": { - "my-remote-cluster-1": { - "seeds": [ - "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:1" - ], - "proxy": "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:9400" - } - } - } - } -} -``` - -::::: - - - -### Using the Elasticsearch Service RESTful API [ec_using_the_elasticsearch_service_restful_api_2] +### Using the {{ecloud}} RESTful API [ec_using_the_elasticsearch_service_restful_api_2] ::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same organization (for other scenarios,the Elasticsearch API should be used instead): +This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same organization. For other scenarios, the [{{es}} API](#ec_using_the_elasticsearch_api_2) should be used instead. :::: @@ -250,7 +218,7 @@ curl -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC_AP `REF_ID_REMOTE` : The unique ID of the {{es}} resources inside your remote deployment (you can obtain these values through the API). -Note the following when using the Elasticsearch Service RESTful API: +Note the following when using the {{ecloud}} RESTful API: 1. A cluster alias must contain only letters, numbers, dashes (-), or underscores (_). 2. To learn about skipping disconnected clusters, refer to the [{{es}} documentation](/solutions/search/cross-cluster-search.md#skip-unavailable-clusters). @@ -263,11 +231,10 @@ curl -X GET -H "Authorization: ApiKey $EC_API_KEY" https://api.elastic-cloud.com ``` ::::{note} -The response will include just the remote clusters from the same organization in Elasticsearch Service. In order to obtain the whole list of remote clusters, use Kibana or the Elasticsearch API [Elasticsearch API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. +The response will include just the remote clusters from the same {{ecloud}} organization. In order to obtain the whole list of remote clusters, use {{kib}} or the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. :::: - ## Configure roles and users [ec_configure_roles_and_users_2] -To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). +To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). \ No newline at end of file diff --git a/deploy-manage/remote-clusters/ec-remote-cluster-same-ess.md b/deploy-manage/remote-clusters/ec-remote-cluster-same-ess.md index 838ae0f5c3..8f34234a08 100644 --- a/deploy-manage/remote-clusters/ec-remote-cluster-same-ess.md +++ b/deploy-manage/remote-clusters/ec-remote-cluster-same-ess.md @@ -1,28 +1,84 @@ --- +applies_to: + deployment: + ess: ga +navigation_title: Within the same {{ecloud}} organization mapped_pages: - https://www.elastic.co/guide/en/cloud/current/ec-remote-cluster-same-ess.html --- -# Access other deployments of the same Elasticsearch Service organization [ec-remote-cluster-same-ess] +# Access other deployments of the same {{ecloud}} organization [ec-remote-cluster-same-ess] -This section explains how to configure a deployment to connect remotely to clusters belonging to the same Elasticsearch Service organization. +This section explains how to configure a deployment to connect remotely to clusters belonging to the same {{ecloud}} organization. ## Allow the remote connection [ec_allow_the_remote_connection] Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps. API key -: For deployments based on {{stack}} version 8.10 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. +: For deployments based on {{stack}} 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. -TLS certificate +TLS certificate (deprecated in {{stack}} 9.0.0) : This model uses mutual TLS authentication for cross-cluster operations. User authentication is performed on the local cluster and a user’s role names are passed to the remote cluster. A superuser on the local deployment gains total read access to the remote deployment, so it is only suitable for deployments that are in the same security domain. :::::::{tab-set} -::::::{tab-item} TLS certificate -#### Set the default trust with other clusters in the same Elasticsearch Service organization [ec_set_the_default_trust_with_other_clusters_in_the_same_elasticsearch_service_organization] +::::::{tab-item} API key +API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. + +All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key. + +On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. + +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). + -By default, any deployment that you create trusts all other deployments in the same organization. You can change this behavior in the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body) under **Features** > **Trust**, so that when a new deployment is created it does not automatically trust any other deployment. You can choose one of the following options: +### Prerequisites and limitations [ec_prerequisites_and_limitations] + +* The local and remote deployments must be on {{stack}} 8.14 or later. +* API key authentication can’t be used in combination with traffic filters. +* Contrary to the certificate security model, the API key security model does not require that both local and remote clusters trust each other. + + +### Create a cross-cluster API key on the remote deployment [ec_create_a_cross_cluster_api_key_on_the_remote_deployment] + +* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. +* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. + + +### Add the cross-cluster API key to the keystore of the local deployment [ec_add_the_cross_cluster_api_key_to_the_keystore_of_the_local_deployment] + +The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. + +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). +2. On the home page, find your hosted deployment and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. + + On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. + +3. From the deployment menu, select **Security**. +4. Locate **Remote connections** and select **Add an API key**. + + 1. Fill both fields. + + * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. + * For the **Secret**, paste the encoded cross-cluster API key. + + 2. Click **Add** to save the API key to the keystore. + +5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
+ + ::::{note} + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + :::: + + +If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ec-edit-remove-trusted-environment.md#ec-edit-remove-trusted-environment-api-key). +:::::: + +::::::{tab-item} TLS certificate (deprecated) +### Set the default trust with other clusters in the same {{ecloud}} organization [ec_set_the_default_trust_with_other_clusters_in_the_same_elasticsearch_service_organization] + +By default, any deployment that you create trusts all other deployments in the same organization. You can change this behavior in the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body) under **Features** > **Trust**, so that when a new deployment is created it does not automatically trust any other deployment. You can choose one of the following options: * Trust all my deployments - All of your organization’s deployments created while this option is selected already trust each other. If you keep this option, that includes any deployments you’ll create in the future. You can directly jump to [Connect to the remote cluster](/deploy-manage/remote-clusters/ec-remote-cluster-same-ess.md#ec_connect_to_the_remote_cluster) to finalize the CCS or CCR configuration. * Trust no deployment - New deployments won’t trust any other deployment when they are created. You can instead configure trust individually for each of them in their security settings, as described in the next section. @@ -34,13 +90,13 @@ By default, any deployment that you create trusts all other deployments in the s ::::{note} * The level of trust of existing deployments is not modified when you change this setting. You must instead update the trust settings individually for each deployment you wish to change. -* Deployments created before the Elasticsearch Service February 2021 release trust only themselves. You have to update the trust setting for each deployment that you want to either use as a remote cluster or configure to work with a remote cluster. +* Deployments created before the {{ecloud}} February 2021 release trust only themselves. You have to update the trust setting for each deployment that you want to either use as a remote cluster or configure to work with a remote cluster. :::: -#### Specify the deployments trusted to be used as remote clusters [ec_specify_the_deployments_trusted_to_be_used_as_remote_clusters] +### Specify the deployments trusted to be used as remote clusters [ec_specify_the_deployments_trusted_to_be_used_as_remote_clusters] If your organization’s deployments already trust each other by default, you can skip this section. If that’s not the case, follow these steps to configure which are the specific deployments that should be trusted. @@ -50,17 +106,16 @@ If your organization’s deployments already trust each other by default, you ca * Trust all deployments - This deployment trusts all other deployments in this environment, including new deployments when they are created. * Trust specific deployments - Choose which of the existing deployments from your environment you want to trust. - * Trust no deployment - No deployment in this Elasticsearch Service environment is trusted. - + * Trust no deployment - No deployment in this {{ech}} environment is trusted. -::::{note} -When trusting specific deployments, the more restrictive [CCS](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) version policy is used (even if you only want to use [CCR](/deploy-manage/tools/cross-cluster-replication.md)). To work around this restriction for CCR-only trust, it is necessary to use the API as described below. -:::: + ::::{note} + When trusting specific deployments, the more restrictive [CCS](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) version policy is used (even if you only want to use [CCR](/deploy-manage/tools/cross-cluster-replication.md)). To work around this restriction for CCR-only trust, it is necessary to use the API as described below. + :::: 1. Repeat these steps from each of the deployments you want to use for CCS or CCR. You will only be able to connect 2 deployments successfully when both of them trust each other. -::::{dropdown} **Using the API** +::::{dropdown} Using the API You can update a deployment using the appropriate trust settings for the {{es}} payload. The current trust settings can be found in the path `.resources.elasticsearch[0].info.settings.trust` when calling: @@ -102,109 +157,56 @@ The `account_id` above represents the only account in an {{es}} environment, and :::: :::::: - -::::::{tab-item} API key -API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. - -All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key. - -On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. - -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). - - -#### Prerequisites and limitations [ec_prerequisites_and_limitations] - -* The local and remote deployments must be on version 8.12 or later. -* API key authentication can’t be used in combination with traffic filters. -* Contrary to the certificate security model, the API key security model does not require that both local and remote clusters trust each other. - - -#### Create a cross-cluster API key on the remote deployment [ec_create_a_cross_cluster_api_key_on_the_remote_deployment] - -* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. -* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. - - -#### Add the cross-cluster API key to the keystore of the local deployment [ec_add_the_cross_cluster_api_key_to_the_keystore_of_the_local_deployment] - -The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. - -1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page in the Elasticsearch Service card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. - - On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. - -3. From the deployment menu, select **Security**. -4. Locate **Remote connections** and select **Add an API key**. - - 1. Fill both fields. - - * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. - * For the **Secret**, paste the encoded cross-cluster API key. - - 2. Click **Add** to save the API key to the keystore. - -5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
- - ::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. - :::: - - -If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ec-edit-remove-trusted-environment.md#ec-edit-remove-trusted-environment-api-key). -:::::: - ::::::: You can now connect remotely to the trusted clusters. ## Connect to the remote cluster [ec_connect_to_the_remote_cluster] -On the local cluster, add the remote cluster using Kibana or the {{es}} API. +On the local cluster, add the remote cluster using {{kib}} or the {{es}} API. -### Using Kibana [ec_using_kibana] +### Using {{kib}} [ec_using_kibana] 1. Open the {{kib}} main menu, and select **Stack Management > Data > Remote Clusters > Add a remote cluster**. 2. Enable **Manually enter proxy address and server name**. 3. Fill in the following fields: * **Name**: This *cluster alias* is a unique identifier that represents the connection to the remote cluster and is used to distinguish between local and remote indices. - * **Proxy address**: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote.
+ * **Proxy address**: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote.
- ::::{tip} - If you’re using API keys as security model, change the port into `9443`. - :::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: - * **Server name**: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. + * **Server name**: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. - :::{image} ../../images/cloud-ce-copy-remote-cluster-parameters.png - :alt: Remote Cluster Parameters in Deployment - :class: screenshot - ::: + :::{image} ../../images/cloud-ce-copy-remote-cluster-parameters.png + :alt: Remote Cluster Parameters in Deployment + :class: screenshot + ::: - ::::{note} - If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). - :::: + ::::{note} + If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). + :::: 4. Click **Next**. 5. Click **Add remote cluster** (you have already established trust in a previous step). -### Using the Elasticsearch API [ec_using_the_elasticsearch_api] +### Using the {{es}} API [ec_using_the_elasticsearch_api] To configure a deployment as a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields: * `mode`: `proxy` -* `proxy_address`: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9400` using a semicolon. +* `proxy_address`: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9400` using a semicolon. -::::{tip} -If you’re using API keys as security model, change the port into `9443`. -:::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: -* `server_name`: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. +* `server_name`: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. This is an example of the API call to `_cluster/settings`: @@ -225,45 +227,11 @@ PUT /_cluster/settings } ``` -:::::{dropdown} **Stack Version above 6.7.0 and below 7.6.0** -::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model. -:::: - - -When the cluster to be configured as a remote is above 6.7.0 and below 7.6.0, the remote cluster must be configured using the [sniff mode](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) with the proxy field. For each remote cluster you need to pass the following fields: - -* **Proxy**: This value can be found on the **Security** page of the deployment you want to use as a remote under the name `Proxy Address`. Also, using the API, this can be obtained from the elasticsearch resource info, concatenating the fields `metadata.endpoint` and `metadata.ports.transport_passthrough` using a semicolon. -* **Seeds**: This field is an array that must contain only one value, which is the `server name` that can be found on the **Security** page of the {{es}} deployment you want to use as a remote concatenated with `:1`. Also, using the API, this can be obtained from the {{es}} resource info, concatenating the fields `metadata.endpoint` and `1` with a semicolon. -* **Mode**: sniff (or empty, since sniff is the default value) - -This is an example of the API call to `_cluster/settings`: - -```json -{ - "persistent": { - "cluster": { - "remote": { - "my-remote-cluster-1": { - "seeds": [ - "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:1" - ], - "proxy": "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:9400" - } - } - } - } -} -``` - -::::: - - -### Using the Elasticsearch Service RESTful API [ec_using_the_elasticsearch_service_restful_api] +### Using the {{ecloud}} RESTful API [ec_using_the_elasticsearch_service_restful_api] ::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same organization (for other scenarios,the Elasticsearch API should be used instead): +This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same organization. For other scenarios, the [{{es}} API](#ec_using_the_elasticsearch_api) should be used instead. :::: @@ -287,7 +255,7 @@ curl -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC_AP `REF_ID_REMOTE` : The unique ID of the {{es}} resources inside your remote deployment (you can obtain these values through the API). -Note the following when using the Elasticsearch Service RESTful API: +Note the following when using the {{ecloud}} RESTful API: 1. A cluster alias must contain only letters, numbers, dashes (-), or underscores (_). 2. To learn about skipping disconnected clusters, refer to the [{{es}} documentation](/solutions/search/cross-cluster-search.md#skip-unavailable-clusters). @@ -300,11 +268,10 @@ curl -X GET -H "Authorization: ApiKey $EC_API_KEY" https://api.elastic-cloud.com ``` ::::{note} -The response will include just the remote clusters from the same organization in Elasticsearch Service. In order to obtain the whole list of remote clusters, use Kibana or the Elasticsearch API [Elasticsearch API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. +The response will include just the remote clusters from the same {{ecloud}} organization. In order to obtain the whole list of remote clusters, use {{kib}} or the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. :::: - ## Configure roles and users [ec_configure_roles_and_users] To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). diff --git a/deploy-manage/remote-clusters/ec-remote-cluster-self-managed.md b/deploy-manage/remote-clusters/ec-remote-cluster-self-managed.md index 9079c44b72..2f90496dc4 100644 --- a/deploy-manage/remote-clusters/ec-remote-cluster-self-managed.md +++ b/deploy-manage/remote-clusters/ec-remote-cluster-self-managed.md @@ -1,4 +1,9 @@ --- +applies_to: + deployment: + ess: ga + self: ga +navigation_title: With a self-managed cluster mapped_pages: - https://www.elastic.co/guide/en/cloud/current/ec-remote-cluster-self-managed.html --- @@ -12,15 +17,106 @@ This section explains how to configure a deployment to connect remotely to self- Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps. API key -: For deployments based on {{stack}} version 8.10 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. +: For deployments based on {{stack}} 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. -TLS certificate +TLS certificate (deprecated in {{stack}} 9.0.0) : This model uses mutual TLS authentication for cross-cluster operations. User authentication is performed on the local cluster and a user’s role names are passed to the remote cluster. A superuser on the local deployment gains total read access to the remote deployment, so it is only suitable for deployments that are in the same security domain. :::::::{tab-set} -::::::{tab-item} TLS certificate -#### Specify the deployments trusted to be used as remote clusters [ec-trust-self-managed] +::::::{tab-item} API key +API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. + +All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key. + +On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. + +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). + + +### Prerequisites and limitations [ec_prerequisites_and_limitations_4] + +* The local and remote deployments must be on {{stack}} 8.14 or later. +* API key authentication can’t be used in combination with traffic filters. +* Contrary to the certificate security model, the API key security model does not require that both local and remote clusters trust each other. + + +### Create a cross-cluster API key on the remote deployment [ec_create_a_cross_cluster_api_key_on_the_remote_deployment_4] + +* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. +* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. + + +### Configure the local deployment [ec_configure_the_local_deployment_2] + +The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. + +The steps to follow depend on whether the Certificate Authority (CA) of the remote environment’s {{es}} HTTPS server, proxy or, load balancing infrastructure is public or private. + +::::{dropdown} The CA is public +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). +2. On the home page, find your hosted deployment and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. + + On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. + +3. From the deployment menu, select **Security**. +4. Locate **Remote connections** and select **Add an API key**. + + 1. Add a setting: + + * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. + * For the **Secret**, paste the encoded cross-cluster API key. + + 2. Click **Add** to save the API key to the keystore. + +5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
+ + ::::{note} + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + :::: + + +If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ec-edit-remove-trusted-environment.md#ec-edit-remove-trusted-environment-api-key). + +:::: + + +::::{dropdown} The CA is private +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). +2. On the home page, find your hosted deployment and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. + + On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. + +3. Access the **Security** page of the deployment. +4. Select **Remote Connections > Add trusted environment** and choose **Self-managed**. Then click **Next**. +5. Select **API keys** as authentication mechanism and click **Next**. +6. Add a the API key: + + 1. Fill both fields. + + * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. + * For the **Secret**, paste the encoded cross-cluster API key. + + 2. Click **Add** to save the API key to the keystore. + 3. Repeat these steps for each API key you want to add. For example, if you want to use several clusters of the remote environment for CCR or CCS. + +7. Add the CA certificate of the remote self-managed environment. +8. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s **Security** page. +9. Select **Create trust** to complete the configuration. +10. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
+ + ::::{note} + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + :::: + + +If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ec-edit-remove-trusted-environment.md#ec-edit-remove-trusted-environment-api-key). + +:::: +:::::: + +::::::{tab-item} TLS certificate (deprecated) +### Specify the deployments trusted to be used as remote clusters [ec-trust-self-managed] A deployment can be configured to trust all or specific deployments in any environment: @@ -29,21 +125,21 @@ A deployment can be configured to trust all or specific deployments in any envir 3. Upload the public certificate for the Certificate Authority of the self-managed environment (the one used to sign all the cluster certificates). The certificate needs to be in PEM format and should not contain the private key. If you only have the key in p12 format, then you can create the necessary file like this: `openssl pkcs12 -in elastic-stack-ca.p12 -out newfile.crt.pem -clcerts -nokeys` 4. Select the clusters to trust. There are two options here depending on the subject name of the certificates presented by the nodes in your self managed cluster: - * Following the {{ecloud}} pattern. In {{ecloud}}, the certificates of all Elasticsearch nodes follow this convention: `CN = {{node_id}}.node.{{cluster_id}}.cluster.{{scope_id}}`. If you follow the same convention in your self-managed environment, then choose this option and you will be able to select all or specific clusters to trust. + * Following the {{ecloud}} pattern. In {{ecloud}}, the certificates of all {{es}} nodes follow this convention: `CN = {{node_id}}.node.{{cluster_id}}.cluster.{{scope_id}}`. If you follow the same convention in your self-managed environment, then choose this option and you will be able to select all or specific clusters to trust. * If your clusters don’t follow the previous convention for the certificates subject name of your nodes, you can still specify the node name of each of the nodes that should be trusted by this deployment. (Keep in mind that following this convention will simplify the management of this cluster since otherwise this configuration will need to be updated every time the topology of your self-managed cluster changes along with the trust restriction file. For this reason, it is recommended migrating your cluster certificates to follow the previous convention). ::::{note} - Trust management will not work properly in clusters without an `otherName` value specified, as is the case by default in an out-of-the-box [Elasticsearch installation](../deploy/self-managed/installing-elasticsearch.md). To have the Elasticsearch certutil generate new certificates with the `otherName` attribute, use the file input with the `cn` attribute as in the example below. + Trust management will not work properly in clusters without an `otherName` value specified, as is the case by default in an out-of-the-box [{{es}} installation](../deploy/self-managed/installing-elasticsearch.md). To have the {{es}} certutil generate new certificates with the `otherName` attribute, use the file input with the `cn` attribute as in the example below. :::: -5. . Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s Security page. +5. . Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s **Security** page. 6. Select **Create trust** to complete the configuration. 7. Configure the self-managed cluster to trust this deployment, so that both deployments are configured to trust each other: - * Download the Certificate Authority used to sign the certificates of your deployment nodes (it can be found in the Security page of your deployment) - * Trust this CA either using the [setting](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/security-settings.md) `xpack.security.transport.ssl.certificate_authorities` in `elasticsearch.yml` or by [adding it to the trust store](../security/different-ca.md). + * Download the Certificate Authority used to sign the certificates of your deployment nodes (it can be found in the Security page of your deployment) + * Trust this CA either using the [setting](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/security-settings.md) `xpack.security.transport.ssl.certificate_authorities` in `elasticsearch.yml` or by [adding it to the trust store](../security/different-ca.md). -8. Generate certificates with an `otherName` attribute using the Elasticsearch certutil. Create a file called `instances.yaml` with all the details of the nodes in your on-premise cluster like below. The `dns` and `ip` settings are optional, but `cn` is mandatory for use with the `trust_restrictions` path setting in the next step. Next, run `./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 -in instances.yaml` to create new certificates for all the nodes at once. You can then copy the resulting files into each node. +8. Generate certificates with an `otherName` attribute using the {{es}} certutil. Create a file called `instances.yaml` with all the details of the nodes in your on-premise cluster like below. The `dns` and `ip` settings are optional, but `cn` is mandatory for use with the `trust_restrictions` path setting in the next step. Next, run `./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 -in instances.yaml` to create new certificates for all the nodes at once. You can then copy the resulting files into each node. ```yaml instances: @@ -80,7 +176,7 @@ Generate new node certificates for an entire cluster using the file input mode o :::: -::::{dropdown} **Using the API** +::::{dropdown} Using the API You can update a deployment using the appropriate trust settings for the {{es}} payload. In order to trust a cluster whose nodes present certificates with the subject names: "CN = node1.example.com", "CN = node2.example.com" and "CN = node3.example.com" in a self-managed environment, you could update the trust settings with an additional direct trust relationship like this: @@ -113,152 +209,56 @@ In order to trust a cluster whose nodes present certificates with the subject na :::: :::::: - -::::::{tab-item} API key -API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. - -All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key. - -On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. - -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). - - -#### Prerequisites and limitations [ec_prerequisites_and_limitations_4] - -* The local and remote deployments must be on version 8.12 or later. -* API key authentication can’t be used in combination with traffic filters. -* Contrary to the certificate security model, the API key security model does not require that both local and remote clusters trust each other. - - -#### Create a cross-cluster API key on the remote deployment [ec_create_a_cross_cluster_api_key_on_the_remote_deployment_4] - -* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. -* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. - - -#### Configure the local deployment [ec_configure_the_local_deployment_2] - -The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. - -The steps to follow depend on whether the Certificate Authority (CA) of the remote environment’s Elasticsearch HTTPS server, proxy or, load balancing infrastructure is public or private. - -**The CA is public** - -::::{dropdown} -1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page in the Elasticsearch Service card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. - - On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. - -3. From the deployment menu, select **Security**. -4. Locate **Remote connections** and select **Add an API key**. - - 1. Add a setting: - - * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. - * For the **Secret**, paste the encoded cross-cluster API key. - - 2. Click **Add** to save the API key to the keystore. - -5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
- - ::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. - :::: - - -If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ec-edit-remove-trusted-environment.md#ec-edit-remove-trusted-environment-api-key). - -:::: - - -**The CA is private** - -::::{dropdown} -1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page in the Elasticsearch Service card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. - - On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. - -3. Access the **Security** page of the deployment. -4. Select **Remote Connections > Add trusted environment** and choose **Self-managed**. Then click **Next**. -5. Select **API keys** as authentication mechanism and click **Next**. -6. Add a the API key: - - 1. Fill both fields. - - * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. - * For the **Secret**, paste the encoded cross-cluster API key. - - 2. Click **Add** to save the API key to the keystore. - 3. Repeat these steps for each API key you want to add. For example, if you want to use several clusters of the remote environment for CCR or CCS. - -7. Add the CA certificate of the remote self-managed environment. -8. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s Security page. -9. Select **Create trust** to complete the configuration. -10. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
- - ::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. - :::: - - -If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ec-edit-remove-trusted-environment.md#ec-edit-remove-trusted-environment-api-key). - -:::: -:::::: - ::::::: You can now connect remotely to the trusted clusters. ## Connect to the remote cluster [ec_connect_to_the_remote_cluster_4] -On the local cluster, add the remote cluster using Kibana or the {{es}} API. +On the local cluster, add the remote cluster using {{kib}} or the {{es}} API. -### Using Kibana [ec_using_kibana_4] +### Using {{kib}} [ec_using_kibana_4] 1. Open the {{kib}} main menu, and select **Stack Management > Data > Remote Clusters > Add a remote cluster**. 2. Enable **Manually enter proxy address and server name**. 3. Fill in the following fields: * **Name**: This *cluster alias* is a unique identifier that represents the connection to the remote cluster and is used to distinguish between local and remote indices. - * **Proxy address**: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote.
+ * **Proxy address**: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote.
- ::::{tip} - If you’re using API keys as security model, change the port into `9443`. - :::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: - * **Server name**: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. + * **Server name**: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. - :::{image} ../../images/cloud-ce-copy-remote-cluster-parameters.png - :alt: Remote Cluster Parameters in Deployment - :class: screenshot - ::: + :::{image} ../../images/cloud-ce-copy-remote-cluster-parameters.png + :alt: Remote Cluster Parameters in Deployment + :class: screenshot + ::: - ::::{note} - If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). - :::: + ::::{note} + If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). + :::: 4. Click **Next**. 5. Click **Add remote cluster** (you have already established trust in a previous step). -### Using the Elasticsearch API [ec_using_the_elasticsearch_api_4] +### Using the {{es}} API [ec_using_the_elasticsearch_api_4] To configure a deployment as a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields: * `mode`: `proxy` -* `proxy_address`: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9400` using a semicolon. +* `proxy_address`: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9400` using a semicolon. -::::{tip} -If you’re using API keys as security model, change the port into `9443`. -:::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: -* `server_name`: This value can be found on the **Security** page of the Elasticsearch Service deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. +* `server_name`: This value can be found on the **Security** page of the {{ech}} deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. This is an example of the API call to `_cluster/settings`: @@ -279,45 +279,10 @@ PUT /_cluster/settings } ``` -:::::{dropdown} **Stack Version above 6.7.0 and below 7.6.0** -::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model. -:::: - - -When the cluster to be configured as a remote is above 6.7.0 and below 7.6.0, the remote cluster must be configured using the [sniff mode](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) with the proxy field. For each remote cluster you need to pass the following fields: - -* **Proxy**: This value can be found on the **Security** page of the deployment you want to use as a remote under the name `Proxy Address`. Also, using the API, this can be obtained from the elasticsearch resource info, concatenating the fields `metadata.endpoint` and `metadata.ports.transport_passthrough` using a semicolon. -* **Seeds**: This field is an array that must contain only one value, which is the `server name` that can be found on the **Security** page of the {{es}} deployment you want to use as a remote concatenated with `:1`. Also, using the API, this can be obtained from the {{es}} resource info, concatenating the fields `metadata.endpoint` and `1` with a semicolon. -* **Mode**: sniff (or empty, since sniff is the default value) - -This is an example of the API call to `_cluster/settings`: - -```json -{ - "persistent": { - "cluster": { - "remote": { - "my-remote-cluster-1": { - "seeds": [ - "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:1" - ], - "proxy": "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:9400" - } - } - } - } -} -``` - -::::: - - - -### Using the Elasticsearch Service RESTful API [ec_using_the_elasticsearch_service_restful_api_4] +### Using the {{ecloud}} RESTful API [ec_using_the_elasticsearch_service_restful_api_4] ::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same organization (for other scenarios,the Elasticsearch API should be used instead): +This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same organization. For other scenarios, the [{{es}} API](#ec_using_the_elasticsearch_api_4) should be used instead. :::: @@ -341,7 +306,7 @@ curl -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC_AP `REF_ID_REMOTE` : The unique ID of the {{es}} resources inside your remote deployment (you can obtain these values through the API). -Note the following when using the Elasticsearch Service RESTful API: +Note the following when using the {{ecloud}} RESTful API: 1. A cluster alias must contain only letters, numbers, dashes (-), or underscores (_). 2. To learn about skipping disconnected clusters, refer to the [{{es}} documentation](/solutions/search/cross-cluster-search.md#skip-unavailable-clusters). @@ -354,11 +319,9 @@ curl -X GET -H "Authorization: ApiKey $EC_API_KEY" https://api.elastic-cloud.com ``` ::::{note} -The response will include just the remote clusters from the same organization in Elasticsearch Service. In order to obtain the whole list of remote clusters, use Kibana or the Elasticsearch API [Elasticsearch API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. +The response will include just the remote clusters from the same {{ecloud}} organization. In order to obtain the whole list of remote clusters, use {{kib}} or the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. :::: - - ## Configure roles and users [ec_configure_roles_and_users_4] -To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). +To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). \ No newline at end of file diff --git a/deploy-manage/remote-clusters/ece-edit-remove-trusted-environment.md b/deploy-manage/remote-clusters/ece-edit-remove-trusted-environment.md index b49a529f34..5fb727793f 100644 --- a/deploy-manage/remote-clusters/ece-edit-remove-trusted-environment.md +++ b/deploy-manage/remote-clusters/ece-edit-remove-trusted-environment.md @@ -1,4 +1,7 @@ --- +applies_to: + deployment: + ece: ga mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-edit-remove-trusted-environment.html --- @@ -12,7 +15,7 @@ From a deployment’s **Security** page, you can manage trusted environments tha * You want to remove or update the access level granted by a cross-cluster API key. -## Remove a trusted environment [ece_remove_a_trusted_environment] +## Remove a certificate-based trusted environment [ece_remove_a_trusted_environment] By removing a trusted environment, this deployment will no longer be able to establish remote connections using certificate trust to clusters of that environment. The remote environment will also no longer be able to connect to this deployment using certificate trust. @@ -25,11 +28,11 @@ With this method, you can only remove trusted environments relying exclusively o 2. In the list of trusted environments, locate the one you want to remove. 3. Remove it using the corresponding `delete` icon. - :::{image} ../../images/cloud-enterprise-delete-trust-environment.png - :alt: button for deleting a trusted environment - ::: + :::{image} ../../images/cloud-enterprise-delete-trust-environment.png + :alt: button for deleting a trusted environment + ::: -4. In Kibana, go to **Stack Management** > **Remote Clusters**. +4. In {{kib}}, go to **Stack Management** > **Remote Clusters**. 5. In the list of existing remote clusters, delete the ones corresponding to the trusted environment you removed earlier. @@ -39,14 +42,14 @@ With this method, you can only remove trusted environments relying exclusively o 2. In the list of trusted environments, locate the one you want to edit. 3. Open its details by selecting the `Edit` icon. - :::{image} ../../images/cloud-enterprise-edit-trust-environment.png - :alt: button for editing a trusted environment - ::: + :::{image} ../../images/cloud-enterprise-edit-trust-environment.png + :alt: button for editing a trusted environment + ::: 4. Edit the trust configuration for that environment: - * From the **Trust level** tab, you can add or remove trusted deployments. - * From the **Environment settings** tab, you can manage the certificates and the label of the environment. + * From the **Trust level** tab, you can add or remove trusted deployments. + * From the **Environment settings** tab, you can manage the certificates and the label of the environment. 5. Save your changes. @@ -56,28 +59,26 @@ With this method, you can only remove trusted environments relying exclusively o This section describes the steps to change the API key used for an existing remote connection. For example, if the previous key expired and you need to rotate it with a new one. ::::{note} -If you need to update the permissions granted by a cross-cluster API key for a remote connection, you only need to update the privileges granted by the API key directly in Kibana. +If you need to update the permissions granted by a cross-cluster API key for a remote connection, you only need to update the privileges granted by the API key directly in {{kib}}. :::: -1. On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key with the appropriate permissions. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. +1. On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key with the appropriate permissions. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. 2. Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next steps. 3. Go to the **Security** page of the local deployment and locate the **Remote connections** section. 4. Locate the API key currently used for connecting to the remote cluster, copy its current alias, and delete it. 5. Add the new API key by selecting **Add an API key**. - * For the **Setting name**, enter the same alias that was used for the previous key. + * For the **Setting name**, enter the same alias that was used for the previous key. - ::::{note} - If you use a different alias, you also need to re-create the remote cluster in Kibana with a **Name** that matches the new alias. - :::: + ::::{note} + If you use a different alias, you also need to re-create the remote cluster in {{kib}} with a **Name** that matches the new alias. + :::: - * For the **Secret**, paste the encoded cross-cluster API key. + * For the **Secret**, paste the encoded cross-cluster API key, then click **Add** to save the API key to the keystore. - 1. Click **Add** to save the API key to the keystore. +6. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
-6. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
- - ::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. - :::: + ::::{note} + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + :::: diff --git a/deploy-manage/remote-clusters/ece-enable-ccs-for-eck.md b/deploy-manage/remote-clusters/ece-enable-ccs-for-eck.md index 70cfae6ecf..5fbc6611ca 100644 --- a/deploy-manage/remote-clusters/ece-enable-ccs-for-eck.md +++ b/deploy-manage/remote-clusters/ece-enable-ccs-for-eck.md @@ -1,11 +1,16 @@ --- +applies_to: + deployment: + ece: ga + eck: ga +navigation_title: With {{eck}} mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-enable-ccs-for-eck.html --- -# Enabling CCS/R between Elastic Cloud Enterprise and ECK [ece-enable-ccs-for-eck] +# Remote clusters between {{ece}} and ECK [ece-enable-ccs-for-eck] -These steps describe how to configure remote clusters between an {{es}} cluster in Elastic Cloud Enterprise and an {{es}} cluster running within [Elastic Cloud on Kubernetes (ECK)](/deploy-manage/deploy/cloud-on-k8s.md). Once that’s done, you’ll be able to [run CCS queries from {{es}}](/solutions/search/cross-cluster-search.md) or [set up CCR](/deploy-manage/tools/cross-cluster-replication/set-up-cross-cluster-replication.md). +These steps describe how to configure remote clusters between an {{es}} cluster in {{ece}} and an {{es}} cluster running within [{{eck}} (ECK)](/deploy-manage/deploy/cloud-on-k8s.md). Once that’s done, you’ll be able to [run CCS queries from {{es}}](/solutions/search/cross-cluster-search.md) or [set up CCR](/deploy-manage/tools/cross-cluster-replication/set-up-cross-cluster-replication.md). ## Establish trust between two clusters [ece_establish_trust_between_two_clusters] @@ -13,7 +18,7 @@ These steps describe how to configure remote clusters between an {{es}} cluster The first step is to establish trust between the two clusters. -### Establish trust in the Elastic Cloud Enterprise cluster [ece_establish_trust_in_the_elastic_cloud_enterprise_cluster] +### Establish trust in the {{ece}} cluster [ece_establish_trust_in_the_elastic_cloud_enterprise_cluster] 1. Save the ECK CA certificate to a file. For a cluster named `quickstart`, run: @@ -22,7 +27,7 @@ The first step is to establish trust between the two clusters. ``` -1. Update the trust settings for the Elastic Cloud Enterprise deployment. Follow the steps provided in [Access clusters of a self-managed environment](ece-remote-cluster-self-managed.md), and specifically the first three steps in **Specify the deployments trusted to be used as remote clusters** using TLS certificate as security model. +1. Update the trust settings for the {{ece}} deployment. Follow the steps provided in [Access clusters of a self-managed environment](ece-remote-cluster-self-managed.md), and specifically the first three steps in **Specify the deployments trusted to be used as remote clusters** using TLS certificate as security model. * Use the certificate file saved in the first step. * Select the {{ecloud}} pattern and enter `default.es.local` for the `Scope ID`. @@ -32,7 +37,7 @@ The first step is to establish trust between the two clusters. ### Establish trust in the ECK cluster [ece_establish_trust_in_the_eck_cluster] -1. Upload the Elastic Cloud Enterprise certificate (that you downloaded in the last step of the previous section) as a Kubernetes secret. +1. Upload the {{ece}} certificate (that you downloaded in the last step of the previous section) as a Kubernetes secret. ```sh kubectl create secret generic ce-aws-cert --from-file= @@ -73,16 +78,16 @@ The first step is to establish trust between the two clusters. -## Setup CCS/R [ece_setup_ccsr] +## Set up CCS/R [ece_setup_ccsr] -Now that trust has been established, you can set up CCS/R from the ECK cluster to the Elastic Cloud Enterprise cluster or from the Elastic Cloud Enterprise cluster to the ECK cluster. +Now that trust has been established, you can set up CCS/R from the ECK cluster to the {{ece}} cluster or from the {{ece}} cluster to the ECK cluster. -### ECK Cluster to Elastic Cloud Enterprise cluster [ece_eck_cluster_to_elastic_cloud_enterprise_cluster] +### ECK Cluster to {{ece}} cluster [ece_eck_cluster_to_elastic_cloud_enterprise_cluster] Configure the ECK cluster [using certificate based authentication](ece-remote-cluster-self-managed.md). -### Elastic Cloud Enterprise cluster to ECK Cluster [ece_elastic_cloud_enterprise_cluster_to_eck_cluster] +### {{ece}} cluster to ECK Cluster [ece_elastic_cloud_enterprise_cluster_to_eck_cluster] Follow the steps outlined in the [ECK documentation](/deploy-manage/remote-clusters/eck-remote-clusters.md#k8s_configure_the_remote_cluster_connection_through_the_elasticsearch_rest_api). diff --git a/deploy-manage/remote-clusters/ece-enable-ccs.md b/deploy-manage/remote-clusters/ece-enable-ccs.md index a11864d98f..eab268c980 100644 --- a/deploy-manage/remote-clusters/ece-enable-ccs.md +++ b/deploy-manage/remote-clusters/ece-enable-ccs.md @@ -1,63 +1,65 @@ --- +applies_to: + deployment: + ece: ga +navigation_title: Elastic Cloud Enterprise mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-enable-ccs.html --- -# Enable cross-cluster search and cross-cluster replication [ece-enable-ccs] +# Remote clusters with {{ece}} [ece-enable-ccs] -[Cross-cluster search (CCS)](/solutions/search/cross-cluster-search.md) allows you to configure multiple remote clusters across different locations and to enable federated search queries across all of the configured remote clusters. +You can configure an {{ece}} deployment to remotely access or (be accessed by) a cluster from: -[Cross-cluster replication (CCR)](/deploy-manage/tools/cross-cluster-replication.md) allows you to replicate indices across multiple remote clusters regardless of where they’re located. This provides tremendous benefit in scenarios of disaster recovery or data locality. - -These remote clusters could be: - -* Another {{es}} cluster of your ECE installation -* An {{es}} cluster in a remote ECE installation -* An {{es}} cluster hosted on {{ecloud}} -* Any other self-managed {{es}} cluster +* Another deployment of your ECE installation +* A deployment running on a different ECE installation +* An {{ech}} deployment +* A deployment running on an {{eck}} installation +* A self-managed installation ## Prerequisites [ece-ccs-ccr-prerequisites] To use CCS or CCR, your environment must meet the following criteria: -* Local and remote clusters must be in compatible versions. Review the [{{es}} version compatibility](/deploy-manage/remote-clusters/remote-clusters-cert.md#remote-clusters-prerequisites-cert) table. - - * System deployments cannot be used as remote clusters or have remote clusters. - +* The local and remote clusters must run on compatible versions of {{es}}. Review the version compatibility table. + + :::{include} _snippets/remote-cluster-certificate-compatibility.md + ::: + * Proxies must answer TCP requests on the port 9400. Check the [prerequisites for the ports that must permit outbound or inbound traffic](../deploy/cloud-enterprise/ece-networking-prereq.md). * Load balancers must pass-through TCP requests on port 9400. Check the [configuration details](../deploy/cloud-enterprise/ece-load-balancers.md). +* If your deployment was created before ECE version `2.9.0`, the Remote clusters page in {{kib}} must be enabled manually from the **Security** page of your deployment, by selecting **Enable CCR** under **Trust management**. + +::::{note} +System deployments cannot be used as remote clusters or have remote clusters. +:::: + +## Set up remote clusters with {{ece}} The steps, information, and authentication method required to configure CCS and CCR can vary depending on where the clusters you want to use as remote are hosted. -* Connect remotely to other clusters from your Elastic Cloud Enterprise deployments +* Connect remotely to other clusters from your {{ece}} deployments - * [Access other deployments of the same Elastic Cloud Enterprise environment](ece-remote-cluster-same-ece.md) - * [Access deployments of a different Elastic Cloud Enterprise environment](ece-remote-cluster-other-ece.md) - * [Access deployments of an {{ess}} environment](ece-remote-cluster-ece-ess.md) + * [Access other deployments of the same {{ece}} environment](ece-remote-cluster-same-ece.md) + * [Access deployments of a different {{ece}} environment](ece-remote-cluster-other-ece.md) + * [Access deployments of an {{ecloud}} environment](ece-remote-cluster-ece-ess.md) * [Access clusters of a self-managed environment](ece-remote-cluster-self-managed.md) * [Access deployments of an ECK environment](ece-enable-ccs-for-eck.md) -* Use clusters from your Elastic Cloud Enterprise deployments as remote +* Use clusters from your {{ece}} deployments as remote - * [From another deployment of the same Elastic Cloud Enterprise environment](ece-remote-cluster-same-ece.md) - * [From a deployment of another Elastic Cloud Enterprise environment](ece-remote-cluster-other-ece.md) - * [From an {{ess}} deployment](/deploy-manage/remote-clusters/ec-remote-cluster-ece.md) + * [From another deployment of the same {{ece}} environment](ece-remote-cluster-same-ece.md) + * [From a deployment of another {{ece}} environment](ece-remote-cluster-other-ece.md) + * [From an {{ech}} deployment](/deploy-manage/remote-clusters/ec-remote-cluster-ece.md) * [From a self-managed cluster](/deploy-manage/remote-clusters/remote-clusters-self-managed.md) - - - -## Enable CCR and the Remote Clusters UI in Kibana [ece-enable-ccr] - -If your deployment was created before ECE version `2.9.0`, CCR won’t be enabled by default and you won’t find the Remote Clusters UI in Kibana even though your deployment meets all the [criteria](#ece-ccs-ccr-prerequisites). - -To enable these features, go to the **Security** page of your deployment and under **Trust management** select **Enable CCR**. + * [From an ECK environment](ece-enable-ccs-for-eck.md) ## Remote clusters and traffic filtering [ece-ccs-ccr-traffic-filtering] ::::{note} -Traffic filtering isn’t supported for cross-cluster operations initiated from an {{ece}} environment to a remote {{ess}} deployment. +Traffic filtering isn’t supported for cross-cluster operations initiated from an {{ece}} environment to a remote {{ech}} deployment. :::: @@ -66,8 +68,8 @@ For remote clusters configured using TLS certificate authentication, [traffic fi Traffic filtering for remote clusters supports 2 methods: * [Filtering by IP addresses and Classless Inter-Domain Routing (CIDR) masks](../security/ip-traffic-filtering.md) -* Filtering by Organization or Elasticsearch cluster ID with a Remote cluster type filter. You can configure this type of filter from the **Platform** > **Security** page of your environment or using the [Elastic Cloud Enterprise API](https://www.elastic.co/docs/api/doc/cloud-enterprise) and apply it from each deployment’s **Security** page. +* Filtering by Organization or {{es}} cluster ID with a Remote cluster type filter. You can configure this type of filter from the **Platform** > **Security** page of your environment or using the [{{ece}} API](https://www.elastic.co/docs/api/doc/cloud-enterprise) and apply it from each deployment’s **Security** page. ::::{note} -When setting up traffic filters for a remote connection to an {{ece}} environment, you also need to upload the region’s TLS certificate of the local cluster to the {{ece}} environment’s proxy. You can find that region’s TLS certificate in the Security page of any deployment of the environment initiating the remote connection. +When setting up traffic filters for a remote connection to an {{ece}} environment, you also need to upload the region’s TLS certificate of the local cluster to the {{ece}} environment’s proxy. You can find that region’s TLS certificate in the **Security** page of any deployment of the environment initiating the remote connection. :::: diff --git a/deploy-manage/remote-clusters/ece-migrate-ccs.md b/deploy-manage/remote-clusters/ece-migrate-ccs.md index 1b788c0a72..96a95775ea 100644 --- a/deploy-manage/remote-clusters/ece-migrate-ccs.md +++ b/deploy-manage/remote-clusters/ece-migrate-ccs.md @@ -1,27 +1,31 @@ --- +applies_to: + deployment: + ece: ga mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-migrate-ccs.html +navigation_title: "Migrate the CCS deployment template" --- # Migrate the cross-cluster search deployment template [ece-migrate-ccs] -The cross-cluster search deployment template is now deprecated was removed in 3.0. You no longer need to use the dedicated cross-cluster template to search across deployments. Instead, you can now use any template to [configure remote clusters](ece-enable-ccs.md) and search across them. Existing deployments created using this template are not affected, but they are required to migrate to another template before upgrading to version 8.x. +The cross-cluster search deployment template is now deprecated was removed in {{ece}} 3.0. You no longer need to use the dedicated cross-cluster template to search across deployments. Instead, you can now use any template to [configure remote clusters](ece-enable-ccs.md) and search across them. Existing deployments created using this template are not affected, but they are required to migrate to another template before upgrading to {{stack}} 8.x. In order to migrate your existing CCS deployment using the CCS Deployment template to the new mechanism which supports CCR and cross-environment remote clusters you will need to migrate your data a new deployment [following these steps](#ece-migrate-ccs-deployment-using-snapshot). ## Use a snapshot to migrate deployments that use the cross-cluster search deployment template [ece-migrate-ccs-deployment-using-snapshot] -You can make this change in the user Cloud UI. The only drawback of this method is that it changes the URL used to access the {{es}} cluster and Kibana. +You can make this change in the user Cloud UI. The only drawback of this method is that it changes the URL used to access the {{es}} cluster and {{kib}}. 1. The first step for any approach is to remove the remote clusters from your deployment. You will need to add them back later. 2. From the deployment menu, open the **Snapshots** page and click **Take Snapshot now**. Wait for the snapshot to finish. 3. From the main **Deployments** page, click **Create deployment**. Next to **Settings** toggle on **Restore snapshot data**, and then select your deployment and the snapshot that you created. - :::{image} ../../images/cloud-enterprise-ce-create-from-snapshot-updated.png - :alt: Create a Deployment using a snapshot - :class: screenshot - ::: + :::{image} ../../images/cloud-enterprise-ce-create-from-snapshot-updated.png + :alt: Create a Deployment using a snapshot + :class: screenshot + ::: 4. Finally, [configure the remote clusters](/deploy-manage/remote-clusters/ece-remote-cluster-other-ece.md). diff --git a/deploy-manage/remote-clusters/ece-remote-cluster-ece-ess.md b/deploy-manage/remote-clusters/ece-remote-cluster-ece-ess.md index ad9d667bd7..149e21b28b 100644 --- a/deploy-manage/remote-clusters/ece-remote-cluster-ece-ess.md +++ b/deploy-manage/remote-clusters/ece-remote-cluster-ece-ess.md @@ -1,11 +1,16 @@ --- +applies_to: + deployment: + ece: ga + ess: ga +navigation_title: With {{ecloud}} mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-remote-cluster-ece-ess.html --- -# Access deployments of an Elasticsearch Service organization [ece-remote-cluster-ece-ess] +# Access deployments of an {{ecloud}} organization [ece-remote-cluster-ece-ess] -This section explains how to configure a deployment to connect remotely to clusters belonging to an {{ess}} organization. +This section explains how to configure a deployment to connect remotely to clusters belonging to an {{ecloud}} organization. ## Allow the remote connection [ece_allow_the_remote_connection_3] @@ -13,31 +18,81 @@ This section explains how to configure a deployment to connect remotely to clust Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps. API key -: For deployments based on {{stack}} version 8.10 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. +: For deployments based on {{stack}} 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. -TLS certificate +TLS certificate (deprecated in {{stack}} 9.0.0) : This model uses mutual TLS authentication for cross-cluster operations. User authentication is performed on the local cluster and a user’s role names are passed to the remote cluster. A superuser on the local deployment gains total read access to the remote deployment, so it is only suitable for deployments that are in the same security domain. :::::::{tab-set} -::::::{tab-item} TLS certificate +::::::{tab-item} API key +API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. + +All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key. + +On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. + +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). + + +### Prerequisites and limitations [ece_prerequisites_and_limitations_3] + +* The local and remote deployments must be on {{stack}} 8.14 or later. + + +### Create a cross-cluster API key on the remote deployment [ece_create_a_cross_cluster_api_key_on_the_remote_deployment_3] + +* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. +* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. + + +### Add the cross-cluster API key to the keystore of the local deployment [ece_add_the_cross_cluster_api_key_to_the_keystore_of_the_local_deployment_2] + +The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. + +1. [Log into the Cloud UI](../deploy/cloud-enterprise/log-into-cloud-ui.md). +2. On the deployments page, select your deployment. + + Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. + +3. From the deployment menu, select **Security**. +4. Locate **Remote connections** and select **Add an API key**. + + 1. Fill both fields. + + * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. + * For the **Secret**, paste the encoded cross-cluster API key. + + 2. Click **Add** to save the API key to the keystore. + +5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
+ + ::::{note} + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + :::: + + +If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ece-edit-remove-trusted-environment.md#ece-edit-remove-trusted-environment-api-key). +:::::: + +::::::{tab-item} TLS certificate (deprecated) ### Configuring trust with clusters in {{ecloud}} [ece-trust-ec] A deployment can be configured to trust all or specific deployments from an organization in [{{ecloud}}](https://www.elastic.co/guide/en/cloud/current): 1. From the **Security** menu, select **Remote Connections > Add trusted environment** and select **{{ecloud}} Organization**. 2. Enter the organization ID (which can be found near the organization name). -3. Upload the Certificate Authorities of the deployments you want to trust. These can be downloaded from the Security page of each deployment (not only the current CA, but also future certificates in case they are expiring soon since they are periodically rotated). Deployments from the same region are signed by the same CA, so you will only need to upload one for each region. +3. Upload the Certificate Authorities of the deployments you want to trust. These can be downloaded from the **Security** page of each deployment (not only the current CA, but also future certificates in case they are expiring soon since they are periodically rotated). Deployments from the same region are signed by the same CA, so you will only need to upload one for each region. 4. Choose one of following options to configure the level of trust with the Organization: * All deployments - This deployment trusts all deployments in the organization in the regions whose certificate authorities have been uploaded, including new deployments when they are created. - * Specific deployments - Specify which of the existing deployments you want to trust from this organization. The full Elasticsearch cluster ID must be entered for each remote cluster. The Elasticsearch `Cluster ID` can be found in the deployment overview page under **Applications**. + * Specific deployments - Specify which of the existing deployments you want to trust from this organization. The full {{es}} cluster ID must be entered for each remote cluster. The {{es}} `Cluster ID` can be found in the deployment overview page under **Applications**. 5. Configure the deployment in {{ecloud}} to [trust this deployment](/deploy-manage/remote-clusters/ec-remote-cluster-ece.md#ec-trust-ece), so that both deployments are configured to trust each other. Note that the organization ID and cluster IDs must be entered fully and correctly. For security reasons, no verification of the IDs is possible. If cross-environment trust does not appear to be working, double-checking the IDs is a good place to start. -::::{dropdown} **Using the API** +::::{dropdown} Using the API You can update a deployment using the appropriate trust settings for the {{es}} payload. In order to trust a deployment with cluster id `cf659f7fe6164d9691b284ae36811be1` (NOTE: use the {{es}} cluster ID, not the deployment ID) in an organization with organization ID `803289842`, you need to update the trust settings with an additional direct trust relationship like this: @@ -73,89 +128,38 @@ In order to trust a deployment with cluster id `cf659f7fe6164d9691b284ae36811be1 :::: :::::: - -::::::{tab-item} API key -API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. - -All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key. - -On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. - -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). - - -### Prerequisites and limitations [ece_prerequisites_and_limitations_3] - -* The local and remote deployments must be on version 8.12 or later. - - -### Create a cross-cluster API key on the remote deployment [ece_create_a_cross_cluster_api_key_on_the_remote_deployment_3] - -* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. -* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. - - -### Add the cross-cluster API key to the keystore of the local deployment [ece_add_the_cross_cluster_api_key_to_the_keystore_of_the_local_deployment_2] - -The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. - -1. [Log into the Cloud UI](../deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the deployments page, select your deployment. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - -3. From the deployment menu, select **Security**. -4. Locate **Remote connections** and select **Add an API key**. - - 1. Fill both fields. - - * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. - * For the **Secret**, paste the encoded cross-cluster API key. - - 2. Click **Add** to save the API key to the keystore. - -5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
- - ::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. - :::: - - -If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ece-edit-remove-trusted-environment.md#ece-edit-remove-trusted-environment-api-key). -:::::: - ::::::: You can now connect remotely to the trusted clusters. ## Connect to the remote cluster [ece_connect_to_the_remote_cluster_3] -On the local cluster, add the remote cluster using Kibana or the {{es}} API. +On the local cluster, add the remote cluster using {{kib}} or the {{es}} API. -### Using Kibana [ece_using_kibana_3] +### Using {{kib}} [ece_using_kibana_3] 1. Open the {{kib}} main menu, and select **Stack Management > Data > Remote Clusters > Add a remote cluster**. 2. Enable **Manually enter proxy address and server name**. 3. Fill in the following fields: * **Name**: This *cluster alias* is a unique identifier that represents the connection to the remote cluster and is used to distinguish between local and remote indices. - * **Proxy address**: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote.
+ * **Proxy address**: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote.
- ::::{tip} - If you’re using API keys as security model, change the port into `9443`. - :::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: - * **Server name**: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. + * **Server name**: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. - :::{image} ../../images/cloud-enterprise-ce-copy-remote-cluster-parameters.png - :alt: Remote Cluster Parameters in Deployment - :class: screenshot - ::: + :::{image} ../../images/cloud-enterprise-ce-copy-remote-cluster-parameters.png + :alt: Remote Cluster Parameters in Deployment + :class: screenshot + ::: - ::::{note} - If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). - :::: + ::::{note} + If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). + :::: 4. Click **Next**. 5. Click **Add remote cluster** (you have already established trust in a previous step). @@ -166,19 +170,19 @@ This configuration of remote clusters uses the [Proxy mode](/deploy-manage/remot -### Using the Elasticsearch API [ece_using_the_elasticsearch_api_3] +### Using the {{es}} API [ece_using_the_elasticsearch_api_3] To configure a deployment as a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields: * `mode`: `proxy` -* `proxy_address`: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9300` using a semicolon. +* `proxy_address`: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9300` using a semicolon. -::::{tip} -If you’re using API keys as security model, change the port into `9443`. -:::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: -* `server_name`: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. +* `server_name`: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. This is an example of the API call to `_cluster/settings`: @@ -199,45 +203,10 @@ PUT /_cluster/settings } ``` -:::::{dropdown} **Stack Version above 6.7.0 and below 7.6.0** -::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model. -:::: - - -When the cluster to be configured as a remote is above 6.7.0 and below 7.6.0, the remote cluster must be configured using the [sniff mode](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) with the proxy field. For each remote cluster you need to pass the following fields: - -* **Proxy**: This value can be found on the **Security** page of the deployment you want to use as a remote under the name `Proxy Address`. Also, using the API, this can be obtained from the elasticsearch resource info, concatenating the fields `metadata.endpoint` and `metadata.ports.transport_passthrough` using a semicolon. -* **Seeds**: This field is an array that must contain only one value, which is the `server name` that can be found on the **Security** page of the ECE deployment you want to use as a remote concatenated with `:1`. Also, using the API, this can be obtained from the {{es}} resource info, concatenating the fields `metadata.endpoint` and `1` with a semicolon. -* **Mode**: sniff (or empty, since sniff is the default value) - -This is an example of the API call to `_cluster/settings`: - -```json -{ - "persistent": { - "cluster": { - "remote": { - "my-remote-cluster-1": { - "seeds": [ - "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:1" - ], - "proxy": "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:9400" - } - } - } - } -} -``` - -::::: - - - -### Using the Elastic Cloud Enterprise RESTful API [ece_using_the_elastic_cloud_enterprise_restful_api_3] +### Using the {{ece}} RESTful API [ece_using_the_elastic_cloud_enterprise_restful_api_3] ::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same ECE environment (for other scenarios, the {{es}} API should be used instead): +This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same ECE environment. For other scenarios, the [{{es}} API](#ece_using_the_elasticsearch_api_3) should be used instead. :::: @@ -261,7 +230,7 @@ curl -k -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC `REF_ID_REMOTE` : The unique ID of the {{es}} resources inside your remote deployment (you can obtain these values through the API). -Note the following when using the Elastic Cloud Enterprise RESTful API: +Note the following when using the {{ece}} RESTful API: 1. A cluster alias must contain only letters, numbers, dashes (-), or underscores (_). 2. To learn about skipping disconnected clusters, refer to the [{{es}} documentation](/solutions/search/cross-cluster-search.md#skip-unavailable-clusters). @@ -274,11 +243,9 @@ curl -k -X GET -H "Authorization: ApiKey $ECE_API_KEY" https://COORDINATOR_HOST: ``` ::::{note} -The response includes just the remote clusters from the same ECE environment. In order to obtain the whole list of remote clusters, use Kibana or the {{es}} API [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. +The response includes just the remote clusters from the same ECE environment. In order to obtain the whole list of remote clusters, use {{kib}} or the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. :::: - - ## Configure roles and users [ece_configure_roles_and_users_3] -To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). +To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). \ No newline at end of file diff --git a/deploy-manage/remote-clusters/ece-remote-cluster-other-ece.md b/deploy-manage/remote-clusters/ece-remote-cluster-other-ece.md index 9dcd76e4b1..b96658fcbd 100644 --- a/deploy-manage/remote-clusters/ece-remote-cluster-other-ece.md +++ b/deploy-manage/remote-clusters/ece-remote-cluster-other-ece.md @@ -1,11 +1,15 @@ --- +applies_to: + deployment: + ece: ga +navigation_title: With a different ECE environment mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-remote-cluster-other-ece.html --- -# Access deployments of another Elastic Cloud Enterprise environment [ece-remote-cluster-other-ece] +# Access deployments of another {{ece}} environment [ece-remote-cluster-other-ece] -This section explains how to configure a deployment to connect remotely to clusters belonging to a different Elastic Cloud Enterprise environment. +This section explains how to configure a deployment to connect remotely to clusters belonging to a different {{ece}} environment. ## Allow the remote connection [ece_allow_the_remote_connection_2] @@ -13,92 +17,13 @@ This section explains how to configure a deployment to connect remotely to clust Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps. API key -: For deployments based on {{stack}} version 8.10 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. +: For deployments based on {{stack}} 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. -TLS certificate +TLS certificate (deprecated in {{stack}} 9.0.0) : This model uses mutual TLS authentication for cross-cluster operations. User authentication is performed on the local cluster and a user’s role names are passed to the remote cluster. A superuser on the local deployment gains total read access to the remote deployment, so it is only suitable for deployments that are in the same security domain. :::::::{tab-set} -::::::{tab-item} TLS certificate -### Configuring platform level trust [ece-trust-remote-environments] - -In order to configure remote clusters in other ECE environments, you first need to establish a bi-directional trust relationship between both ECE environment’s platform: - -1. Download the certificate and copy the environment ID from your first ECE environment under **Platform** > **Trust Management** > **Trust parameters**. -2. Create a new trust relationship in the other ECE environment under **Platform** > **Trust Management** > **Trusted environments** using the certificate and environment ID from the previous step. -3. Download the certificate and copy the environment ID from your second ECE environment and create a new trust relationship with those in the first ECE environment. - -Now, deployments in those environments will be able to configure trust with deployments in the other environment. Trust must always be bi-directional (local cluster must trust remote cluster and vice versa) and it can be configured in each deployment’s security settings. - - -### Configuring trust with clusters of an {{ece}} environment [ece-trust-ece] - -1. Access the **Security** page of the deployment you want to use for cross-cluster operations. -2. Select **Remote Connections > Add trusted environment** and choose **{{ece}}**. Then click **Next**. -3. Select **Certificates** as authentication mechanism and click **Next**. -4. From the dropdown, select one of the environments configured in [Configuring platform level trust](#ece-trust-remote-environments). -5. Choose one of following options to configure the level of trust with the ECE environment: - - * All deployments - This deployment trusts all deployments in the ECE environment, including new deployments when they are created. - * Specific deployments - Specify which of the existing deployments you want to trust in the ECE environment. The full Elasticsearch cluster ID must be entered for each remote cluster. The Elasticsearch `Cluster ID` can be found in the deployment overview page under **Applications**. - -6. Select **Create trust** to complete the configuration. -7. Configure the corresponding deployments of the ECE environment to [trust this deployment](/deploy-manage/remote-clusters/ece-enable-ccs.md). You will only be able to connect 2 deployments successfully when both of them trust each other. - -Note that the environment ID and cluster IDs must be entered fully and correctly. For security reasons, no verification of the IDs is possible. If cross-environment trust does not appear to be working, double-checking the IDs is a good place to start. - -::::{dropdown} **Using the API** -You can update a deployment using the appropriate trust settings for the {{es}} payload. - -Establishing the trust between the two {{ece}} environments can be done using the [trust relationships API](https://www.elastic.co/docs/api/doc/cloud-enterprise/group/endpoint-platformconfigurationtrustrelationships). For example, the list of trusted environments can be obtained calling the [list trust relationships endpoint](https://www.elastic.co/docs/api/doc/cloud-enterprise/group/endpoint-platformconfigurationtrustrelationships): - -```sh -curl -k -X GET -H "Authorization: ApiKey $ECE_API_KEY" https://COORDINATOR_HOST:12443//api/v1/regions/ece-region/platform/configuration/trust-relationships?include_certificate=false -``` - -For each remote ECE environment, it will return something like this: - -```json -{ - "id":"83a7b03f2a4343fe99f09bd27ca3d9ec", - "name":"ECE2", - "trust_by_default":false, - "account_ids":[ - "651598b101e54ccab1bfdcd8b6e3b8be" - ], - "local":false, - "last_modified":"2022-01-9T14:33:20.465Z" -} -``` - -In order to trust a deployment with cluster id `cf659f7fe6164d9691b284ae36811be1` (NOTE: use the {{es}} cluster ID, not the deployment ID) in this environment named `ECE2`, you need to update the trust settings with an external trust relationship like this: - -```json -{ - "trust":{ - "accounts":[ - { - "account_id":"ec38dd0aa45f4a69909ca5c81c27138a", - "trust_all":true - } - ], - "external":[ - { - "trust_relationship_id":"83a7b03f2a4343fe99f09bd27ca3d9ec", - "trust_all":false, - "trust_allowlist":[ - "cf659f7fe6164d9691b284ae36811be1" - ] - } - ] - } -} -``` - -:::: -:::::: - ::::::{tab-item} API key API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. @@ -106,17 +31,16 @@ All cross-cluster requests from the local cluster are bound by the API key’s p On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). ### Prerequisites and limitations [ece_prerequisites_and_limitations_2] -* The local and remote deployments must be on version 8.12 or later. - +* The local and remote deployments must be on {{stack}} 8.14 or later. ### Create a cross-cluster API key on the remote deployment [ece_create_a_cross_cluster_api_key_on_the_remote_deployment_2] -* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. +* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. * Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. @@ -126,9 +50,7 @@ The API key created previously will be used by the local deployment to authentic The steps to follow depend on whether the Certificate Authority (CA) of the remote ECE environment’s proxy or load balancing infrastructure is public or private. -**The CA is public** - -::::{dropdown} +::::{dropdown} The CA is public 1. [Log into the Cloud UI](../deploy/cloud-enterprise/log-into-cloud-ui.md). 2. On the deployments page, select your deployment. @@ -144,10 +66,10 @@ The steps to follow depend on whether the Certificate Authority (CA) of the remo 2. Click **Add** to save the API key to the keystore. -5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
+5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. :::: @@ -156,9 +78,7 @@ If you later need to update the remote connection with different permissions, yo :::: -**The CA is private** - -::::{dropdown} +::::{dropdown} The CA is private 1. [Log into the Cloud UI](../deploy/cloud-enterprise/log-into-cloud-ui.md). 2. On the deployments page, select your deployment. @@ -188,12 +108,12 @@ If you later need to update the remote connection with different permissions, yo :alt: Certificate to copy from the chain ::: -8. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s Security page. +8. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s **Security** page. 9. Select **Create trust** to complete the configuration. -10. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
+10. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. :::: @@ -202,38 +122,116 @@ If you later need to update the remote connection with different permissions, yo :::: :::::: +::::::{tab-item} TLS certificate (deprecated) +### Configuring platform level trust [ece-trust-remote-environments] + +In order to configure remote clusters in other ECE environments, you first need to establish a bi-directional trust relationship between both ECE environment’s platform: + +1. Download the certificate and copy the environment ID from your first ECE environment under **Platform** > **Trust Management** > **Trust parameters**. +2. Create a new trust relationship in the other ECE environment under **Platform** > **Trust Management** > **Trusted environments** using the certificate and environment ID from the previous step. +3. Download the certificate and copy the environment ID from your second ECE environment and create a new trust relationship with those in the first ECE environment. + +Now, deployments in those environments will be able to configure trust with deployments in the other environment. Trust must always be bi-directional (local cluster must trust remote cluster and vice versa) and it can be configured in each deployment’s security settings. + + +### Configuring trust with clusters of an {{ece}} environment [ece-trust-ece] + +1. Access the **Security** page of the deployment you want to use for cross-cluster operations. +2. Select **Remote Connections > Add trusted environment** and choose **{{ece}}**. Then click **Next**. +3. Select **Certificates** as authentication mechanism and click **Next**. +4. From the dropdown, select one of the environments configured in [Configuring platform level trust](#ece-trust-remote-environments). +5. Choose one of following options to configure the level of trust with the ECE environment: + + * All deployments - This deployment trusts all deployments in the ECE environment, including new deployments when they are created. + * Specific deployments - Specify which of the existing deployments you want to trust in the ECE environment. The full {{es}} cluster ID must be entered for each remote cluster. The {{es}} `Cluster ID` can be found in the deployment overview page under **Applications**. + +6. Select **Create trust** to complete the configuration. +7. Configure the corresponding deployments of the ECE environment to [trust this deployment](/deploy-manage/remote-clusters/ece-enable-ccs.md). You will only be able to connect 2 deployments successfully when both of them trust each other. + +Note that the environment ID and cluster IDs must be entered fully and correctly. For security reasons, no verification of the IDs is possible. If cross-environment trust does not appear to be working, double-checking the IDs is a good place to start. + +::::{dropdown} Using the API +You can update a deployment using the appropriate trust settings for the {{es}} payload. + +Establishing the trust between the two {{ece}} environments can be done using the [trust relationships API](https://www.elastic.co/docs/api/doc/cloud-enterprise/group/endpoint-platformconfigurationtrustrelationships). For example, the list of trusted environments can be obtained calling the [list trust relationships endpoint](https://www.elastic.co/docs/api/doc/cloud-enterprise/group/endpoint-platformconfigurationtrustrelationships): + +```sh +curl -k -X GET -H "Authorization: ApiKey $ECE_API_KEY" https://COORDINATOR_HOST:12443//api/v1/regions/ece-region/platform/configuration/trust-relationships?include_certificate=false +``` + +For each remote ECE environment, it will return something like this: + +```json +{ + "id":"83a7b03f2a4343fe99f09bd27ca3d9ec", + "name":"ECE2", + "trust_by_default":false, + "account_ids":[ + "651598b101e54ccab1bfdcd8b6e3b8be" + ], + "local":false, + "last_modified":"2022-01-9T14:33:20.465Z" +} +``` + +In order to trust a deployment with cluster id `cf659f7fe6164d9691b284ae36811be1` (NOTE: use the {{es}} cluster ID, not the deployment ID) in this environment named `ECE2`, you need to update the trust settings with an external trust relationship like this: + +```json +{ + "trust":{ + "accounts":[ + { + "account_id":"ec38dd0aa45f4a69909ca5c81c27138a", + "trust_all":true + } + ], + "external":[ + { + "trust_relationship_id":"83a7b03f2a4343fe99f09bd27ca3d9ec", + "trust_all":false, + "trust_allowlist":[ + "cf659f7fe6164d9691b284ae36811be1" + ] + } + ] + } +} +``` + +:::: +:::::: ::::::: You can now connect remotely to the trusted clusters. ## Connect to the remote cluster [ece_connect_to_the_remote_cluster_2] -On the local cluster, add the remote cluster using Kibana or the {{es}} API. +On the local cluster, add the remote cluster using {{kib}} or the {{es}} API. -### Using Kibana [ece_using_kibana_2] +### Using {{kib}} [ece_using_kibana_2] 1. Open the {{kib}} main menu, and select **Stack Management > Data > Remote Clusters > Add a remote cluster**. 2. Enable **Manually enter proxy address and server name**. 3. Fill in the following fields: * **Name**: This *cluster alias* is a unique identifier that represents the connection to the remote cluster and is used to distinguish between local and remote indices. - * **Proxy address**: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote.
+ * **Proxy address**: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote.
- ::::{tip} - If you’re using API keys as security model, change the port into `9443`. - :::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: - * **Server name**: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. + * **Server name**: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. - :::{image} ../../images/cloud-enterprise-ce-copy-remote-cluster-parameters.png - :alt: Remote Cluster Parameters in Deployment - :class: screenshot - ::: + :::{image} ../../images/cloud-enterprise-ce-copy-remote-cluster-parameters.png + :alt: Remote Cluster Parameters in Deployment + :class: screenshot + ::: - ::::{note} - If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). - :::: + ::::{note} + If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). + :::: 4. Click **Next**. 5. Click **Add remote cluster** (you have already established trust in a previous step). @@ -244,19 +242,19 @@ This configuration of remote clusters uses the [Proxy mode](/deploy-manage/remot -### Using the Elasticsearch API [ece_using_the_elasticsearch_api_2] +### Using the {{es}} API [ece_using_the_elasticsearch_api_2] To configure a deployment as a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields: * `mode`: `proxy` -* `proxy_address`: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9300` using a semicolon. +* `proxy_address`: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9300` using a semicolon. -::::{tip} -If you’re using API keys as security model, change the port into `9443`. -:::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: -* `server_name`: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. +* `server_name`: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. This is an example of the API call to `_cluster/settings`: @@ -277,45 +275,11 @@ PUT /_cluster/settings } ``` -:::::{dropdown} **Stack Version above 6.7.0 and below 7.6.0** -::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model. -:::: - -When the cluster to be configured as a remote is above 6.7.0 and below 7.6.0, the remote cluster must be configured using the [sniff mode](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) with the proxy field. For each remote cluster you need to pass the following fields: - -* **Proxy**: This value can be found on the **Security** page of the deployment you want to use as a remote under the name `Proxy Address`. Also, using the API, this can be obtained from the elasticsearch resource info, concatenating the fields `metadata.endpoint` and `metadata.ports.transport_passthrough` using a semicolon. -* **Seeds**: This field is an array that must contain only one value, which is the `server name` that can be found on the **Security** page of the ECE deployment you want to use as a remote concatenated with `:1`. Also, using the API, this can be obtained from the {{es}} resource info, concatenating the fields `metadata.endpoint` and `1` with a semicolon. -* **Mode**: sniff (or empty, since sniff is the default value) - -This is an example of the API call to `_cluster/settings`: - -```json -{ - "persistent": { - "cluster": { - "remote": { - "my-remote-cluster-1": { - "seeds": [ - "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:1" - ], - "proxy": "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:9400" - } - } - } - } -} -``` - -::::: - - - -### Using the Elastic Cloud Enterprise RESTful API [ece_using_the_elastic_cloud_enterprise_restful_api_2] +### Using the {{ece}} RESTful API [ece_using_the_elastic_cloud_enterprise_restful_api_2] ::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same ECE environment (for other scenarios, the {{es}} API should be used instead): +This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same ECE environment. For other scenarios, the [{{es}} API](#ece_using_the_elasticsearch_api_2) should be used instead. :::: @@ -339,7 +303,7 @@ curl -k -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC `REF_ID_REMOTE` : The unique ID of the {{es}} resources inside your remote deployment (you can obtain these values through the API). -Note the following when using the Elastic Cloud Enterprise RESTful API: +Note the following when using the {{ece}} RESTful API: 1. A cluster alias must contain only letters, numbers, dashes (-), or underscores (_). 2. To learn about skipping disconnected clusters, refer to the [{{es}} documentation](/solutions/search/cross-cluster-search.md#skip-unavailable-clusters). @@ -352,11 +316,9 @@ curl -k -X GET -H "Authorization: ApiKey $ECE_API_KEY" https://COORDINATOR_HOST: ``` ::::{note} -The response includes just the remote clusters from the same ECE environment. In order to obtain the whole list of remote clusters, use Kibana or the {{es}} API [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. +The response includes just the remote clusters from the same ECE environment. In order to obtain the whole list of remote clusters, use {{kib}} or the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. :::: - - ## Configure roles and users [ece_configure_roles_and_users_2] -To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). +To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). \ No newline at end of file diff --git a/deploy-manage/remote-clusters/ece-remote-cluster-same-ece.md b/deploy-manage/remote-clusters/ece-remote-cluster-same-ece.md index 5f5f4f4116..e68977211d 100644 --- a/deploy-manage/remote-clusters/ece-remote-cluster-same-ece.md +++ b/deploy-manage/remote-clusters/ece-remote-cluster-same-ece.md @@ -1,11 +1,15 @@ --- +applies_to: + deployment: + ece: ga +navigation_title: Within the same ECE environment mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-remote-cluster-same-ece.html --- -# Access other deployments of the same Elastic Cloud Enterprise environment [ece-remote-cluster-same-ece] +# Access other deployments of the same {{ece}} environment [ece-remote-cluster-same-ece] -This section explains how to configure a deployment to connect remotely to clusters belonging to the same Elastic Cloud Enterprise environment. +This section explains how to configure a deployment to connect remotely to clusters belonging to the same {{ece}} environment. ## Allow the remote connection [ece_allow_the_remote_connection] @@ -13,17 +17,67 @@ This section explains how to configure a deployment to connect remotely to clust Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps. API key -: For deployments based on {{stack}} version 8.10 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. +: For deployments based on {{stack}} 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. -TLS certificate +TLS certificate (deprecated in {{stack}} 9.0.0) : This model uses mutual TLS authentication for cross-cluster operations. User authentication is performed on the local cluster and a user’s role names are passed to the remote cluster. A superuser on the local deployment gains total read access to the remote deployment, so it is only suitable for deployments that are in the same security domain. :::::::{tab-set} -::::::{tab-item} TLS certificate +::::::{tab-item} API key +API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. + +All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key. + +On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. + +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). + + +### Prerequisites and limitations [ece_prerequisites_and_limitations] + +* The local and remote deployments must be on {{stack}} 8.14 or later. + + +### Create a cross-cluster API key on the remote deployment [ece_create_a_cross_cluster_api_key_on_the_remote_deployment] + +* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. +* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. + + +### Add the cross-cluster API key to the keystore of the local deployment [ece_add_the_cross_cluster_api_key_to_the_keystore_of_the_local_deployment] + +The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. + +1. [Log into the Cloud UI](../deploy/cloud-enterprise/log-into-cloud-ui.md). +2. On the deployments page, select your deployment. + + Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. + +3. From the deployment menu, select **Security**. +4. Locate **Remote connections** and select **Add an API key**. + + 1. Fill both fields. + + * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. + * For the **Secret**, paste the encoded cross-cluster API key. + + 2. Click **Add** to save the API key to the keystore. + +5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
+ + ::::{note} + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + :::: + + +If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ece-edit-remove-trusted-environment.md#ece-edit-remove-trusted-environment-api-key). +:::::: + +::::::{tab-item} TLS certificate (deprecated) ### Default trust with other clusters in the same ECE environment [ece_default_trust_with_other_clusters_in_the_same_ece_environment] -By default, any deployment that you or your users create trusts all other deployments in the same Elastic Cloud Enterprise environment. You can change this behavior in the Cloud UI under **Platform** > **Trust Management**, so that when a new deployment is created it does not automatically trust any other deployment. You can choose one of the following options: +By default, any deployment that you or your users create trusts all other deployments in the same {{ece}} environment. You can change this behavior in the Cloud UI under **Platform** > **Trust Management**, so that when a new deployment is created it does not automatically trust any other deployment. You can choose one of the following options: * Trust all my deployments - All of your organization’s deployments created while this option is selected already trust each other. If you keep this option, that includes any deployments you’ll create in the future. You can directly jump to [Connect to the remote cluster](/deploy-manage/remote-clusters/ece-remote-cluster-same-ece.md#ece_connect_to_the_remote_cluster) to finalize the CCS or CCR configuration. * Trust no deployment - New deployments won’t trust any other deployment when they are created. You can instead configure trust individually for each of them in their security settings, as described in the next section. @@ -35,7 +89,7 @@ By default, any deployment that you or your users create trusts all other deploy ::::{note} * The level of trust of existing deployments is not modified when you change this setting. You must instead update the trust settings individually for each deployment you wish to change. -* Deployments created before Elastic Cloud Enterprise version `2.9.0` trust only themselves. You have to update the trust setting for each deployment that you want to either use as a remote cluster or configure to work with a remote cluster. +* Deployments created before {{ece}} version `2.9.0` trust only themselves. You have to update the trust setting for each deployment that you want to either use as a remote cluster or configure to work with a remote cluster. :::: @@ -51,17 +105,16 @@ If your organization’s deployments already trust each other by default, you ca * Trust all deployments - This deployment trusts all other deployments in this environment, including new deployments when they are created. * Trust specific deployments - Choose which of the existing deployments from your environment you want to trust. - * Trust no deployment - No deployment in this Elastic Cloud Enterprise environment is trusted. - + * Trust no deployment - No deployment in this {{ece}} environment is trusted. -::::{note} -When trusting specific deployments, the more restrictive [CCS](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) version policy is used (even if you only want to use [CCR](/deploy-manage/tools/cross-cluster-replication.md)). To work around this restriction for CCR-only trust, it is necessary to use the API as described below. -:::: + ::::{note} + When trusting specific deployments, the more restrictive [CCS](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) version policy is used (even if you only want to use [CCR](/deploy-manage/tools/cross-cluster-replication.md)). To work around this restriction for CCR-only trust, it is necessary to use the API as described below. + :::: 1. Repeat these steps from each of the deployments you want to use for CCS or CCR. You will only be able to connect 2 deployments successfully when both of them trust each other. -::::{dropdown} **Using the API** +::::{dropdown} Using the API You can update a deployment using the appropriate trust settings for the {{es}} payload. The current trust settings can be found in the path `.resources.elasticsearch[0].info.settings.trust` when calling: @@ -103,89 +156,38 @@ The `account_id` above represents the only account in an ECE environment, and th :::: :::::: - -::::::{tab-item} API key -API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. - -All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key. - -On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. - -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). - - -### Prerequisites and limitations [ece_prerequisites_and_limitations] - -* The local and remote deployments must be on version 8.12 or later. - - -### Create a cross-cluster API key on the remote deployment [ece_create_a_cross_cluster_api_key_on_the_remote_deployment] - -* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. -* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. - - -### Add the cross-cluster API key to the keystore of the local deployment [ece_add_the_cross_cluster_api_key_to_the_keystore_of_the_local_deployment] - -The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. - -1. [Log into the Cloud UI](../deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the deployments page, select your deployment. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - -3. From the deployment menu, select **Security**. -4. Locate **Remote connections** and select **Add an API key**. - - 1. Fill both fields. - - * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. - * For the **Secret**, paste the encoded cross-cluster API key. - - 2. Click **Add** to save the API key to the keystore. - -5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
- - ::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. - :::: - - -If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ece-edit-remove-trusted-environment.md#ece-edit-remove-trusted-environment-api-key). -:::::: - ::::::: You can now connect remotely to the trusted clusters. ## Connect to the remote cluster [ece_connect_to_the_remote_cluster] -On the local cluster, add the remote cluster using Kibana or the {{es}} API. +On the local cluster, add the remote cluster using {{kib}} or the {{es}} API. -### Using Kibana [ece_using_kibana] +### Using {{kib}} [ece_using_kibana] 1. Open the {{kib}} main menu, and select **Stack Management > Data > Remote Clusters > Add a remote cluster**. 2. Enable **Manually enter proxy address and server name**. 3. Fill in the following fields: * **Name**: This *cluster alias* is a unique identifier that represents the connection to the remote cluster and is used to distinguish between local and remote indices. - * **Proxy address**: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote.
+ * **Proxy address**: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote.
- ::::{tip} - If you’re using API keys as security model, change the port into `9443`. - :::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: - * **Server name**: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. + * **Server name**: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. - :::{image} ../../images/cloud-enterprise-ce-copy-remote-cluster-parameters.png - :alt: Remote Cluster Parameters in Deployment - :class: screenshot - ::: + :::{image} ../../images/cloud-enterprise-ce-copy-remote-cluster-parameters.png + :alt: Remote Cluster Parameters in Deployment + :class: screenshot + ::: - ::::{note} - If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). - :::: + ::::{note} + If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). + :::: 4. Click **Next**. 5. Click **Add remote cluster** (you have already established trust in a previous step). @@ -196,19 +198,19 @@ This configuration of remote clusters uses the [Proxy mode](/deploy-manage/remot -### Using the Elasticsearch API [ece_using_the_elasticsearch_api] +### Using the {{es}} API [ece_using_the_elasticsearch_api] To configure a deployment as a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields: * `mode`: `proxy` -* `proxy_address`: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9300` using a semicolon. +* `proxy_address`: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9300` using a semicolon. -::::{tip} -If you’re using API keys as security model, change the port into `9443`. -:::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: -* `server_name`: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. +* `server_name`: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. This is an example of the API call to `_cluster/settings`: @@ -229,45 +231,11 @@ PUT /_cluster/settings } ``` -:::::{dropdown} **Stack Version above 6.7.0 and below 7.6.0** -::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model. -:::: - - -When the cluster to be configured as a remote is above 6.7.0 and below 7.6.0, the remote cluster must be configured using the [sniff mode](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) with the proxy field. For each remote cluster you need to pass the following fields: - -* **Proxy**: This value can be found on the **Security** page of the deployment you want to use as a remote under the name `Proxy Address`. Also, using the API, this can be obtained from the elasticsearch resource info, concatenating the fields `metadata.endpoint` and `metadata.ports.transport_passthrough` using a semicolon. -* **Seeds**: This field is an array that must contain only one value, which is the `server name` that can be found on the **Security** page of the ECE deployment you want to use as a remote concatenated with `:1`. Also, using the API, this can be obtained from the {{es}} resource info, concatenating the fields `metadata.endpoint` and `1` with a semicolon. -* **Mode**: sniff (or empty, since sniff is the default value) - -This is an example of the API call to `_cluster/settings`: - -```json -{ - "persistent": { - "cluster": { - "remote": { - "my-remote-cluster-1": { - "seeds": [ - "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:1" - ], - "proxy": "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:9400" - } - } - } - } -} -``` - -::::: - - -### Using the Elastic Cloud Enterprise RESTful API [ece_using_the_elastic_cloud_enterprise_restful_api] +### Using the {{ece}} RESTful API [ece_using_the_elastic_cloud_enterprise_restful_api] ::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same ECE environment (for other scenarios, the {{es}} API should be used instead): +This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same ECE environment. For other scenarios, the [{{es}} API](#ece_using_the_elasticsearch_api) should be used instead. :::: @@ -291,7 +259,7 @@ curl -k -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC `REF_ID_REMOTE` : The unique ID of the {{es}} resources inside your remote deployment (you can obtain these values through the API). -Note the following when using the Elastic Cloud Enterprise RESTful API: +Note the following when using the {{ece}} RESTful API: 1. A cluster alias must contain only letters, numbers, dashes (-), or underscores (_). 2. To learn about skipping disconnected clusters, refer to the [{{es}} documentation](/solutions/search/cross-cluster-search.md#skip-unavailable-clusters). @@ -304,11 +272,9 @@ curl -k -X GET -H "Authorization: ApiKey $ECE_API_KEY" https://COORDINATOR_HOST: ``` ::::{note} -The response includes just the remote clusters from the same ECE environment. In order to obtain the whole list of remote clusters, use Kibana or the {{es}} API [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. +The response includes just the remote clusters from the same ECE environment. In order to obtain the whole list of remote clusters, use {{kib}} or the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. :::: - - ## Configure roles and users [ece_configure_roles_and_users] To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). diff --git a/deploy-manage/remote-clusters/ece-remote-cluster-self-managed.md b/deploy-manage/remote-clusters/ece-remote-cluster-self-managed.md index 276dbed8e6..7cbedacc15 100644 --- a/deploy-manage/remote-clusters/ece-remote-cluster-self-managed.md +++ b/deploy-manage/remote-clusters/ece-remote-cluster-self-managed.md @@ -1,4 +1,9 @@ --- +applies_to: + deployment: + ece: ga + self: ga +navigation_title: With a self-managed cluster mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-remote-cluster-self-managed.html --- @@ -13,14 +18,103 @@ This section explains how to configure a deployment to connect remotely to self- Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps. API key -: For deployments based on {{stack}} version 8.10 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. +: For deployments based on {{stack}} 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. -TLS certificate +TLS certificate (deprecated in {{stack}} 9.0.0) : This model uses mutual TLS authentication for cross-cluster operations. User authentication is performed on the local cluster and a user’s role names are passed to the remote cluster. A superuser on the local deployment gains total read access to the remote deployment, so it is only suitable for deployments that are in the same security domain. :::::::{tab-set} -::::::{tab-item} TLS certificate +::::::{tab-item} API key +API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. + +All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key. + +On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. + +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). + + +### Prerequisites and limitations [ece_prerequisites_and_limitations_4] + +* The local and remote deployments must be on {{stack}} 8.14 or later. + + +### Create a cross-cluster API key on the remote deployment [ece_create_a_cross_cluster_api_key_on_the_remote_deployment_4] + +* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [{{kib}}](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. +* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. + + +### Configure the local deployment [ece_configure_the_local_deployment_2] + +The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. + +The steps to follow depend on whether the Certificate Authority (CA) of the remote environment’s {{es}} HTTPS server, proxy or, load balancing infrastructure is public or private. + +::::{dropdown} The CA is public +1. [Log into the Cloud UI](../deploy/cloud-enterprise/log-into-cloud-ui.md). +2. On the deployments page, select your deployment. + + Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. + +3. From the deployment menu, select **Security**. +4. Locate **Remote connections** and select **Add an API key**. + + 1. Add a setting: + + * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. + * For the **Secret**, paste the encoded cross-cluster API key. + + 2. Click **Add** to save the API key to the keystore. + +5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
+ + ::::{note} + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + :::: + + +If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ece-edit-remove-trusted-environment.md#ece-edit-remove-trusted-environment-api-key). + +:::: + + +::::{dropdown} The CA is private +1. [Log into the Cloud UI](../deploy/cloud-enterprise/log-into-cloud-ui.md). +2. On the deployments page, select your deployment. + + Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. + +3. Access the **Security** page of the deployment. +4. Select **Remote Connections > Add trusted environment** and choose **Self-managed**. Then click **Next**. +5. Select **API keys** as authentication mechanism and click **Next**. +6. Add a the API key: + + 1. Fill both fields. + + * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. + * For the **Secret**, paste the encoded cross-cluster API key. + + 2. Click **Add** to save the API key to the keystore. + 3. Repeat these steps for each API key you want to add. For example, if you want to use several clusters of the remote environment for CCR or CCS. + +7. Add the CA certificate of the remote self-managed environment. +8. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s **Security** page. +9. Select **Create trust** to complete the configuration. +10. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart {{es}}**.
+ + ::::{note} + If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. + :::: + + +If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ece-edit-remove-trusted-environment.md#ece-edit-remove-trusted-environment-api-key). + +:::: +:::::: + +::::::{tab-item} TLS certificate (deprecated) ### Specify the deployments trusted to be used as remote clusters [ece-trust-self-managed] A deployment can be configured to trust all or specific deployments in any environment: @@ -30,21 +124,21 @@ A deployment can be configured to trust all or specific deployments in any envir 3. Upload the public certificate for the Certificate Authority of the self-managed environment (the one used to sign all the cluster certificates). The certificate needs to be in PEM format and should not contain the private key. If you only have the key in p12 format, then you can create the necessary file like this: `openssl pkcs12 -in elastic-stack-ca.p12 -out newfile.crt.pem -clcerts -nokeys` 4. Select the clusters to trust. There are two options here depending on the subject name of the certificates presented by the nodes in your self managed cluster: - * Following the {{ecloud}} pattern. In {{ecloud}}, the certificates of all Elasticsearch nodes follow this convention: `CN = {{node_id}}.node.{{cluster_id}}.cluster.{{scope_id}}`. If you follow the same convention in your self-managed environment, then choose this option and you will be able to select all or specific clusters to trust. + * Following the {{ecloud}} pattern. In {{ecloud}}, the certificates of all {{es}} nodes follow this convention: `CN = {{node_id}}.node.{{cluster_id}}.cluster.{{scope_id}}`. If you follow the same convention in your self-managed environment, then choose this option and you will be able to select all or specific clusters to trust. * If your clusters don’t follow the previous convention for the certificates subject name of your nodes, you can still specify the node name of each of the nodes that should be trusted by this deployment. (Keep in mind that following this convention will simplify the management of this cluster since otherwise this configuration will need to be updated every time the topology of your self-managed cluster changes along with the trust restriction file. For this reason, it is recommended migrating your cluster certificates to follow the previous convention). ::::{note} - Trust management will not work properly in clusters without an `otherName` value specified, as is the case by default in an out-of-the-box [Elasticsearch installation](../deploy/self-managed/installing-elasticsearch.md). To have the Elasticsearch certutil generate new certificates with the `otherName` attribute, use the file input with the `cn` attribute as in the example below. + Trust management will not work properly in clusters without an `otherName` value specified, as is the case by default in an out-of-the-box [{{es}} installation](../deploy/self-managed/installing-elasticsearch.md). To have the {{es}} certutil generate new certificates with the `otherName` attribute, use the file input with the `cn` attribute as in the example below. :::: -5. . Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s Security page. +5. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s **Security** page. 6. Select **Create trust** to complete the configuration. 7. Configure the self-managed cluster to trust this deployment, so that both deployments are configured to trust each other: - * Download the Certificate Authority used to sign the certificates of your deployment nodes (it can be found in the Security page of your deployment) - * Trust this CA either using the [setting](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/security-settings.md) `xpack.security.transport.ssl.certificate_authorities` in `elasticsearch.yml` or by [adding it to the trust store](../security/different-ca.md). + * Download the Certificate Authority used to sign the certificates of your deployment nodes (it can be found in the Security page of your deployment) + * Trust this CA either using the [setting](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/security-settings.md) `xpack.security.transport.ssl.certificate_authorities` in `elasticsearch.yml` or by [adding it to the trust store](../security/different-ca.md). -8. Generate certificates with an `otherName` attribute using the Elasticsearch certutil. Create a file called `instances.yaml` with all the details of the nodes in your on-premise cluster like below. The `dns` and `ip` settings are optional, but `cn` is mandatory for use with the `trust_restrictions` path setting in the next step. Next, run `./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 -in instances.yaml` to create new certificates for all the nodes at once. You can then copy the resulting files into each node. +8. Generate certificates with an `otherName` attribute using the {{es}} certutil. Create a file called `instances.yaml` with all the details of the nodes in your on-premise cluster like below. The `dns` and `ip` settings are optional, but `cn` is mandatory for use with the `trust_restrictions` path setting in the next step. Next, run `./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 -in instances.yaml` to create new certificates for all the nodes at once. You can then copy the resulting files into each node. ```yaml instances: @@ -60,7 +154,7 @@ A deployment can be configured to trust all or specific deployments in any envir 9. Restrict the trusted clusters to allow only the ones which your self-managed cluster should trust. - * All the clusters in your Elastic Cloud Enterprise environment are signed by the same certificate authority. Therefore, adding this CA would make the self-managed cluster trust all your clusters in your ECE environment. This should be limited using the setting `xpack.security.transport.ssl.trust_restrictions.path` in `elasticsearch.yml`, which points to a file that limits the certificates to trust based on their `otherName`-attribute. + * All the clusters in your {{ece}} environment are signed by the same certificate authority. Therefore, adding this CA would make the self-managed cluster trust all your clusters in your ECE environment. This should be limited using the setting `xpack.security.transport.ssl.trust_restrictions.path` in `elasticsearch.yml`, which points to a file that limits the certificates to trust based on their `otherName`-attribute. * For example, the following file would trust: ``` @@ -80,7 +174,7 @@ Generate new node certificates for an entire cluster using the file input mode o :::: -::::{dropdown} **Using the API** +::::{dropdown} Using the API You can update a deployment using the appropriate trust settings for the {{es}} payload. In order to trust a cluster whose nodes present certificates with the subject names: "CN = node1.example.com", "CN = node2.example.com" and "CN = node3.example.com" in a self-managed environment, you could update the trust settings with an additional direct trust relationship like this: @@ -113,132 +207,38 @@ In order to trust a cluster whose nodes present certificates with the subject na :::: :::::: - -::::::{tab-item} API key -API key authentication enables a local cluster to authenticate itself with a remote cluster via a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). The API key needs to be created by an administrator of the remote cluster. The local cluster is configured to provide this API key on each request to the remote cluster. The remote cluster verifies the API key and grants access, based on the API key’s privileges. - -All cross-cluster requests from the local cluster are bound by the API key’s privileges, regardless of local users associated with the requests. For example, if the API key only allows read access to `my-index` on the remote cluster, even a superuser from the local cluster is limited by this constraint. This mechanism enables the remote cluster’s administrator to have full control over who can access what data with cross-cluster search and/or cross-cluster replication. The remote cluster’s administrator can be confident that no access is possible beyond what is explicitly assigned to the API key. - -On the local cluster side, not every local user needs to access every piece of data allowed by the API key. An administrator of the local cluster can further configure additional permission constraints on local users so each user only gets access to the necessary remote data. Note it is only possible to further reduce the permissions allowed by the API key for individual local users. It is impossible to increase the permissions to go beyond what is allowed by the API key. - -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). - - -### Prerequisites and limitations [ece_prerequisites_and_limitations_4] - -* The local and remote deployments must be on version 8.12 or later. - - -### Create a cross-cluster API key on the remote deployment [ece_create_a_cross_cluster_api_key_on_the_remote_deployment_4] - -* On the deployment you will use as remote, use the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](../api-keys/elasticsearch-api-keys.md) to create a cross-cluster API key. Configure it with access to the indices you want to use for {{ccs}} or {{ccr}}. -* Copy the encoded key (`encoded` in the response) to a safe location. You will need it in the next step. - - -### Configure the local deployment [ece_configure_the_local_deployment_2] - -The API key created previously will be used by the local deployment to authenticate with the corresponding set of permissions to the remote deployment. For that, you need to add the API key to the local deployment’s keystore. - -The steps to follow depend on whether the Certificate Authority (CA) of the remote environment’s Elasticsearch HTTPS server, proxy or, load balancing infrastructure is public or private. - -**The CA is public** - -::::{dropdown} -1. [Log into the Cloud UI](../deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the deployments page, select your deployment. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - -3. From the deployment menu, select **Security**. -4. Locate **Remote connections** and select **Add an API key**. - - 1. Add a setting: - - * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. - * For the **Secret**, paste the encoded cross-cluster API key. - - 2. Click **Add** to save the API key to the keystore. - -5. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
- - ::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. - :::: - - -If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ece-edit-remove-trusted-environment.md#ece-edit-remove-trusted-environment-api-key). - -:::: - - -**The CA is private** - -::::{dropdown} -1. [Log into the Cloud UI](../deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the deployments page, select your deployment. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - -3. Access the **Security** page of the deployment. -4. Select **Remote Connections > Add trusted environment** and choose **Self-managed**. Then click **Next**. -5. Select **API keys** as authentication mechanism and click **Next**. -6. Add a the API key: - - 1. Fill both fields. - - * For the **Setting name**, enter the the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. - * For the **Secret**, paste the encoded cross-cluster API key. - - 2. Click **Add** to save the API key to the keystore. - 3. Repeat these steps for each API key you want to add. For example, if you want to use several clusters of the remote environment for CCR or CCS. - -7. Add the CA certificate of the remote self-managed environment. -8. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment’s Security page. -9. Select **Create trust** to complete the configuration. -10. Restart the local deployment to reload the keystore with its new setting. To do that, go to the deployment’s main page (named after your deployment’s name), locate the **Actions** menu, and select **Restart Elasticsearch**.
- - ::::{note} - If the local deployment runs on version 8.13 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. - :::: - - -If you later need to update the remote connection with different permissions, you can replace the API key as detailed in [Update the access level of a remote cluster connection relying on a cross-cluster API key](ece-edit-remove-trusted-environment.md#ece-edit-remove-trusted-environment-api-key). - -:::: -:::::: - ::::::: You can now connect remotely to the trusted clusters. ## Connect to the remote cluster [ece_connect_to_the_remote_cluster_4] -On the local cluster, add the remote cluster using Kibana or the {{es}} API. +On the local cluster, add the remote cluster using {{kib}} or the {{es}} API. -### Using Kibana [ece_using_kibana_4] +### Using {{kib}} [ece_using_kibana_4] 1. Open the {{kib}} main menu, and select **Stack Management > Data > Remote Clusters > Add a remote cluster**. 2. Enable **Manually enter proxy address and server name**. 3. Fill in the following fields: * **Name**: This *cluster alias* is a unique identifier that represents the connection to the remote cluster and is used to distinguish between local and remote indices. - * **Proxy address**: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote.
+ * **Proxy address**: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote.
- ::::{tip} - If you’re using API keys as security model, change the port into `9443`. - :::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: - * **Server name**: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. + * **Server name**: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. - :::{image} ../../images/cloud-enterprise-ce-copy-remote-cluster-parameters.png - :alt: Remote Cluster Parameters in Deployment - :class: screenshot - ::: + :::{image} ../../images/cloud-enterprise-ce-copy-remote-cluster-parameters.png + :alt: Remote Cluster Parameters in Deployment + :class: screenshot + ::: - ::::{note} - If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). - :::: + ::::{note} + If you’re having issues establishing the connection and the remote cluster is part of an {{ece}} environment with a private certificate, make sure that the proxy address and server name match with the the certificate information. For more information, refer to [Administering endpoints in {{ece}}](/deploy-manage/deploy/cloud-enterprise/change-endpoint-urls.md). + :::: 4. Click **Next**. 5. Click **Add remote cluster** (you have already established trust in a previous step). @@ -249,19 +249,19 @@ This configuration of remote clusters uses the [Proxy mode](/deploy-manage/remot -### Using the Elasticsearch API [ece_using_the_elasticsearch_api_4] +### Using the {{es}} API [ece_using_the_elasticsearch_api_4] To configure a deployment as a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields: * `mode`: `proxy` -* `proxy_address`: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9300` using a semicolon. +* `proxy_address`: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. Also, using the API, this value can be obtained from the {{es}} resource info, concatenating the field `metadata.endpoint` and port `9300` using a semicolon. -::::{tip} -If you’re using API keys as security model, change the port into `9443`. -:::: + ::::{tip} + If you’re using API keys as security model, change the port into `9443`. + :::: -* `server_name`: This value can be found on the **Security** page of the Elastic Cloud Enterprise deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. +* `server_name`: This value can be found on the **Security** page of the {{ece}} deployment you want to use as a remote. Also, using the API, this can be obtained from the {{es}} resource info field `metadata.endpoint`. This is an example of the API call to `_cluster/settings`: @@ -282,45 +282,11 @@ PUT /_cluster/settings } ``` -:::::{dropdown} **Stack Version above 6.7.0 and below 7.6.0** -::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model. -:::: - - -When the cluster to be configured as a remote is above 6.7.0 and below 7.6.0, the remote cluster must be configured using the [sniff mode](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#sniff-mode) with the proxy field. For each remote cluster you need to pass the following fields: -* **Proxy**: This value can be found on the **Security** page of the deployment you want to use as a remote under the name `Proxy Address`. Also, using the API, this can be obtained from the elasticsearch resource info, concatenating the fields `metadata.endpoint` and `metadata.ports.transport_passthrough` using a semicolon. -* **Seeds**: This field is an array that must contain only one value, which is the `server name` that can be found on the **Security** page of the ECE deployment you want to use as a remote concatenated with `:1`. Also, using the API, this can be obtained from the {{es}} resource info, concatenating the fields `metadata.endpoint` and `1` with a semicolon. -* **Mode**: sniff (or empty, since sniff is the default value) - -This is an example of the API call to `_cluster/settings`: - -```json -{ - "persistent": { - "cluster": { - "remote": { - "my-remote-cluster-1": { - "seeds": [ - "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:1" - ], - "proxy": "a542184a7a7d45b88b83f95392f450ab.192.168.44.10.ip.es.io:9400" - } - } - } - } -} -``` - -::::: - - - -### Using the Elastic Cloud Enterprise RESTful API [ece_using_the_elastic_cloud_enterprise_restful_api_4] +### Using the {{ece}} RESTful API [ece_using_the_elastic_cloud_enterprise_restful_api_4] ::::{note} -This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same ECE environment (for other scenarios, the {{es}} API should be used instead): +This section only applies if you’re using TLS certificates as cross-cluster security model and when both clusters belong to the same ECE environment. For other scenarios, the [{{es}} API](#ece_using_the_elasticsearch_api_4) should be used instead. :::: @@ -344,7 +310,7 @@ curl -k -H 'Content-Type: application/json' -X PUT -H "Authorization: ApiKey $EC `REF_ID_REMOTE` : The unique ID of the {{es}} resources inside your remote deployment (you can obtain these values through the API). -Note the following when using the Elastic Cloud Enterprise RESTful API: +Note the following when using the {{ece}} RESTful API: 1. A cluster alias must contain only letters, numbers, dashes (-), or underscores (_). 2. To learn about skipping disconnected clusters, refer to the [{{es}} documentation](/solutions/search/cross-cluster-search.md#skip-unavailable-clusters). @@ -357,11 +323,9 @@ curl -k -X GET -H "Authorization: ApiKey $ECE_API_KEY" https://COORDINATOR_HOST: ``` ::::{note} -The response includes just the remote clusters from the same ECE environment. In order to obtain the whole list of remote clusters, use Kibana or the {{es}} API [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. +The response includes just the remote clusters from the same ECE environment. In order to obtain the whole list of remote clusters, use {{kib}} or the [{{es}} API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) directly. :::: - - ## Configure roles and users [ece_configure_roles_and_users_4] -To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). +To use a remote cluster for {{ccr}} or {{ccs}}, you need to create user roles with [remote indices privileges](../users-roles/cluster-or-deployment-auth/defining-roles.md#roles-remote-indices-priv) on the local cluster. Refer to [Configure roles and users](remote-clusters-api-key.md#remote-clusters-privileges-api-key). \ No newline at end of file diff --git a/deploy-manage/remote-clusters/eck-remote-clusters.md b/deploy-manage/remote-clusters/eck-remote-clusters.md index 33d951e7c4..10c4e0f989 100644 --- a/deploy-manage/remote-clusters/eck-remote-clusters.md +++ b/deploy-manage/remote-clusters/eck-remote-clusters.md @@ -1,26 +1,36 @@ --- +applies_to: + deployment: + eck: ga +navigation_title: Elastic Cloud on Kubernetes mapped_pages: - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-remote-clusters.html --- -# ECK remote clusters [k8s-remote-clusters] +# Remote clusters with {{eck}} [k8s-remote-clusters] -The [remote clusters module](/deploy-manage/remote-clusters/remote-clusters-self-managed.md) in Elasticsearch enables you to establish uni-directional connections to a remote cluster. This functionality is used in cross-cluster replication and cross-cluster search. +The [remote clusters module](/deploy-manage/remote-clusters.md) in Elasticsearch enables you to establish uni-directional connections to a remote cluster. This functionality is used in cross-cluster replication and cross-cluster search. When using remote cluster connections with ECK, the setup process depends on where the remote cluster is deployed. -## Connect from an Elasticsearch cluster running in the same Kubernetes cluster [k8s-remote-clusters-connect-internal] +## Connect from an {{es}} cluster running in the same Kubernetes cluster [k8s-remote-clusters-connect-internal] ::::{note} The remote clusters feature requires a valid Enterprise license or Enterprise trial license. Check [the license documentation](../license/manage-your-license-in-eck.md) for more details about managing licenses. :::: -To create a remote cluster connection to another Elasticsearch cluster deployed within the same Kubernetes cluster, specify the `remoteClusters` attribute in your Elasticsearch spec. +To create a remote cluster connection to another {{es}} cluster deployed within the same Kubernetes cluster, specify the `remoteClusters` attribute in your {{es}} spec. -### Security Models [k8s_security_models] +### Security models [k8s_security_models] -ECK supports two different security models: the API key based security model, and the certificate security model. These two security models are described in the [Remote clusters](/deploy-manage/remote-clusters/remote-clusters-self-managed.md#remote-clusters-security-models) section of the {{es}} documentation. +Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps. + +API key +: For deployments based on {{stack}} 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote deployment fine-grained access controls. + +TLS certificate (deprecated in {{stack}} 9.0.0) +: This model uses mutual TLS authentication for cross-cluster operations. User authentication is performed on the local cluster and a user’s role names are passed to the remote cluster. A superuser on the local deployment gains total read access to the remote deployment, so it is only suitable for deployments that are in the same security domain. ### Using the API key security model [k8s_using_the_api_key_security_model] @@ -29,7 +39,7 @@ To enable the API key security model you must first enable the remote cluster se ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 -kind: Elasticsearch +kind: {{es}} metadata: name: cluster-two namespace: ns-two @@ -47,13 +57,13 @@ Enabling the remote cluster server triggers a restart of the {{es}} cluster. :::: -Once the remote cluster server is enabled and started on the remote cluster you can configure the Elasticsearch reference on the local cluster to include the desired permissions for cross-cluster search, and cross-cluster replication. +Once the remote cluster server is enabled and started on the remote cluster you can configure the {{es}} reference on the local cluster to include the desired permissions for cross-cluster search, and cross-cluster replication. -Permissions have to be included under the `apiKey` field. The API model of the Elasticsearch resource is compatible with the [{{es}} Cross-Cluster API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) model. Fine-grained permissions can therefore be configured in both the `search` and `replication` fields: +Permissions have to be included under the `apiKey` field. The API model of the {{es}} resource is compatible with the [{{es}} Cross-Cluster API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) model. Fine-grained permissions can therefore be configured in both the `search` and `replication` fields: ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 -kind: Elasticsearch +kind: {{es}} metadata: name: cluster-one namespace: ns-one @@ -89,7 +99,7 @@ The following example describes how to configure `cluster-two` as a remote clust ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 -kind: Elasticsearch +kind: {{es}} metadata: name: cluster-one namespace: ns-one @@ -110,17 +120,17 @@ spec: -## Connect from an Elasticsearch cluster running outside the Kubernetes cluster [k8s-remote-clusters-connect-external] +## Connect from an {{es}} cluster running outside the Kubernetes cluster [k8s-remote-clusters-connect-external] ::::{note} -While it is technically possible to configure remote cluster connections using older versions of Elasticsearch, this guide only covers the setup for Elasticsearch 7.6 and later. The setup process is significantly simplified in Elasticsearch 7.6 due to improved support for the indirection of Kubernetes services. +While it is technically possible to configure remote cluster connections using older versions of {{es}}, this guide only covers the setup for {{es}} 7.6 and later. The setup process is significantly simplified in {{es}} 7.6 due to improved support for the indirection of Kubernetes services. :::: -You can configure a remote cluster connection to an ECK-managed Elasticsearch cluster from another cluster running outside the Kubernetes cluster as follows: +You can configure a remote cluster connection to an ECK-managed {{es}} cluster from another cluster running outside the Kubernetes cluster as follows: 1. Make sure that both clusters trust each other’s certificate authority. -2. Configure the remote cluster connection through the Elasticsearch REST API. +2. Configure the remote cluster connection through the {{es}} REST API. Consider the following example: @@ -131,7 +141,7 @@ To configure `cluster-one` as a remote cluster in `cluster-two`: ### Make sure both clusters trust each other’s certificate authority [k8s_make_sure_both_clusters_trust_each_others_certificate_authority] -The certificate authority (CA) used by ECK to issue certificates for the Elasticsearch transport layer is stored in a secret named `-es-transport-certs-public`. Extract the certificate for `cluster-one` as follows: +The certificate authority (CA) used by ECK to issue certificates for the {{es}} transport layer is stored in a secret named `-es-transport-certs-public`. Extract the certificate for `cluster-one` as follows: ```sh kubectl get secret cluster-one-es-transport-certs-public \ @@ -162,7 +172,7 @@ If `cluster-two` is also managed by an ECK instance, proceed as follows: ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 - kind: Elasticsearch + kind: {{es}} metadata: name: cluster-two spec: @@ -179,13 +189,13 @@ If `cluster-two` is also managed by an ECK instance, proceed as follows: 3. Repeat steps 1 and 2 to add the CA of `cluster-two` to `cluster-one` as well. -### Configure the remote cluster connection through the Elasticsearch REST API [k8s_configure_the_remote_cluster_connection_through_the_elasticsearch_rest_api] +### Configure the remote cluster connection through the {{es}} REST API [k8s_configure_the_remote_cluster_connection_through_the_elasticsearch_rest_api] Expose the transport layer of `cluster-one`. ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 -kind: Elasticsearch +kind: {{es}} metadata: name: cluster-one spec: @@ -198,7 +208,7 @@ spec: 1. On cloud providers which support external load balancers, setting the type field to LoadBalancer provisions a load balancer for your Service. Alternatively, expose the service through one of the Kubernetes Ingress controllers that support TCP services. -Finally, configure `cluster-one` as a remote cluster in `cluster-two` using the Elasticsearch REST API: +Finally, configure `cluster-one` as a remote cluster in `cluster-two` using the {{es}} REST API: ```sh PUT _cluster/settings diff --git a/deploy-manage/remote-clusters/remote-clusters-api-key.md b/deploy-manage/remote-clusters/remote-clusters-api-key.md index b56ec50eac..63e2436733 100644 --- a/deploy-manage/remote-clusters/remote-clusters-api-key.md +++ b/deploy-manage/remote-clusters/remote-clusters-api-key.md @@ -1,4 +1,7 @@ --- +applies_to: + deployment: + self: ga mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/remote-clusters-api-key.html --- @@ -22,19 +25,19 @@ To add a remote cluster using API key authentication: 3. [Connect to a remote cluster](#remote-clusters-connect-api-key) 4. [Configure roles and users](#remote-clusters-privileges-api-key) -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). ## Prerequisites [remote-clusters-prerequisites-api-key] * The {{es}} security features need to be enabled on both clusters, on every node. Security is enabled by default. If it’s disabled, set `xpack.security.enabled` to `true` in `elasticsearch.yml`. Refer to [General security settings](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/security-settings.md#general-security-settings). -* The nodes of the local and remote clusters must be on version 8.10 or later. +* The nodes of the local and remote clusters must be on {{stack}} 8.14 or later. * The local and remote clusters must have an appropriate license. For more information, refer to [https://www.elastic.co/subscriptions](https://www.elastic.co/subscriptions). ## Establish trust with a remote cluster [remote-clusters-security-api-key] ::::{note} -If a remote cluster is part of an {{ess}} deployment, it has a valid certificate by default. You can therefore skip steps related to certificates in these instructions. +If a remote cluster is part of an {{ech}} deployment, it has a valid certificate by default. You can therefore skip steps related to certificates in these instructions. :::: @@ -99,7 +102,7 @@ If a remote cluster is part of an {{ess}} deployment, it has a valid certificate When prompted, enter the `CERT_PASSWORD` from the earlier step. 4. Restart the remote cluster. -5. On the remote cluster, generate a cross-cluster API key that provides access to the indices you want to use for {{ccs}} or {{ccr}}. You can use the [Create Cross-Cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) API or [Kibana](../api-keys/elasticsearch-api-keys.md). +5. On the remote cluster, generate a cross-cluster API key that provides access to the indices you want to use for {{ccs}} or {{ccr}}. You can use the [Create Cross-Cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) API or [{{kib}}](../api-keys/elasticsearch-api-keys.md). 6. Copy the encoded key (`encoded` in the response) to a safe location. You will need it to connect to the remote cluster later. diff --git a/deploy-manage/remote-clusters/remote-clusters-cert.md b/deploy-manage/remote-clusters/remote-clusters-cert.md index 14e428d069..54514c3cf8 100644 --- a/deploy-manage/remote-clusters/remote-clusters-cert.md +++ b/deploy-manage/remote-clusters/remote-clusters-cert.md @@ -1,4 +1,7 @@ --- +applies_to: + deployment: + self: ga mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/remote-clusters-cert.html --- @@ -19,45 +22,15 @@ To add a remote cluster using TLS certificate authentication: 3. [Connect to a remote cluster](#remote-clusters-connect-cert) 4. [Configure roles and users for remote clusters](#remote-clusters-privileges-cert) -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). ## Prerequisites [remote-clusters-prerequisites-cert] 1. The {{es}} security features need to be enabled on both clusters, on every node. Security is enabled by default. If it’s disabled, set `xpack.security.enabled` to `true` in `elasticsearch.yml`. Refer to [General security settings](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/security-settings.md#general-security-settings). 2. The local and remote clusters versions must be compatible. - * Any node can communicate with another node on the same major version. For example, 7.0 can talk to any 7.x node. - * Only nodes on the last minor version of a certain major version can communicate with nodes on the following major version. In the 6.x series, 6.8 can communicate with any 7.x node, while 6.7 can only communicate with 7.0. - * Version compatibility is symmetric, meaning that if 6.7 can communicate with 7.0, 7.0 can also communicate with 6.7. The following table depicts version compatibility between local and remote nodes. - - :::::{dropdown} Version compatibility table - | | | - | --- | --- | - | | Local cluster | - | Remote cluster | 5.0–5.5 | 5.6 | 6.0–6.6 | 6.7 | 6.8 | 7.0 | 7.1–7.16 | 7.17 | 8.0–9.0 | - | 5.0–5.5 | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | - | 5.6 | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | - | 6.0–6.6 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | - | 6.7 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | - | 6.8 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | - | 7.0 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | - | 7.1–7.16 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | - | 7.17 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | - | 8.0–9.0 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | - - ::::{note} - This documentation is for {{es}} version 9.0.0-beta1, which is not yet released. The above compatibility table applies if both clusters are running a released version of {{es}}, or if one of the clusters is running a released version and the other is running a pre-release build with a later build date. A cluster running a pre-release build of {{es}} can also communicate with remote clusters running the same pre-release build. Running a mix of pre-release builds is unsupported and typically will not work, even if the builds have the same version number. - :::: - - - ::::: - - - ::::{important} - Elastic only supports {{ccs}} on a subset of these configurations. See [Supported {{ccs}} configurations](../../solutions/search/cross-cluster-search.md#ccs-supported-configurations). - :::: - - + :::{include} _snippets/remote-cluster-certificate-compatibility.md + ::: ## Establish trust with a remote cluster [remote-clusters-security-cert] @@ -280,7 +253,7 @@ The following requests use the [create or update roles API](https://www.elastic. The {{ccr}} user requires different cluster and index privileges on the remote cluster and local cluster. Use the following requests to create separate roles on the local and remote clusters, and then create a user with the required roles. -##### Remote cluster [_remote_cluster] +#### Remote cluster [_remote_cluster] On the remote cluster that contains the leader index, the {{ccr}} role requires the `read_ccr` cluster privilege, and `monitor` and `read` privileges on the leader index. @@ -317,7 +290,7 @@ POST /_security/role/remote-replication ``` -##### Local cluster [_local_cluster] +#### Local cluster [_local_cluster] On the local cluster that contains the follower index, the `remote-replication` role requires the `manage_ccr` cluster privilege, and `monitor`, `read`, `write`, and `manage_follow_index` privileges on the follower index. @@ -368,7 +341,7 @@ You can then [configure {{ccr}}](../tools/cross-cluster-replication/set-up-cross The {{ccs}} user requires different cluster and index privileges on the remote cluster and local cluster. The following requests create separate roles on the local and remote clusters, and then create a user with the required roles. -##### Remote cluster [_remote_cluster_2] +#### Remote cluster [_remote_cluster_2] On the remote cluster, the {{ccs}} role requires the `read` and `read_cross_cluster` privileges for the target indices. @@ -402,7 +375,7 @@ POST /_security/role/remote-search ``` -##### Local cluster [_local_cluster_2] +#### Local cluster [_local_cluster_2] On the local cluster, which is the cluster used to initiate cross cluster search, a user only needs the `remote-search` role. The role privileges can be empty. @@ -445,7 +418,7 @@ To grant users read access on the remote data streams and indices, you must crea For example, you might be actively indexing {{ls}} data on a local cluster and and periodically offload older time-based indices to an archive on your remote cluster. You want to search across both clusters, so you must enable {{kib}} users on both clusters. -##### Local cluster [_local_cluster_3] +#### Local cluster [_local_cluster_3] On the local cluster, create a `logstash-reader` role that grants `read` and `view_index_metadata` privileges on the local `logstash-*` indices. @@ -485,7 +458,7 @@ PUT /_security/user/cross-cluster-kibana ``` -##### Remote cluster [_remote_cluster_3] +#### Remote cluster [_remote_cluster_3] On the remote cluster, create a `logstash-reader` role that grants the `read_cross_cluster` privilege and `read` and `view_index_metadata` privileges for the `logstash-*` indices. diff --git a/deploy-manage/remote-clusters/remote-clusters-migrate.md b/deploy-manage/remote-clusters/remote-clusters-migrate.md index a666ca6a53..fc36820dfc 100644 --- a/deploy-manage/remote-clusters/remote-clusters-migrate.md +++ b/deploy-manage/remote-clusters/remote-clusters-migrate.md @@ -1,4 +1,7 @@ --- +applies_to: + deployment: + self: ga navigation_title: "Migrate from certificate to API key authentication" mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/remote-clusters-migrate.html @@ -25,11 +28,11 @@ For these reasons, you may prefer to migrate a remote cluster in-place, by follo 5. [Resume cross-cluster operations](#remote-clusters-migration-resume) 6. [Disable certificate based authentication and authorization](#remote-clusters-migration-disable-cert) -If you run into any issues, refer to [Troubleshooting](remote-clusters-troubleshooting.md). +If you run into any issues, refer to [Troubleshooting](/troubleshoot/elasticsearch/remote-clusters.md). ## Prerequisites [remote-clusters-migration-prerequisites] -* The nodes of the local and remote clusters must be on version 8.10 or later. +* The nodes of the local and remote clusters must be on {{stack}} 8.14 or later. * The local and remote clusters must have an appropriate license. For more information, refer to [https://www.elastic.co/subscriptions](https://www.elastic.co/subscriptions). @@ -96,7 +99,7 @@ On the remote cluster: When prompted, enter the `CERT_PASSWORD` from the earlier step. 4. Restart the remote cluster. -5. On the remote cluster, generate a cross-cluster API key that provides access to the indices you want to use for {{ccs}} or {{ccr}}. You can use the [Create Cross-Cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) API or [Kibana](../api-keys/elasticsearch-api-keys.md). +5. On the remote cluster, generate a cross-cluster API key that provides access to the indices you want to use for {{ccs}} or {{ccr}}. You can use the [Create Cross-Cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) API or [{{kib}}](../api-keys/elasticsearch-api-keys.md). 6. Copy the encoded key (`encoded` in the response) to a safe location. You will need it to connect to the remote cluster later. @@ -222,7 +225,7 @@ Resume any persistent tasks that you stopped earlier. Tasks should be restarted ## Disable certificate based authentication and authorization [remote-clusters-migration-disable-cert] ::::{note} -Only proceed with this step if the migration has been proved successful on the local cluster. If the migration is unsuccessful, either [find out what the problem is and attempt to fix it](remote-clusters-troubleshooting.md) or [roll back](#remote-clusters-migration-rollback). +Only proceed with this step if the migration has been proved successful on the local cluster. If the migration is unsuccessful, either [find out what the problem is and attempt to fix it](/troubleshoot/elasticsearch/remote-clusters.md) or [roll back](#remote-clusters-migration-rollback). :::: diff --git a/deploy-manage/remote-clusters/remote-clusters-self-managed.md b/deploy-manage/remote-clusters/remote-clusters-self-managed.md index 0e952b2d0c..624fb4e0c5 100644 --- a/deploy-manage/remote-clusters/remote-clusters-self-managed.md +++ b/deploy-manage/remote-clusters/remote-clusters-self-managed.md @@ -1,48 +1,30 @@ --- +applies_to: + deployment: + self: ga +navigation_title: Self-managed {{stack}} mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/remote-clusters.html --- -# Remote clusters (self-managed) [remote-clusters] +# Remote clusters with self-managed installations [remote-clusters] -You can connect a local cluster to other {{es}} clusters, known as *remote clusters*. Remote clusters can be located in different datacenters or geographic regions, and contain indices or data streams that can be replicated with {{ccr}} or searched by a local cluster using {{ccs}}. - - -## {{ccr-cap}} [remote-clusters-ccr] - -With [{{ccr}}](/deploy-manage/tools/cross-cluster-replication.md), you ingest data to an index on a remote cluster. This *leader* index is replicated to one or more read-only *follower* indices on your local cluster. Creating a multi-cluster architecture with {{ccr}} enables you to configure disaster recovery, bring data closer to your users, or establish a centralized reporting cluster to process reports locally. - - -## {{ccs-cap}} [remote-clusters-ccs] - -[{{ccs-cap}}](/solutions/search/cross-cluster-search.md) enables you to run a search request against one or more remote clusters. This capability provides each region with a global view of all clusters, allowing you to send a search request from a local cluster and return results from all connected remote clusters. For full {{ccs}} capabilities, the local and remote cluster must be on the same [subscription level](https://www.elastic.co/subscriptions). +The instructions that follow describe how to create a remote connection from a self-managed cluster. You can also set up {{ccs}} and {{ccr}} from an [{{ech}} deployment](/deploy-manage/remote-clusters/ec-enable-ccs.md) or from an [{{ece}} deployment](/deploy-manage/remote-clusters/ece-enable-ccs.md). ## Add remote clusters [add-remote-clusters] -::::{note} -The instructions that follow describe how to create a remote connection from a self-managed cluster. You can also set up {{ccs}} and {{ccr}} from an [{{ess}} deployment](/deploy-manage/remote-clusters/ec-enable-ccs.md) or from an [{{ece}} deployment](/deploy-manage/remote-clusters/ece-enable-ccs.md). -:::: - - To add remote clusters, you can choose between [two security models](#remote-clusters-security-models) and [two connection modes](#sniff-proxy-modes). Both security models are compatible with either of the connection modes. ### Security models [remote-clusters-security-models] -API key based security model -: For clusters on version 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote cluster fine-grained access controls. [Add remote clusters using API key authentication](remote-clusters-api-key.md). +API key +: For clusters on {{stack}} 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model offers administrators of both the local and the remote cluster fine-grained access controls. [Add remote clusters using API key authentication](remote-clusters-api-key.md). -Certificate based security model +TLS certificate (deprecated in {{stack}} 9.0.0) : Uses mutual TLS authentication for cross-cluster operations. User authentication is performed on the local cluster and a user’s role names are passed to the remote cluster. In this model, a superuser on the local cluster gains total read access to the remote cluster, so it is only suitable for clusters that are in the same security domain. [Add remote clusters using TLS certificate authentication](remote-clusters-cert.md). - ::::{admonition} Deprecated in 9.0.0. - :class: warning - - Use [API key based security model](remote-clusters-api-key.md) instead. - :::: - - ### Connection modes [sniff-proxy-modes] diff --git a/deploy-manage/remote-clusters/remote-clusters-settings.md b/deploy-manage/remote-clusters/remote-clusters-settings.md index d733f90f8f..acf9c3610b 100644 --- a/deploy-manage/remote-clusters/remote-clusters-settings.md +++ b/deploy-manage/remote-clusters/remote-clusters-settings.md @@ -1,4 +1,7 @@ --- +applies_to: + deployment: + self: ga mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/remote-clusters-settings.html --- @@ -20,7 +23,7 @@ The following settings apply to both [sniff mode](/deploy-manage/remote-clusters : Per cluster boolean setting that allows to skip specific clusters when no nodes belonging to them are available and they are the target of a remote cluster request. ::::{important} -In Elasticsearch 8.15, the default value for `skip_unavailable` was changed from `false` to `true`. Before Elasticsearch 8.15, if you want a cluster to be treated as optional for a {{ccs}}, then you need to set that configuration. From Elasticsearch 8.15 forward, you need to set the configuration in order to make a cluster required for the {{ccs}}. Once you upgrade the local ("querying") cluster search coordinator node (the node you send CCS requests to) to 8.15 or later, any remote clusters that do not have an explicit setting for `skip_unavailable` will immediately change over to using the new default of true. This is true regardless of whether you have upgraded the remote clusters to 8.15, as the `skip_unavailable` search behavior is entirely determined by the setting on the local cluster where you configure the remotes. +In {{es}} 8.15, the default value for `skip_unavailable` was changed from `false` to `true`. Before {{es}} 8.15, if you want a cluster to be treated as optional for a {{ccs}}, then you need to set that configuration. From {{es}} 8.15 forward, you need to set the configuration in order to make a cluster required for the {{ccs}}. Once you upgrade the local ("querying") cluster search coordinator node (the node you send CCS requests to) to 8.15 or later, any remote clusters that do not have an explicit setting for `skip_unavailable` will immediately change over to using the new default of true. This is true regardless of whether you have upgraded the remote clusters to 8.15, as the `skip_unavailable` search behavior is entirely determined by the setting on the local cluster where you configure the remotes. :::: diff --git a/deploy-manage/remote-clusters/remote-clusters-troubleshooting.md b/deploy-manage/remote-clusters/remote-clusters-troubleshooting.md deleted file mode 100644 index d5f068b7f2..0000000000 --- a/deploy-manage/remote-clusters/remote-clusters-troubleshooting.md +++ /dev/null @@ -1,406 +0,0 @@ ---- -navigation_title: "Troubleshooting" -mapped_pages: - - https://www.elastic.co/guide/en/elasticsearch/reference/current/remote-clusters-troubleshooting.html ---- - - - -# Troubleshooting [remote-clusters-troubleshooting] - - -You may encounter several issues when setting up a remote cluster for {{ccr}} or {{ccs}}. - -## General troubleshooting [remote-clusters-troubleshooting-general] - -### Checking whether a remote cluster has connected successfully [remote-clusters-troubleshooting-check-connection] - -A successful call to the cluster settings update API for adding or updating remote clusters does not necessarily mean the configuration is successful. Use the [remote cluster info API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) to verify that a local cluster is successfully connected to a remote cluster. - -```console -GET /_remote/info -``` - -The API should return `"connected" : true`. When using [API key authentication](remote-clusters-api-key.md), it should also return `"cluster_credentials": "::es_redacted::"`. - -```console-result -{ - "cluster_one" : { - "seeds" : [ - "127.0.0.1:9443" - ], - "connected" : true, <1> - "num_nodes_connected" : 1, - "max_connections_per_cluster" : 3, - "initial_connect_timeout" : "30s", - "skip_unavailable" : false, - "cluster_credentials": "::es_redacted::", <2> - "mode" : "sniff" - } -} -``` - -1. The remote cluster has connected successfully. -2. If present, indicates the remote cluster has connected using [API key authentication](remote-clusters-api-key.md) instead of [certificate based authentication](remote-clusters-cert.md). - - - -### Enabling the remote cluster server [remote-clusters-troubleshooting-enable-server] - -When using API key authentication, cross-cluster traffic happens on the remote cluster interface, instead of the transport interface. The remote cluster interface is not enabled by default. This means a node is not ready to accept incoming cross-cluster requests by default, while it is ready to send outgoing cross-cluster requests. Ensure you’ve enabled the remote cluster server on every node of the remote cluster. In `elasticsearch.yml`: - -* Set [`remote_cluster_server.enabled`](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/networking-settings.md#remote-cluster-network-settings) to `true`. -* Configure the bind and publish address for remote cluster server traffic, for example using [`remote_cluster.host`](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/networking-settings.md#remote-cluster-network-settings). Without configuring the address, remote cluster traffic may be bound to the local interface, and remote clusters running on other machines can’t connect. -* Optionally, configure the remote server port using [`remote_cluster.port`](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/networking-settings.md#remote_cluster.port) (defaults to `9443`). - - - -## Common issues [remote-clusters-troubleshooting-common-issues] - -The following issues are listed in the order they may occur while setting up a remote cluster. - -### Remote cluster not reachable [remote-clusters-not-reachable] - -#### Symptom [_symptom] - -A local cluster may not be able to reach a remote cluster for many reasons. For example, the remote cluster server may not be enabled, an incorrect host or port may be configured, or a firewall may be blocking traffic. When a remote cluster is not reachable, check the logs of the local cluster for a `connect_exception`. - -When the remote cluster is configured using proxy mode: - -```txt -[2023-06-28T16:36:47,264][WARN ][o.e.t.ProxyConnectionStrategy] [local-node] failed to open any proxy connections to cluster [my] -org.elasticsearch.transport.ConnectTransportException: [][192.168.0.42:9443] **connect_exception** -``` - -When the remote cluster is configured using sniff mode: - -```txt -[2023-06-28T16:38:37,731][WARN ][o.e.t.SniffConnectionStrategy] [local-node] fetching nodes from external cluster [my] failed -org.elasticsearch.transport.ConnectTransportException: [][192.168.0.42:9443] **connect_exception** -``` - - -#### Resolution [_resolution] - -* Check the host and port for the remote cluster are correct. -* Ensure the [remote cluster server is enabled](#remote-clusters-troubleshooting-enable-server) on the remote cluster. -* Ensure no firewall is blocking the communication. - - - -### Remote cluster connection is unreliable [remote-clusters-unreliable-network] - -#### Symptom [_symptom_2] - -The local cluster can connect to the remote cluster, but the connection does not work reliably. For example, some cross-cluster requests may succeed while others report connection errors, time out, or appear to be stuck waiting for the remote cluster to respond. - -When {{es}} detects that the remote cluster connection is not working, it will report the following message in its logs: - -```txt -[2023-06-28T16:36:47,264][INFO ][o.e.t.ClusterConnectionManager] [local-node] transport connection to [{my-remote#192.168.0.42:9443}{...}] closed by remote -``` - -This message will also be logged if the node of the remote cluster to which {{es}} is connected is shut down or restarted. - -Note that with some network configurations it could take minutes or hours for the operating system to detect that a connection has stopped working. Until the failure is detected and reported to {{es}}, requests involving the remote cluster may time out or may appear to be stuck. - - -#### Resolution [_resolution_2] - -* Ensure that the network between the clusters is as reliable as possible. -* Ensure that the network is configured to permit [Long-lived idle connections](asciidocalypse://docs/elasticsearch/docs/reference/elasticsearch/configuration-reference/networking-settings.md#long-lived-connections). -* Ensure that the network is configured to detect faulty connections quickly. In particular, you must enable and fully support TCP keepalives, and set a short [retransmission timeout](../deploy/self-managed/system-config-tcpretries.md). -* On Linux systems, execute `ss -tonie` to verify the details of the configuration of each network connection between the clusters. -* If the problems persist, capture network packets at both ends of the connection and analyse the traffic to look for delays and lost messages. - - - -### TLS trust not established [remote-clusters-troubleshooting-tls-trust] - -TLS can be misconfigured on the local or the remote cluster. The result is that the local cluster does not trust the certificate presented by the remote cluster. - -#### Symptom [_symptom_3] - -The local cluster logs `failed to establish trust with server`: - -```txt -[2023-06-29T09:40:55,465][WARN ][o.e.c.s.DiagnosticTrustManager] [local-node] **failed to establish trust with server** at [192.168.0.42]; the server provided a certificate with subject name [CN=remote_cluster], fingerprint [529de35e15666ffaa26afa50876a2a48119db03a], no keyUsage and no extendedKeyUsage; the certificate is valid between [2023-01-29T12:08:37Z] and [2032-08-29T12:08:37Z] (current time is [2023-08-16T23:40:55.464275Z], certificate dates are valid); the session uses cipher suite [TLS_AES_256_GCM_SHA384] and protocol [TLSv1.3]; the certificate has subject alternative names [DNS:localhost,DNS:localhost6.localdomain6,IP:127.0.0.1,IP:0:0:0:0:0:0:0:1,DNS:localhost4,DNS:localhost6,DNS:localhost.localdomain,DNS:localhost4.localdomain4,IP:192.168.0.42]; the certificate is issued by [CN=Elastic Auto RemoteCluster CA] but the server did not provide a copy of the issuing certificate in the certificate chain; this ssl context ([(shared) (with trust configuration: JDK-trusted-certs)]) is not configured to trust that issuer but trusts [97] other issuers -sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target -``` - -The remote cluster logs `client did not trust this server's certificate`: - -```txt -[2023-06-29T09:40:55,478][WARN ][o.e.x.c.s.t.n.SecurityNetty4Transport] [remote-node] **client did not trust this server's certificate**, closing connection Netty4TcpChannel{localAddress=/192.168.0.42:9443, remoteAddress=/192.168.0.84:57305, profile=_remote_cluster} -``` - - -#### Resolution [_resolution_3] - -Read the warn log message on the local cluster carefully to determine the exact cause of the failure. For example: - -* Is the remote cluster certificate not signed by a trusted CA? This is the most likely cause. -* Is hostname verification failing? -* Is the certificate expired? - -Once you know the cause, you should be able to fix it by adjusting the remote cluster related SSL settings on either the local cluster or the remote cluster. - -Often, the issue is on the local cluster. For example, fix it by configuring necessary trusted CAs (`xpack.security.remote_cluster_client.ssl.certificate_authorities`). - -If you change the `elasticsearch.yml` file, the associated cluster needs to be restarted for the changes to take effect. - - - - -## API key authentication issues [remote-clusters-troubleshooting-api-key] - -### Connecting to transport port when using API key authentication [remote-clusters-troubleshooting-transport-port-api-key] - -When using API key authentication, a local cluster should connect to a remote cluster’s remote cluster server port (defaults to `9443`) instead of the transport port (defaults to `9300`). A misconfiguration can lead to a number of symptoms: - -#### Symptom 1 [_symptom_1] - -It’s recommended to use different CAs and certificates for the transport interface and the remote cluster server interface. If this recommendation is followed, a remote cluster client node does not trust the server certificate presented by a remote cluster on the transport interface. - -The local cluster logs `failed to establish trust with server`: - -```txt -[2023-06-28T12:48:46,575][WARN ][o.e.c.s.DiagnosticTrustManager] [local-node] **failed to establish trust with server** at [1192.168.0.42]; the server provided a certificate with subject name [CN=transport], fingerprint [c43e628be2a8aaaa4092b82d78f2bc206c492322], no keyUsage and no extendedKeyUsage; the certificate is valid between [2023-01-29T12:05:53Z] and [2032-08-29T12:05:53Z] (current time is [2023-06-28T02:48:46.574738Z], certificate dates are valid); the session uses cipher suite [TLS_AES_256_GCM_SHA384] and protocol [TLSv1.3]; the certificate has subject alternative names [DNS:localhost,DNS:localhost6.localdomain6,IP:127.0.0.1,IP:0:0:0:0:0:0:0:1,DNS:localhost4,DNS:localhost6,DNS:localhost.localdomain,DNS:localhost4.localdomain4,IP:192.168.0.42]; the certificate is issued by [CN=Elastic Auto Transport CA] but the server did not provide a copy of the issuing certificate in the certificate chain; this ssl context ([xpack.security.remote_cluster_client.ssl (with trust configuration: PEM-trust{/rcs2/ssl/remote-cluster-ca.crt})]) is not configured to trust that issuer, it only trusts the issuer [CN=Elastic Auto RemoteCluster CA] with fingerprint [ba2350661f66e46c746c1629f0c4b645a2587ff4] -sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target -``` - -The remote cluster logs `client did not trust this server's certificate`: - -```txt -[2023-06-28T12:48:46,584][WARN ][o.e.x.c.s.t.n.SecurityNetty4Transport] [remote-node] **client did not trust this server's certificate**, closing connection Netty4TcpChannel{localAddress=/192.168.0.42:9309, remoteAddress=/192.168.0.84:60810, profile=default} -``` - - -#### Symptom 2 [_symptom_2_2] - -The CA and certificate can be shared between the transport and remote cluster server interface. Since a remote cluster client does not have a client certificate by default, the server will fail to verify the client certificate. - -The local cluster logs `Received fatal alert: bad_certificate`: - -```txt -[2023-06-28T12:43:30,705][WARN ][o.e.t.TcpTransport ] [local-node] exception caught on transport layer [Netty4TcpChannel{localAddress=/192.168.0.84:60738, remoteAddress=/192.168.0.42:9309, profile=_remote_cluster}], closing connection -io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: **Received fatal alert: bad_certificate** -``` - -The remote cluster logs `Empty client certificate chain`: - -```txt -[2023-06-28T12:43:30,772][WARN ][o.e.t.TcpTransport ] [remote-node] exception caught on transport layer [Netty4TcpChannel{localAddress=/192.168.0.42:9309, remoteAddress=/192.168.0.84:60783, profile=default}], closing connection -io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: **Empty client certificate chain** -``` - - -#### Symptom 3 [_symptom_3_2] - -If the remote cluster client is configured for mTLS and provides a valid client certificate, the connection fails because the client does not send the expected authentication header. - -The local cluster logs `missing authentication`: - -```txt -[2023-06-28T13:04:52,710][WARN ][o.e.t.ProxyConnectionStrategy] [local-node] failed to open any proxy connections to cluster [my] -org.elasticsearch.transport.RemoteTransportException: [remote-node][192.168.0.42:9309][cluster:internal/remote_cluster/handshake] -Caused by: org.elasticsearch.ElasticsearchSecurityException: **missing authentication** credentials for action [cluster:internal/remote_cluster/handshake] -``` - -This does not show up in the logs of the remote cluster. - - -#### Symptom 4 [_symptom_4] - -If anonymous access is enabled on the remote cluster and it does not require authentication, depending on the privileges of the anonymous user, the local cluster may log the following. - -If the anonymous user does not the have necessary privileges to make a connection, the local cluster logs `unauthorized`: - -```txt -org.elasticsearch.transport.RemoteTransportException: [remote-node][192.168.0.42:9309][cluster:internal/remote_cluster/handshake] -Caused by: org.elasticsearch.ElasticsearchSecurityException: action [cluster:internal/remote_cluster/handshake] is **unauthorized** for user [anonymous_foo] with effective roles [reporting_user], this action is granted by the cluster privileges [cross_cluster_search,cross_cluster_replication,manage,all] -``` - -If the anonymous user has necessary privileges, for example it is a superuser, the local cluster logs `requires channel profile to be [_remote_cluster], but got [default]`: - -```txt -[2023-06-28T13:09:52,031][WARN ][o.e.t.ProxyConnectionStrategy] [local-node] failed to open any proxy connections to cluster [my] -org.elasticsearch.transport.RemoteTransportException: [remote-node][192.168.0.42:9309][cluster:internal/remote_cluster/handshake] -Caused by: java.lang.IllegalArgumentException: remote cluster handshake action **requires channel profile to be [_remote_cluster], but got [default]** -``` - - -#### Resolution [_resolution_4] - -Check the port number and ensure you are indeed connecting to the remote cluster server instead of the transport interface. - - - -### Connecting without a cross-cluster API key [remote-clusters-troubleshooting-no-api-key] - -A local cluster uses the presence of a cross-cluster API key to determine the model with which it connects to a remote cluster. If a cross-cluster API key is present, it uses API key based authentication. Otherwise, it uses certificate based authentication. You can check what model is being used with the [remote cluster info API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-remote-info) on the local cluster: - -```console -GET /_remote/info -``` - -The API should return `"connected" : true`. When using [API key authentication](remote-clusters-api-key.md), it should also return `"cluster_credentials": "::es_redacted::"`. - -```console-result -{ - "cluster_one" : { - "seeds" : [ - "127.0.0.1:9443" - ], - "connected" : true, <1> - "num_nodes_connected" : 1, - "max_connections_per_cluster" : 3, - "initial_connect_timeout" : "30s", - "skip_unavailable" : false, - "cluster_credentials": "::es_redacted::", <2> - "mode" : "sniff" - } -} -``` - -1. The remote cluster has connected successfully. -2. If present, indicates the remote cluster has connected using [API key authentication](remote-clusters-api-key.md) instead of [certificate based authentication](remote-clusters-cert.md). - - -Besides checking the response of the remote cluster info API, you can also check the logs. - -#### Symptom 1 [_symptom_1_2] - -If no cross-cluster API key is used, the local cluster uses the certificate based authentication method, and connects to the remote cluster using the TLS configuration of the transport interface. If the remote cluster has different TLS CA and certificate for transport and remote cluster server interfaces (which is the recommendation), TLS verification will fail. - -The local cluster logs `failed to establish trust with server`: - -```txt -[2023-06-28T12:51:06,452][WARN ][o.e.c.s.DiagnosticTrustManager] [local-node] **failed to establish trust with server** at []; the server provided a certificate with subject name [CN=remote_cluster], fingerprint [529de35e15666ffaa26afa50876a2a48119db03a], no keyUsage and no extendedKeyUsage; the certificate is valid between [2023-01-29T12:08:37Z] and [2032-08-29T12:08:37Z] (current time is [2023-06-28T02:51:06.451581Z], certificate dates are valid); the session uses cipher suite [TLS_AES_256_GCM_SHA384] and protocol [TLSv1.3]; the certificate has subject alternative names [DNS:localhost,DNS:localhost6.localdomain6,IP:127.0.0.1,IP:0:0:0:0:0:0:0:1,DNS:localhost4,DNS:localhost6,DNS:localhost.localdomain,DNS:localhost4.localdomain4,IP:192.168.0.42]; the certificate is issued by [CN=Elastic Auto RemoteCluster CA] but the server did not provide a copy of the issuing certificate in the certificate chain; this ssl context ([xpack.security.transport.ssl (with trust configuration: PEM-trust{/rcs2/ssl/transport-ca.crt})]) is not configured to trust that issuer, it only trusts the issuer [CN=Elastic Auto Transport CA] with fingerprint [bbe49e3f986506008a70ab651b188c70df104812] -sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target -``` - -The remote cluster logs `client did not trust this server's certificate`: - -```txt -[2023-06-28T12:52:16,914][WARN ][o.e.x.c.s.t.n.SecurityNetty4Transport] [remote-node] **client did not trust this server's certificate**, closing connection Netty4TcpChannel{localAddress=/192.168.0.42:9443, remoteAddress=/192.168.0.84:60981, profile=_remote_cluster} -``` - - -#### Symptom 2 [_symptom_2_3] - -Even if TLS verification is not an issue, the connection fails due to missing credentials. - -The local cluster logs `Please ensure you have configured remote cluster credentials`: - -```txt -Caused by: java.lang.IllegalArgumentException: Cross cluster requests through the dedicated remote cluster server port require transport header [_cross_cluster_access_credentials] but none found. **Please ensure you have configured remote cluster credentials** on the cluster originating the request. -``` - -This does not show up in the logs of the remote cluster. - - -#### Resolution [_resolution_5] - -Add the cross-cluster API key to {{es}} keystore on every node of the local cluster. Use the [Nodes reload secure settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API to reload the keystore. - - - -### Using the wrong API key type [remote-clusters-troubleshooting-wrong-api-key-type] - -API key based authentication requires [cross-cluster API keys](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). It does not work with [REST API keys](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key). - -#### Symptom [_symptom_5] - -The local cluster logs `authentication expected API key type of [cross_cluster]`: - -```txt -[2023-06-28T13:26:53,962][WARN ][o.e.t.ProxyConnectionStrategy] [local-node] failed to open any proxy connections to cluster [my] -org.elasticsearch.transport.RemoteTransportException: [remote-node][192.168.0.42:9443][cluster:internal/remote_cluster/handshake] -Caused by: org.elasticsearch.ElasticsearchSecurityException: **authentication expected API key type of [cross_cluster]**, but API key [agZXJocBmA2beJfq2yKu] has type [rest] -``` - -This does not show up in the logs of the remote cluster. - - -#### Resolution [_resolution_6] - -Ask the remote cluster administrator to create and distribute a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). Replace the existing API key in the {{es}} keystore with this cross-cluster API key on every node of the local cluster. Use the [Nodes reload secure settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API to reload the keystore. - - - -### Invalid API key [remote-clusters-troubleshooting-non-valid-api-key] - -A cross-cluster API can fail to authenticate. For example, when its credentials are incorrect, or if it’s invalidated or expired. - -#### Symptom [_symptom_6] - -The local cluster logs `unable to authenticate`: - -```txt -[2023-06-28T13:22:58,264][WARN ][o.e.t.ProxyConnectionStrategy] [local-node] failed to open any proxy connections to cluster [my] -org.elasticsearch.transport.RemoteTransportException: [remote-node][192.168.0.42:9443][cluster:internal/remote_cluster/handshake] -Caused by: org.elasticsearch.ElasticsearchSecurityException: **unable to authenticate** user [agZXJocBmA2beJfq2yKu] for action [cluster:internal/remote_cluster/handshake] -``` - -The remote cluster logs `Authentication using apikey failed`: - -```txt -[2023-06-28T13:24:38,744][WARN ][o.e.x.s.a.ApiKeyAuthenticator] [remote-node] **Authentication using apikey failed** - invalid credentials for API key [agZXJocBmA2beJfq2yKu] -``` - - -#### Resolution [_resolution_7] - -Ask the remote cluster administrator to create and distribute a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). Replace the existing API key in the {{es}} keystore with this cross-cluster API key on every node of the local cluster. Use the [Nodes reload secure settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API to reload the keystore. - - - -### API key or local user has insufficient privileges [remote-clusters-troubleshooting-insufficient-privileges] - -The effective permission for a local user running requests on a remote cluster is determined by the intersection of the cross-cluster API key’s privileges and the local user’s `remote_indices` privileges. - -#### Symptom [_symptom_7] - -Request failures due to insufficient privileges result in API responses like: - -```js -{ - "type": "security_exception", - "reason": "action [indices:data/read/search] towards remote cluster is **unauthorized for user** [foo] with assigned roles [foo-role] authenticated by API key id [agZXJocBmA2beJfq2yKu] of user [elastic-admin] on indices [cd], this action is granted by the index privileges [read,all]" -} -``` - -This does not show up in any logs. - - -#### Resolution [_resolution_8] - -1. Check that the local user has the necessary `remote_indices` or `remote_cluster` privileges. Grant sufficient `remote_indices` or `remote_cluster` privileges if necessary. -2. If permission is not an issue locally, ask the remote cluster administrator to create and distribute a [cross-cluster API key](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key). Replace the existing API key in the {{es}} keystore with this cross-cluster API key on every node of the local cluster. Use the [Nodes reload secure settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-reload-secure-settings) API to reload the keystore. - - - -### Local user has no `remote_indices` privileges [remote-clusters-troubleshooting-no-remote_indices-privileges] - -This is a special case of insufficient privileges. In this case, the local user has no `remote_indices` privileges at all for the target remote cluster. {{es}} can detect that and issue a more explicit error response. - -#### Symptom [_symptom_8] - -This results in API responses like: - -```js -{ - "type": "security_exception", - "reason": "action [indices:data/read/search] towards remote cluster [my] is unauthorized for user [foo] with effective roles [] (assigned roles [foo-role] were not found) because **no remote indices privileges apply for the target cluster**" -} -``` - - -#### Resolution [_resolution_9] - -Grant sufficient `remote_indices` privileges to the local user. - - - - diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index fb6a2bdac7..33a6986905 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -536,7 +536,6 @@ toc: - file: remote-clusters/remote-clusters-cert.md - file: remote-clusters/remote-clusters-migrate.md - file: remote-clusters/remote-clusters-settings.md - - file: remote-clusters/remote-clusters-troubleshooting.md - file: remote-clusters/eck-remote-clusters.md - file: security.md children: diff --git a/deploy-manage/tools/cross-cluster-replication.md b/deploy-manage/tools/cross-cluster-replication.md index ae1b0b3c6b..694a657557 100644 --- a/deploy-manage/tools/cross-cluster-replication.md +++ b/deploy-manage/tools/cross-cluster-replication.md @@ -43,13 +43,15 @@ In a uni-directional configuration, the cluster containing follower indices must | 7.17 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | | 8.0–9.0 | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![No](https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | ![Yes](https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png "") | -::::{note} -This documentation is for {{es}} version 9.0.0-beta1, which is not yet released. The above compatibility table applies if both clusters are running a released version of {{es}}, or if one of the clusters is running a released version and the other is running a pre-release build with a later build date. A cluster running a pre-release build of {{es}} can also communicate with remote clusters running the same pre-release build. Running a mix of pre-release builds is unsupported and typically will not work, even if the builds have the same version number. -:::: ::::: +% Moved from another file - hiding this note for now +% ::::{note} +% CCR is not supported for indices used by Enterprise Search. +% :::: + ## Multi-cluster architectures [ccr-multi-cluster-architectures] diff --git a/troubleshoot/elasticsearch/elasticsearch.md b/troubleshoot/elasticsearch/elasticsearch.md index ebddf62866..6a98f87714 100644 --- a/troubleshoot/elasticsearch/elasticsearch.md +++ b/troubleshoot/elasticsearch/elasticsearch.md @@ -64,7 +64,7 @@ If you’re using Elastic Cloud Hosted, then you can use AutoOps to monitor your * [Troubleshooting searches](troubleshooting-searches.md) * [Troubleshooting shards capacity](troubleshooting-shards-capacity-issues.md) * [Troubleshooting an unbalanced cluster](troubleshooting-unbalanced-cluster.md) -* [Troubleshooting remote clusters](../../deploy-manage/remote-clusters/remote-clusters-troubleshooting.md) +* [Troubleshooting remote clusters](/troubleshoot/elasticsearch/remote-clusters.md) ## Contact us [troubleshooting-contact-support] diff --git a/troubleshoot/elasticsearch/remote-clusters.md b/troubleshoot/elasticsearch/remote-clusters.md index 155ed0b4ff..f5e5b2176e 100644 --- a/troubleshoot/elasticsearch/remote-clusters.md +++ b/troubleshoot/elasticsearch/remote-clusters.md @@ -2,6 +2,9 @@ navigation_title: Remote clusters mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/remote-clusters-troubleshooting.html +applies_to: + deployment: + self: ga --- @@ -83,7 +86,7 @@ org.elasticsearch.transport.ConnectTransportException: [][192.168.0.42:9443] **c #### Resolution [_resolution] * Check the host and port for the remote cluster are correct. -* Ensure the [remote cluster server is enabled](../../deploy-manage/remote-clusters/remote-clusters-troubleshooting.md#remote-clusters-troubleshooting-enable-server) on the remote cluster. +* Ensure the [remote cluster server is enabled](#remote-clusters-troubleshooting-enable-server) on the remote cluster. * Ensure no firewall is blocking the communication.