diff --git a/docs/orchestrating-elastic-stack-applications/agent-standalone.asciidoc b/docs/orchestrating-elastic-stack-applications/agent-standalone.asciidoc index 8e96693563..d497591497 100644 --- a/docs/orchestrating-elastic-stack-applications/agent-standalone.asciidoc +++ b/docs/orchestrating-elastic-stack-applications/agent-standalone.asciidoc @@ -273,7 +273,7 @@ The `elasticsearchRefs` element allows ECK to automatically configure Elastic Ag [id="{p}-elastic-agent-set-output"] === Set manually Elastic Agent outputs -If the `elasticsearchRefs` element is specified, ECK populates the outputs section of the Elastic Agent configuration. ECK creates a user with appropriate roles and permissions and uses its credentials. If required, it also mounts the CA certificate in all Agent Pods, and recreates Pods when this certificate changes. Moreover, `elasticsearchRef` element can refer to an ECK-managed Elasticsearch cluster by filling the `name`, `namespace`, `serviceName` fields accordingly, as well as to a Kubernetes secret that contains the connection information to an Elasticsearch cluster not managed by it. In the latter case, for authenticating against the Elasticsearch cluster the secret must contain the fields of `url` and either the `username` with `password` or the `api-key`. +If the `elasticsearchRefs` element is specified, ECK populates the outputs section of the Elastic Agent configuration. ECK creates a user with appropriate roles and permissions and uses its credentials. If required, it also mounts the CA certificate in all Agent Pods, and recreates Pods when this certificate changes. Moreover, `elasticsearchRef` element can refer to an ECK-managed Elasticsearch cluster by filling the `name`, `namespace`, `serviceName` fields accordingly, as well as to a Kubernetes secret that contains the connection information to an Elasticsearch cluster not managed by it. In the latter case, for authenticating against the Elasticsearch cluster the secret must contain the fields of `url` and either the `username` with `password` or the `api-key`. Refer to <<{p}-connect-to-unmanaged-resources>> for additional details. The outputs can also be set manually. To do that, remove the `elasticsearchRefs` element from the specification and include an appropriate output configuration in the `config`, or indirectly through the `configRef` mechanism. diff --git a/docs/orchestrating-elastic-stack-applications/connect-to-unmanaged-resources.asciidoc b/docs/orchestrating-elastic-stack-applications/connect-to-unmanaged-resources.asciidoc new file mode 100644 index 0000000000..5ebfbb96e5 --- /dev/null +++ b/docs/orchestrating-elastic-stack-applications/connect-to-unmanaged-resources.asciidoc @@ -0,0 +1,84 @@ +:page_id: connect-to-unmanaged-resources +ifdef::env-github[] +**** +link:https://www.elastic.co/guide/en/cloud-on-k8s/master/k8s-{page_id}.html[View this document on the Elastic website] +**** +endif::[] +[id="{p}-{page_id}"] += Connect to external Elastic resources + +Fields like `elasticsearchRef` or `kibanaRef` are useful to automatically establish connections between applications managed by the same ECK operator instance. It is however also possible to connect to applications managed by a different ECK operator instance, or to applications not managed by ECK, for example an Elastic Cloud deployment. This can be done by providing connection details and credentials in a `Secret` through the `secretName` attribute: + +[source,yaml,subs="attributes"] +---- +apiVersion: v1 +kind: Secret +metadata: + name: external-es-ref +stringData: + url: https://sample.gcp.elastic-cloud.com + username: "elastic" + password: REDACTED +--- +apiVersion: kibana.k8s.elastic.co/v1 +kind: Kibana +metadata: + name: kibana-sample +spec: + version: 8.14.0 + count: 1 + elasticsearchRef: + secretName: external-es-ref +---- + +In the case of Elastic Agent you can also specify several named references: + +[source,yaml,subs="attributes"] +---- +apiVersion: v1 +kind: Secret +metadata: + name: external-es-ref +stringData: + url: https://abcd-42.xyz.elastic-cloud.com:443 + username: "" + password: "" + api-key: REDACTED + ca.crt: REDACTED +--- +apiVersion: v1 +kind: Secret +metadata: + name: external-es-ref2 +stringData: + url: https://efgh-43.xyz.elastic-cloud.com:443 + username: "" + password: "" + api-key: REDACTED + ca.crt: REDACTED +--- +apiVersion: agent.k8s.elastic.co/v1alpha1 +kind: Agent +metadata: + name: quickstart +spec: + version: {version} + elasticsearchRefs: + - outputName: default + secretName: external-es-ref + - outputName: monitoring + secretName: external-es-ref2 +---- + +The following fields are expected to be set in the referenced `Secret`: + +* `url` (required): URL to be used to access the external resource. +* `username` (required): The username of the user to be authenticated to the Elastic resource. +* `password` (required): The password for the provided user. +* `ca.crt` (optional): The certificate authority to be used to connect to the external resource. + +In the case of Agent and Beats resources the following field can also be used to connect to Elasticsearch: + +* `api-key`: An API key to authenticate against the Elastic resource. + +NOTE: The operator must be able to connect to the external resources to check version compatibility. diff --git a/docs/orchestrating-elastic-stack-applications/enterprise-search.asciidoc b/docs/orchestrating-elastic-stack-applications/enterprise-search.asciidoc index 70eaa23ca3..af45b4d633 100644 --- a/docs/orchestrating-elastic-stack-applications/enterprise-search.asciidoc +++ b/docs/orchestrating-elastic-stack-applications/enterprise-search.asciidoc @@ -333,7 +333,13 @@ The `elasticsearchRef` element allows ECK to automatically configure Enterprise [id="{p}-enterprise-search-connect-non-eck-es"] === Connect to an external Elasticsearch cluster -If you do not want to use the `elasticsearchRef` mechanism or if you want to connect to an Elasticsearch cluster not managed by ECK, you can manually configure Enterprise Search to access any available Elasticsearch cluster: + +==== Automatically + +Refer to <<{p}-connect-to-unmanaged-resources>> to automatically configure Enterprise Search using connection settings from a `Secret`. + +==== Manually +If you do not want to use the `elasticsearchRef` mechanism you can manually configure Enterprise Search to access any available Elasticsearch cluster: [source,yaml,subs="attributes,+macros"] ---- diff --git a/docs/orchestrating-elastic-stack-applications/kibana.asciidoc b/docs/orchestrating-elastic-stack-applications/kibana.asciidoc index 24066c48b5..77d01aa739 100644 --- a/docs/orchestrating-elastic-stack-applications/kibana.asciidoc +++ b/docs/orchestrating-elastic-stack-applications/kibana.asciidoc @@ -61,6 +61,12 @@ The Kibana configuration file is automatically setup by ECK to establish a secur You can also configure Kibana to connect to an Elasticsearch cluster that is managed by a different installation of ECK, or runs outside the Kubernetes cluster. In this case, you need the IP address or URL of the Elasticsearch cluster and a valid username and password pair to access the cluster. +=== Using a Secret + +Refer to <<{p}-connect-to-unmanaged-resources>> to automatically configure Kibana using connection settings from a `Secret`. + +=== Using secure settings + Use the <<{p}-kibana-secure-settings,secure settings>> mechanism to securely store the credentials of the external Elasticsearch cluster: [source,shell] diff --git a/docs/orchestrating-elastic-stack-applications/orchestrating-elastic-stack-applications.asciidoc b/docs/orchestrating-elastic-stack-applications/orchestrating-elastic-stack-applications.asciidoc index 103430e726..8b2fbb0cf6 100644 --- a/docs/orchestrating-elastic-stack-applications/orchestrating-elastic-stack-applications.asciidoc +++ b/docs/orchestrating-elastic-stack-applications/orchestrating-elastic-stack-applications.asciidoc @@ -27,6 +27,7 @@ endif::[] - <<{p}-stateless-autoscaling>> - <<{p}-stack-config-policy>> - <<{p}-upgrading-stack>> +- <<{p}-connect-to-unmanaged-resources>> -- @@ -48,4 +49,4 @@ include::managing-compute-resources.asciidoc[leveloffset=+1] include::autoscaling.asciidoc[leveloffset=+1] include::stack-config-policy.asciidoc[leveloffset=+1] include::upgrading-stack.asciidoc[leveloffset=+1] - +include::connect-to-unmanaged-resources.asciidoc[leveloffset=+1]