Connect Beats to Elastic Cloud

[jcamu]

Hello.
How to use this chart to target Elastic Cloud instance and not Reference to ECK-managed Elasticsearch instance ?
Is it expected ? Schedule ?
If not, what is the recommended solution for elastic cloud ?

[Evanjt1]

@jcamu

See

cloud-on-k8s/config/crds/v1/all-crds.yaml

Lines 2336 to 2346 in 62f2e27

	secretName:
	description: 'SecretName is the name of an existing Kubernetes
	secret that contains connection information for associating
	an Elastic resource not managed by the operator. The referenced
	secret must contain the following: - `url`: the URL to reach
	the Elastic resource - `username`: the username of the user
	to be authenticated to the Elastic resource - `password`: the
	password of the user to be authenticated to the Elastic resource
	- `ca.crt`: the CA certificate in PEM format (optional). This
	field cannot be used in combination with the other fields name,
	namespace or serviceName.'

You can target a beat with an elasticsearch cluster not managed by ECK by creating a secret with connection details and specifying said secret in elasticsearchRef.secretName. Same for associating with kibana not managed by ECK.

I haven't tested with cloud.id or cloud.auth as seen here, but those are just niceties for username, password, etc. anyway.

Caveat, I'm not an elastic employee, simply a user in a similar situation.

[thbkrkr]

Hello @jcamu,

@Evanjt1 is right and you can leverage the elasticsearchRef.secretName field to associate a Beat to an external Elasticsearch not managed by ECK.

secretName in ref field was implemented for the purpose of Stack Monitoring (#5240) but we designed it to be usable for all associations. We will make it generally usable for all associations (#6449).

Here is a usage example for Filebeat:

apiVersion: v1
kind: Secret
metadata:
  name: cloud-es-ref
stringData:
  # explicit port otherwise beat uses 9200
  url: https://monitoring.es.europe-west42.gcp.elastic-cloud.com:443
  username: elastic
  password: REDACTED
---
apiVersion: v1
kind: Secret
metadata:
  name: cloud-kb-ref
stringData:
  url: https://monitoring.kb.europe-west42.gcp.elastic-cloud.com:9243
  username: elastic
  password: REDACTED
---
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: filebeat
spec:
  type: filebeat
  version: 8.6.1
  elasticsearchRef:
    secretName: cloud-es-ref
  kibanaRef:
    secretName: cloud-kb-ref
  config:
    ...
  daemonSet:
    ...

[thbkrkr]

I'm closing as we have #6449 to document this.