Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs for agent managed by fleet on kubernetes #848

Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ To learn how to install, configure, and run your {agent}s, see:
* <<elastic-agent-installation>>
* <<uninstall-elastic-agent>>
* <<run-elastic-agent-standalone>>
//* <<running-on-kubernetes>>
* <<running-on-kubernetes-managed-by-fleet>>
* <<running-on-kubernetes-standalone>>
* <<upgrade-elastic-agent>>
* <<start-elastic-agent>>
Expand All @@ -31,7 +31,7 @@ include::uninstall-elastic-agent.asciidoc[leveloffset=+1]

include::run-elastic-agent-standalone.asciidoc[leveloffset=+1]

//include::running-on-kubernetes.asciidoc[leveloffset=+1]
include::running-on-kubernetes-managed-by-fleet.asciidoc[leveloffset=+1]

include::running-on-kubernetes-standalone.asciidoc[leveloffset=+1]

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
[[running-on-kubernetes-managed-by-fleet]]
[role="xpack"]
= Run {agent} on Kubernetes managed by {fleet}

Use {agent} https://www.docker.elastic.co/r/beats/elastic-agent[Docker images] on Kubernetes to
retrieve cluster metrics.

ifeval::["{release-state}"=="unreleased"]

However, version {version} of {agent} has not yet been
released, so no Docker image is currently available for this version.

endif::[]


[discrete]
== Kubernetes deploy manifests

With {fleet}, each agent enrols at a policy defined in {kib} and stored in
MichaelKatsoulis marked this conversation as resolved.
Show resolved Hide resolved
{es}. The policy specifies how to collect observability data from the services
to be monitored. The {agent} connects to a trusted {fleet-server} instance
to retrieve the policy and report agent events.

We recommend using {fleet} management because it makes the management and
upgrade of agents considerably easier.

On Kubernetes deploy {agent} as a https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/[DaemonSet]
MichaelKatsoulis marked this conversation as resolved.
Show resolved Hide resolved
to ensure that there is a running instance on each node of the cluster.
These instances are used to retrieve metrics from the host, such as system metrics, Docker stats,
MichaelKatsoulis marked this conversation as resolved.
Show resolved Hide resolved
and metrics from all the services running on top of Kubernetes.

In addition, one of the Pods in the DaemonSet will constantly hold a _leader lock_ which makes it responsible for
handling cluster-wide monitoring.
Find more information about leader election configuration options at <<kubernetes_leaderelection-provider, leader election provider>>.
This instance is used to retrieve metrics that are unique for the whole
cluster, such as Kubernetes events or
https://github.com/kubernetes/kube-state-metrics[kube-state-metrics].


Everything is deployed under the `kube-system` namespace by default. To change
the namespace, modify the manifest file.

To download the manifest file, run:

["source", "sh", subs="attributes"]
------------------------------------------------
curl -L -O https://raw.githubusercontent.com/elastic/beats/{branch}/deploy/kubernetes/elastic-agent-managed-kubernetes.yaml
------------------------------------------------

[discrete]
== Settings

{agent} is enrolled to a running {fleet-server} using `FLEET_URL` parameter.
The `FLEET_ENROLLMENT_TOKEN` parameter is used to connect {agent} to a
specific {agent} policy.
Learn more on how to get an enrollment token from {fleet} in the {fleet-guide}/fleet-quick-start.html[Quick start
guide].

If `FLEET_ENROLLMENT_TOKEN` is empty then agent will connect to {kib} and get the default policy.
In that case `KIBANA_HOST`, `KIBANA_FLEET_USERNAME`, `KIBANA_FLEET_PASSWORD` have to be specified.

To specify different destination/credentials,
change the following parameters in the manifest file:

[source,yaml]
------------------------------------------------
- name: FLEET_URL
value: "https://fleet-server_url:port"
- name: FLEET_ENROLLMENT_TOKEN
value: "token"
- name: KIBANA_HOST
value: ""
- name: KIBANA_FLEET_USERNAME
value: ""
- name: KIBANA_FLEET_PASSWORD
value: ""
------------------------------------------------

// Begin collapsed section
[%collapsible]
.Configuration details
====
****

[cols="2*<a"]
|===
| Settings | Description

include::configuration/env/shared-env.asciidoc[tag=fleet-url]

include::configuration/env/shared-env.asciidoc[tag=fleet-enrollment-token]

include::configuration/env/shared-env.asciidoc[tag=kibana-host]

include::configuration/env/shared-env.asciidoc[tag=kibana-fleet-username]

include::configuration/env/shared-env.asciidoc[tag=kibana-fleet-password]
|===

See <<agent-environment-variables>> for all available options.

****
====

[discrete]
=== Run {agent} on master nodes

Kubernetes master nodes can use https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/[taints]
to limit the workloads that can run on them. The manifest for managed {agent} defines
tolerations to run on master nodes. Agents running on master nodes collect metrics from the control plane
components (scheduler, controller manager) of Kuberentes.
MichaelKatsoulis marked this conversation as resolved.
Show resolved Hide resolved
To disable {agent} from running on master nodes, remove the following part of the Daemonset spec:

[source,yaml]
------------------------------------------------
spec:
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
------------------------------------------------


[discrete]
== Deploy

If planing to deploy `state_*` datasets of Kubernetes package,
MichaelKatsoulis marked this conversation as resolved.
Show resolved Hide resolved
https://github.com/kubernetes/kube-state-metrics#usage[kube-state-metrics] needs to be already deployed
in the cluster. If `kube-state-metrics` is not already running, deploy it now (see the
https://github.com/kubernetes/kube-state-metrics#kubernetes-deployment[Kubernetes
deployment] docs).

To deploy {agent} on Kubernetes, run:

["source", "sh", subs="attributes"]
------------------------------------------------
kubectl create -f elastic-agent-managed-kubernetes.yaml
------------------------------------------------

To check the status, run:

["source", "sh", subs="attributes"]
------------------------------------------------
$ kubectl get pod -n kube-system -l app=elastic-agent

NAME READY STATUS RESTARTS AGE
elastic-agent-hrjbg 1/1 Running 0 12m
elastic-agent-olpsd 1/1 Running 0 12m
------------------------------------------------


{agent}s should be enrolled to Fleet and user should be able to deploy Kubernetes package accordingly.
MichaelKatsoulis marked this conversation as resolved.
Show resolved Hide resolved
This can be confirmed in {kib} under Fleet / Agents section.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
[role="xpack"]
= Run {agent} standalone on Kubernetes

You can use {agent} https://www.docker.elastic.co/r/beats/elastic-agent[Docker images] on Kubernetes to
Use {agent} https://www.docker.elastic.co/r/beats/elastic-agent[Docker images] on Kubernetes to
retrieve cluster metrics.

TIP: Running {ecloud} on Kubernetes? See {eck-ref}/k8s-elastic-agent.html[Run {elastic-agent} on ECK].
Expand All @@ -18,15 +18,15 @@ endif::[]
== Kubernetes deploy manifests


You deploy {agent} as a https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/[DaemonSet]
to ensure that there's a running instance on each node of the cluster. These
Deploy {agent} as a https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/[DaemonSet]
to ensure that there is a running instance on each node of the cluster. These
instances are used to retrieve most metrics from the host, such as system
metrics, Docker stats, and metrics from all the services running on top of
Kubernetes.

In addition, one of the Pods in the DaemonSet will constantly hold a _leader lock_ which makes it responsible for
handling cluster-wide monitoring.
You can find more information about leader election configuration options at <<kubernetes_leaderelection-provider, leaderelection provider>>.
Find more information about leader election configuration options at <<kubernetes_leaderelection-provider, leader election provider>>.
This instance is used to retrieve metrics that are unique for the whole
cluster, such as Kubernetes events or
https://github.com/kubernetes/kube-state-metrics[kube-state-metrics]. If `kube-state-metrics` is not already
Expand Down Expand Up @@ -128,7 +128,7 @@ elastic-agent-hs4pb 1/1 Running 0 81m
[discrete]
== Autodiscover targeted Pods

You can define autodiscover conditions to allow {agent} to automatically
Autodiscover conditions can be defined to allow {agent} to automatically
identify Pods and start collecting from them using predefined integrations. For example, if a
user wants to automatically identify a Redis Pod and start monitoring it using the Redis integration,
the following configuration should be added as an extra input in the Daemonset manifest:
Expand Down
133 changes: 0 additions & 133 deletions docs/en/ingest-management/elastic-agent/running-on-kubernetes.asciidoc

This file was deleted.