ECK resources Helm chart - Elastic Agent & Elastic Fleet Server Agent #5889

naemono · 2022-07-26T15:46:56Z

related to #5505

This PR contains the 2nd version of the ECK-managed resources helm charts including

eck-elastic-agent chart for installing elastic-agent
eck-fleet-server for managing fleet server as an Elastic Agent
minor updates to the eck-stack chart.

~~Note that this Chart allows multiple instances of Elastic Agent to be installed, along with allowing multiple instances of serviceAccounts, clusterRoles, and clusterRoleBindings.~~

cc @jmlrt @Kushmaro @framsouza

In Progress

additional manual testing
additional chart unit tests

Edit templates to reference helm chart name properly Attempt to handle missing namespace in elasticref. convert to supporting multiple agent instances working on SAs/clusterRoles fix agent template Adding service accounts, cluster roles, and cluster role bindings to agent templates. Updating values documentation.

Adjusting installation notes for elastic-agent to be consistent. Removing any reference to appVersion for consistency. Adjusting annotations in all templates. Removing unneeded bits in agent helm tests. Adjusting default values for Elastic agent chart. Adjusting wording in fleet-agent example in Elastic agent chart.

…t tests.

naemono · 2022-07-27T16:34:33Z

deploy/eck-elastic-agent/Chart.yaml

+sources:
+  - https://github.com/elastic/cloud-on-k8s
+  - https://github.com/elastic/elastic-agent
+icon: https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt77c2da6e0198746e/620ac24e6662ca0a6f617114/icon-agent-32-color.svg


Anyone know a better icon location than this?

pebrc

LGTM for the approach taken. One thing I am wondering though is if we could make the whole RBAC side of deploying agent easier with the Helm chart. But I understand that is quite difficult if you have list of agents where you do not know what it what.

deploy/eck-elastic-agent/values.yaml

naemono · 2022-07-29T15:20:05Z

make the whole RBAC side of deploying agent easier with the Helm chart

I would love to make this easier for the user, but I'm not sure the Helm chart is the right place to do it. If we wanted to ensure that rbac rules were sufficient to the configuration, that logic would likely make more sense, and be easier to maintain in the operator itself, as complex logic in Helm charts just comes across as complex, ugly, and difficult to maintain.

I can certainly go down that road and attempt this within the Helm charts. Please let me know and I'll make the changes, and see what they look like.

pebrc · 2022-08-01T15:32:52Z

Please let me know and I'll make the changes, and see what they look like.

I don't have something that is ready to implement. I feel like Agent is special enough, compared to Kibana and Elasticsearch, to warrant doing a bit of design work for the Helm chart.

For example one idea I had was to make two different Helm charts one for Fleet Server and one for Elastic Agents. Behind the scenes it is the same CRD of course still.

That would allow us to configure two different sets of default RBAC permissions.

More far reaching permissions for Agent, roughly oriented maybe on the Kubernetes observability recipe or maybe the DaemonSet published by the Agents team

And less permissions for the Fleet server chart.

User could then override the default RBAC permissions for each of these charts to either restrict them further (e.g. for an APM integration we don't need all the K8s API permissions that K8s observability needs) or expand them where necessary.

With a Fleet server chart and an Elastic Agent chart we should be able to cover most if not all use cases we have currently in the recipes section of the repository. This is just an idea at this stage and not ready to implement. Probably worth discussing with the team. My motivation is to find a solution how we can keep the templating logic minimal in the Charts but still add value for our users by starting with sane defaults that cover the most common case.

naemono · 2022-08-04T15:02:36Z

We met and had a discussion about the current implementation, vs a more specialized implementation for, say, Fleet, and comparing what the user experience was like between the two, we decided to implement the more specialized Helm chart for Fleet, which simplifies what the Agent helm chart looks like. I'll move this back to a WIP, and update when there's more information.

Add fleet chart.

Add additional tests Updated README

Update notes to remove cluster. Update names of tests

remove icon for fleet server

barkbay · 2022-09-05T11:19:54Z

deploy/eck-elastic-agent/values.yaml

+  # Configuration of Agent, specifically used in Agent standalone mode.
+  # ref: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent.html
+  #
+  config: {}


Should we let the user set config and elasticsearchRefs explicitly? In fleet mode, using the provided example deploy/eck-elastic-agent/examples/fleet-agents.yaml, it fails with:

Error: INSTALLATION FAILED: admission webhook "elastic-agent-validation-v1alpha1.k8s.elastic.co" denied the request: Agent.agent.k8s.elastic.co "my-agents-eck-elastic-agent" is invalid: [spec.config: Invalid value: v1.Config{Data:map[string]interface {}{}}: remove config, it can't be set in fleet mode, spec.fleetServerEnabled: Invalid value: false: remove Elasticsearch reference, it can't be enabled in fleet mode when Fleet Server is not enabled as well]

I've disabled them explicitly in the examples now. We have to allow these to be adjustable, as the agent chart supports both fleet, and standalone, so config is required in some cases. Are you thinking that config should just be set to null in the default values instead of {}, and the ES ref left empty?

barkbay · 2022-09-05T13:12:34Z

deploy/eck-elastic-agent/values.yaml

+  # At least one is required of [daemonSet, deployment].
+  # ref: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elastic-agent-fleet-configuration.html#k8s-elastic-agent-fleet-configuration-chose-the-deployment-model
+  #
+  daemonSet:


How do we switch to a Deployment?
I get this error when I try to replace daemonSet with deployment in deploy/eck-elastic-agent/examples/fleet-agents.yaml:

admission webhook "elastic-agent-validation-v1alpha1.k8s.elastic.co" denied the request: Agent.agent.k8s.elastic.co "my-agents-eck-elastic-agent" is invalid: [spec.daemonSet: Forbidden: Specify either daemonSet or deployment, not both,

You'd have to set spec.daemonSet: null since it's defined in the default values. This is unfortunate, as we're trying to make the default useful, but this is never really user friendly when making modifications, as you have to set default values to null to disable them.

Update fleet-agents example to disable config, and ES refs. Update fleet-server template to explicitly set mode, and fleetServerEnabled. Add notes to eck-agent fleet-agents, and system-integration examples.

deploy/eck-agent/templates/elastic-agent.yaml

deploy/eck-fleet-server/Chart.yaml

Consistently name fleet server (removing agent)

deploy/eck-stack/README.md

deploy/eck-agent/templates/elastic-agent.yaml

deploy/eck-fleet-server/templates/fleet-server.yaml

deploy/eck-agent/templates/service-account.yaml

deploy/eck-agent/templates/cluster-role.yaml

deploy/eck-agent/templates/cluster-role-binding.yaml

Update wording in eck-stack/readme

thbkrkr · 2022-09-20T15:16:29Z

Is it intentional that labels/annotations are managed differently in the ClusterRoleBinding/ClusterRole/ServiceAccount between the two charts? The eck-agent chart supports dedicated labels/annotations but for labels we also get the labels of the agent, while for the eck-fleet-server no annotation at all and only the labels from the fleet-server.

…agent, and fleet charts.

naemono · 2022-09-20T15:25:48Z

Is it intentional that labels/annotations are managed differently in the ClusterRoleBinding/ClusterRole/ServiceAccount between the two charts? The eck-agent chart supports dedicated labels/annotations but for labels we also get the labels of the agent, while for the eck-fleet-server no annotation at all and only the labels from the fleet-server.

This absolutely was not intentional, and has been resolved. Thanks for catching that @thbkrkr. I think I'll also add additional tests around labels/annotations similar to https://github.com/elastic/cloud-on-k8s/pull/6004/files#diff-9612072d3cca4f0281e66e5bd8fdb713f35792862686355d89ff507348419028R41

…account/clusterrole/clusterrolebinding map. Adding tests for all scenarios

naemono · 2022-09-20T15:56:06Z

@thbkrkr While adding tests, I noticed that .Values.[labels|annotations], and .Values.[serviceaccount|clusterRole|clusterRoleBinding].[labels|annotations] weren't both making it into the rendered templates, and this was fixed, and tests were added for all.

deploy/eck-agent/examples/fleet-agents.yaml

deploy/eck-agent/values.yaml

deploy/eck-stack/examples/fleet-agents.yaml

deploy/eck-stack/Chart.yaml

Rename child chart in eck-stack/Chart.yaml properly.

…xisting webhook validation

…elastic#5889) ECK Elastic Agent and Fleet Helm Charts

naemono added 3 commits July 26, 2022 10:28

Adding elastic agent helm test file

3881848

naemono added the >feature Adds or discusses adding a feature to the product label Jul 26, 2022

naemono added 2 commits July 27, 2022 11:32

Update notes to reference namespace properly

d8fc140

Fix, and add additional tests for elastic-agent in eck-stack helm uni…

8b7dc65

…t tests.

naemono marked this pull request as ready for review July 27, 2022 16:33

naemono commented Jul 27, 2022

View reviewed changes

Fix spelling, and missing space.

a3c3dce

naemono added >enhancement Enhancement of existing functionality and removed >feature Adds or discusses adding a feature to the product labels Jul 28, 2022

pebrc self-assigned this Jul 29, 2022

pebrc reviewed Jul 29, 2022

View reviewed changes

deploy/eck-elastic-agent/values.yaml Outdated Show resolved Hide resolved

Change instances => agents.

7025d6c

naemono changed the title ~~ECK resources Helm chart - Elastic Agent~~ WIP: ECK resources Helm chart - Elastic Agent Aug 4, 2022

Add fleet agent example

de5b3d0

barkbay mentioned this pull request Aug 10, 2022

ECK resources Helm chart - Beats #5899

Merged

naemono added 10 commits August 10, 2022 13:45

Move agent to singular chart.

1e2a07c

Add fleet chart.

Rename files/names to be more accurate to what is being described.

10495e6

Add additional tests Updated README

rename fleetServerRef in eck-elastic-agent example

8db6e89

Add trailing newlines.

aa02f2a

Update notes to remove cluster. Update names of tests

remove additional newline

5550304

Add additional sources for fleet server

1085fa0

remove icon for fleet server

Add more ending newlines

af5559a

Update required message for fleet server agent

e4913b9

Update fleet server agent values to include 'Fleet Server Agent'

a427481

Update eck-stack chart.yaml

d2504d3

barkbay reviewed Sep 5, 2022

View reviewed changes

naemono added 3 commits September 6, 2022 08:49

Rename agent helm chart.

632e752

Update fleet-agents example to disable config, and ES refs. Update fleet-server template to explicitly set mode, and fleetServerEnabled. Add notes to eck-agent fleet-agents, and system-integration examples.

Add license files to helm charts

43728e4

Merge branch 'main' into 5505-agent-eck-resource-helm-chart-v2

6130a21

thbkrkr reviewed Sep 13, 2022

View reviewed changes

deploy/eck-agent/templates/elastic-agent.yaml Outdated Show resolved Hide resolved

deploy/eck-fleet-server/Chart.yaml Show resolved Hide resolved

naemono added 3 commits September 13, 2022 10:49

Merge branch 'main' into 5505-agent-eck-resource-helm-chart-v2

58ddca9

Remove '-' in template.

cf1df80

Consistently name fleet server (removing agent)

Ensure label helper is using toyaml

119641c

thbkrkr self-assigned this Sep 20, 2022

thbkrkr reviewed Sep 20, 2022

View reviewed changes

Ensure using nindent over indent.

28c823e

Update wording in eck-stack/readme

Make ClusterRoleBinding/ClusterRole/ServiceAccount consistent across …

cfa7ed1

…agent, and fleet charts.

Allowing labels/annotations on the root of values, and inside service…

254ae9e

…account/clusterrole/clusterrolebinding map. Adding tests for all scenarios

Fixing '$' in include.

06dc8fe

pebrc reviewed Sep 20, 2022

View reviewed changes

naemono added 2 commits September 20, 2022 12:24

Remove defaults that are in values from examples.

c916f15

Rename child chart in eck-stack/Chart.yaml properly.

Adding newlines to test files

810fa3b

thbkrkr added the v2.5.0 label Sep 22, 2022

Move agent.spec.config default value to null, instead of {} to pass e…

22ffb2c

…xisting webhook validation

pebrc approved these changes Sep 23, 2022

View reviewed changes

naemono merged commit 4569ef2 into elastic:main Sep 26, 2022

naemono deleted the 5505-agent-eck-resource-helm-chart-v2 branch September 26, 2022 18:25

pebrc added >feature Adds or discusses adding a feature to the product and removed >enhancement Enhancement of existing functionality labels Oct 5, 2022

naemono mentioned this pull request Nov 18, 2022

Increment ECK-stack Helm chart version to support addition of Agent/Fleet Server #6179

Merged

fantapsody pushed a commit to fantapsody/cloud-on-k8s that referenced this pull request Feb 7, 2023

ECK resources Helm chart - Elastic Agent & Elastic Fleet Server Agent (…

6ab705f

…elastic#5889) ECK Elastic Agent and Fleet Helm Charts

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ECK resources Helm chart - Elastic Agent & Elastic Fleet Server Agent #5889

ECK resources Helm chart - Elastic Agent & Elastic Fleet Server Agent #5889

naemono commented Jul 26, 2022 •

edited

Loading

naemono Jul 27, 2022

pebrc left a comment

naemono commented Jul 29, 2022

pebrc commented Aug 1, 2022 •

edited

Loading

naemono commented Aug 4, 2022

barkbay Sep 5, 2022

naemono Sep 6, 2022

barkbay Sep 5, 2022

naemono Sep 6, 2022

thbkrkr commented Sep 20, 2022

naemono commented Sep 20, 2022

naemono commented Sep 20, 2022

ECK resources Helm chart - Elastic Agent & Elastic Fleet Server Agent #5889

ECK resources Helm chart - Elastic Agent & Elastic Fleet Server Agent #5889

Conversation

naemono commented Jul 26, 2022 • edited Loading

In Progress

naemono Jul 27, 2022

Choose a reason for hiding this comment

pebrc left a comment

Choose a reason for hiding this comment

naemono commented Jul 29, 2022

pebrc commented Aug 1, 2022 • edited Loading

naemono commented Aug 4, 2022

barkbay Sep 5, 2022

Choose a reason for hiding this comment

naemono Sep 6, 2022

Choose a reason for hiding this comment

barkbay Sep 5, 2022

Choose a reason for hiding this comment

naemono Sep 6, 2022

Choose a reason for hiding this comment

thbkrkr commented Sep 20, 2022

naemono commented Sep 20, 2022

naemono commented Sep 20, 2022

naemono commented Jul 26, 2022 •

edited

Loading

pebrc commented Aug 1, 2022 •

edited

Loading