ECK resources Helm chart - Beats #5899
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # Patterns to ignore when building packages. | ||
| # This supports shell glob matching, relative path matching, and | ||
| # negation (prefixed with !). Only one pattern per line. | ||
| .DS_Store | ||
| # Common VCS dirs | ||
| .git/ | ||
| .gitignore | ||
| .bzr/ | ||
| .bzrignore | ||
| .hg/ | ||
| .hgignore | ||
| .svn/ | ||
| # Common backup files | ||
| *.swp | ||
| *.bak | ||
| *.tmp | ||
| *.orig | ||
| *~ | ||
| # Various IDEs | ||
| .project | ||
| .idea/ | ||
| *.tmproj | ||
| .vscode/ | ||
| templates/tests |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| apiVersion: v2 | ||
| name: eck-beats | ||
| description: A Helm chart to deploy Elastic Beats managed by the ECK Operator. | ||
| kubeVersion: ">= 1.20.0-0" | ||
| type: application | ||
| version: 0.1.0 | ||
naemono marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| sources: | ||
| - https://github.com/elastic/cloud-on-k8s | ||
| - https://github.com/elastic/beats | ||
| icon: https://helm.elastic.co/icons/beats.png | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| name: auditbeat | ||
| type: auditbeat | ||
| version: 8.2.3 | ||
|
There was a problem hiding this comment. There are a lot of references to what, I guess, should be a recent stack version. Should we try to use There was a problem hiding this comment. I've updated this script, and made the appropriate changes to the versions in this PR, but am hesitant to update the other charts in this PR, as we'd need to bump the chart versions when this happens. |
||
| spec: | ||
| elasticsearchRef: | ||
| name: elasticsearch | ||
|
There was a problem hiding this comment. Should we use the default names Elasticsearch and Kibana get when using the corresponding Helm charts? There was a problem hiding this comment. I've updated this per most comments, but want to check into this further tomorrow, as these names are dependent on the name of the chart itself, unless doing a full name override, so I'll want to ensure this is consistent with the other charts. There was a problem hiding this comment. @pebrc I've updated all values and all examples across all charts to be consistent for naming and updated all versions to latest stack 8.5.0, along with updating all tests to match/pass. I've also bumped the versions of all charts where stack version changes were made, and updated the eck-stack Chart.yaml's dependency versions to match. This should be ready for another set of 👀 |
||
| kibanaRef: | ||
| name: kibana | ||
| config: | ||
| # Since filebeat is used in the default values, this needs to be removed with an empty list. | ||
|
There was a problem hiding this comment. This is mentioned in several places, does it not feel a bit odd that the user must explicitly disable a There was a problem hiding this comment. Over the course of the previous Helm Chart PRs, we've tried to maintain that installing with the default values gives some sort of useful "default" installation. Though just recently, in the agent Helm Charts PR, the There was a problem hiding this comment. Sorry for the very late answer. I need to have a look at what is proposed for Agent. Defaulting to There was a problem hiding this comment. I see why defaulting to There was a problem hiding this comment. actually @barkbay, I remember why I defaulted to filebeat once I went back and looked at this. By defaulting to filebeat, we add the minimum amount to the default values and still have a useful "default" Beat install. If we move this to metricbeat, many more default values would be needed to be added to make a useful default Beat install. Let me know your thoughts here. |
||
| filebeat.inputs: [] | ||
| auditbeat.modules: | ||
| - module: file_integrity | ||
| paths: | ||
| - /hostfs/bin | ||
| - /hostfs/usr/bin | ||
| - /hostfs/sbin | ||
| - /hostfs/usr/sbin | ||
| - /hostfs/etc | ||
| exclude_files: | ||
| - '(?i)\.sw[nop]$' | ||
| - '~$' | ||
| - '/\.git($|/)' | ||
| scan_at_start: true | ||
| scan_rate_per_sec: 50 MiB | ||
| max_file_size: 100 MiB | ||
| hash_types: [sha1] | ||
| recursive: true | ||
| - module: auditd | ||
| audit_rules: | | ||
| # Executions | ||
| -a always,exit -F arch=b64 -S execve,execveat -k exec | ||
|
|
||
| # Unauthorized access attempts (amd64 only) | ||
| -a always,exit -F arch=b64 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EACCES -k access | ||
| -a always,exit -F arch=b64 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EPERM -k access | ||
|
|
||
| processors: | ||
| - add_cloud_metadata: {} | ||
| - add_host_metadata: {} | ||
| - add_process_metadata: | ||
| match_pids: ['process.pid'] | ||
| daemonSet: | ||
| podTemplate: | ||
| spec: | ||
| hostPID: true # Required by auditd module | ||
| dnsPolicy: ClusterFirstWithHostNet | ||
| hostNetwork: true # Allows to provide richer host metadata | ||
| automountServiceAccountToken: true # some older Beat versions are depending on this settings presence in k8s context | ||
| securityContext: | ||
| runAsUser: 0 | ||
| volumes: | ||
| - name: bin | ||
| hostPath: | ||
| path: /bin | ||
| - name: usrbin | ||
| hostPath: | ||
| path: /usr/bin | ||
| - name: sbin | ||
| hostPath: | ||
| path: /sbin | ||
| - name: usrsbin | ||
| hostPath: | ||
| path: /usr/sbin | ||
| - name: etc | ||
| hostPath: | ||
| path: /etc | ||
| - name: run-containerd | ||
| hostPath: | ||
| path: /run/containerd | ||
| type: DirectoryOrCreate | ||
| # Uncomment the below when running on GKE. See https://github.com/elastic/beats/issues/8523 for more context. | ||
| #- name: run | ||
| # hostPath: | ||
| # path: /run | ||
| #initContainers: | ||
| #- name: cos-init | ||
| # image: docker.elastic.co/beats/auditbeat:8.3.1 | ||
naemono marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # volumeMounts: | ||
| # - name: run | ||
| # mountPath: /run | ||
| # command: ['sh', '-c', 'export SYSTEMD_IGNORE_CHROOT=1 && systemctl stop systemd-journald-audit.socket && systemctl mask systemd-journald-audit.socket && systemctl restart systemd-journald'] | ||
| containers: | ||
| - name: auditbeat | ||
| securityContext: | ||
| capabilities: | ||
| add: | ||
| # Capabilities needed for auditd module | ||
| - 'AUDIT_READ' | ||
| - 'AUDIT_WRITE' | ||
| - 'AUDIT_CONTROL' | ||
| volumeMounts: | ||
| - name: bin | ||
| mountPath: /hostfs/bin | ||
| readOnly: true | ||
| - name: sbin | ||
| mountPath: /hostfs/sbin | ||
| readOnly: true | ||
| - name: usrbin | ||
| mountPath: /hostfs/usr/bin | ||
| readOnly: true | ||
| - name: usrsbin | ||
| mountPath: /hostfs/usr/sbin | ||
| readOnly: true | ||
| - name: etc | ||
| mountPath: /hostfs/etc | ||
| readOnly: true | ||
| # Directory with root filesystems of containers executed with containerd, this can be | ||
| # different with other runtimes. This volume is needed to monitor the file integrity | ||
| # of files in containers. | ||
| - name: run-containerd | ||
| mountPath: /run/containerd | ||
| readOnly: true | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| name: heartbeat | ||
| type: heartbeat | ||
| version: 8.2.3 | ||
| spec: | ||
| elasticsearchRef: | ||
| name: elasticsearch | ||
| config: | ||
| # Since filebeat is used in the default values, this needs to be removed with an empty list. | ||
| filebeat.inputs: [] | ||
| # Since rpcoessors is used in the default values, this needs to be removed with an empty list. | ||
naemono marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| processors: [] | ||
| heartbeat.monitors: | ||
| - type: tcp | ||
| schedule: '@every 5s' | ||
| # This should directly match the name of the Elasticsearch instance | ||
| # with "-es-http" appended to the name. | ||
| hosts: ["elasticsearch-es-http.default.svc:9200"] | ||
| - type: tcp | ||
| schedule: '@every 5s' | ||
| # This should directly match the names of the Kibana instance | ||
| # with "-kb-http" appended to the name. | ||
| hosts: ["eck-kibana-kb-http.default.svc:5601"] | ||
| deployment: | ||
| replicas: 1 | ||
| podTemplate: | ||
| spec: | ||
| securityContext: | ||
| runAsUser: 0 | ||
| # Since there is an existing daemonSet in the default values, it must be moved by using 'null'. | ||
| daemonSet: null | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,162 @@ | ||
| name: metricbeat | ||
| spec: | ||
| type: metricbeat | ||
| version: 8.2.3 | ||
| elasticsearchRef: | ||
| name: elasticsearch | ||
| kibanaRef: | ||
| name: kibana | ||
| config: | ||
| # Since filebeat is used in the default values, this needs to be removed with an empty list. | ||
| filebeat.inputs: [] | ||
| metricbeat: | ||
| autodiscover: | ||
| providers: | ||
| - hints: | ||
| default_config: {} | ||
| enabled: "true" | ||
| node: ${NODE_NAME} | ||
| type: kubernetes | ||
| modules: | ||
| - module: system | ||
| period: 10s | ||
| metricsets: | ||
| - cpu | ||
| - load | ||
| - memory | ||
| - network | ||
| - process | ||
| - process_summary | ||
| process: | ||
| include_top_n: | ||
| by_cpu: 5 | ||
| by_memory: 5 | ||
| processes: | ||
| - .* | ||
| - module: system | ||
| period: 1m | ||
| metricsets: | ||
| - filesystem | ||
| - fsstat | ||
| processors: | ||
| - drop_event: | ||
| when: | ||
| regexp: | ||
| system: | ||
| filesystem: | ||
| mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib)($|/) | ||
| - module: kubernetes | ||
| period: 10s | ||
| node: ${NODE_NAME} | ||
| hosts: | ||
| - https://${NODE_NAME}:10250 | ||
| bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token | ||
| ssl: | ||
| verification_mode: none | ||
| metricsets: | ||
| - node | ||
| - system | ||
| - pod | ||
| - container | ||
| - volume | ||
| processors: | ||
| - add_cloud_metadata: {} | ||
| - add_host_metadata: {} | ||
| daemonSet: | ||
| podTemplate: | ||
| spec: | ||
| serviceAccountName: metricbeat | ||
| automountServiceAccountToken: true # some older Beat versions are depending on this settings presence in k8s context | ||
| containers: | ||
| - args: | ||
| - -e | ||
| - -c | ||
| - /etc/beat.yml | ||
| - -system.hostfs=/hostfs | ||
| name: metricbeat | ||
| volumeMounts: | ||
| - mountPath: /hostfs/sys/fs/cgroup | ||
| name: cgroup | ||
| - mountPath: /var/run/docker.sock | ||
| name: dockersock | ||
| - mountPath: /hostfs/proc | ||
| name: proc | ||
| env: | ||
| - name: NODE_NAME | ||
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: spec.nodeName | ||
| dnsPolicy: ClusterFirstWithHostNet | ||
| hostNetwork: true # Allows to provide richer host metadata | ||
| securityContext: | ||
| runAsUser: 0 | ||
| terminationGracePeriodSeconds: 30 | ||
| volumes: | ||
| - hostPath: | ||
| path: /sys/fs/cgroup | ||
| name: cgroup | ||
| - hostPath: | ||
| path: /var/run/docker.sock | ||
| name: dockersock | ||
| - hostPath: | ||
| path: /proc | ||
| name: proc | ||
|
|
||
| clusterRole: | ||
| # permissions needed for metricbeat | ||
| # source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-kubernetes.html | ||
| name: metricbeat | ||
| rules: | ||
| - apiGroups: | ||
| - "" | ||
| resources: | ||
| - nodes | ||
| - namespaces | ||
| - events | ||
| - pods | ||
| verbs: | ||
| - get | ||
| - list | ||
| - watch | ||
| - apiGroups: | ||
| - "extensions" | ||
| resources: | ||
| - replicasets | ||
| verbs: | ||
| - get | ||
| - list | ||
| - watch | ||
| - apiGroups: | ||
| - apps | ||
| resources: | ||
| - statefulsets | ||
| - deployments | ||
| - replicasets | ||
| verbs: | ||
| - get | ||
| - list | ||
| - watch | ||
| - apiGroups: | ||
| - "" | ||
| resources: | ||
| - nodes/stats | ||
| verbs: | ||
| - get | ||
| - nonResourceURLs: | ||
| - /metrics | ||
| verbs: | ||
| - get | ||
|
|
||
| serviceAccount: | ||
| name: metricbeat | ||
|
|
||
| clusterRoleBinding: | ||
| name: metricbeat | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: metricbeat | ||
| namespace: default | ||
naemono marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| roleRef: | ||
| kind: ClusterRole | ||
| name: metricbeat | ||
| apiGroup: rbac.authorization.k8s.io | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| name: packetbeat | ||
| spec: | ||
| type: packetbeat | ||
| version: 8.2.3 | ||
| elasticsearchRef: | ||
| name: elasticsearch | ||
| kibanaRef: | ||
| name: kibana | ||
| config: | ||
| # Since filebeat is used in the default values, this needs to be removed with an empty list. | ||
| filebeat.inputs: [] | ||
| packetbeat.interfaces.device: any | ||
| packetbeat.protocols: | ||
| - type: dns | ||
| ports: [53] | ||
| include_authorities: true | ||
| include_additionals: true | ||
| - type: http | ||
| ports: [80, 8000, 8080, 9200] | ||
| packetbeat.flows: | ||
| timeout: 30s | ||
| period: 10s | ||
| processors: | ||
| - add_cloud_metadata: {} | ||
| - add_host_metadata: {} | ||
| daemonSet: | ||
| podTemplate: | ||
| spec: | ||
| terminationGracePeriodSeconds: 30 | ||
| hostNetwork: true | ||
| automountServiceAccountToken: true # some older Beat versions are depending on this settings presence in k8s context | ||
| dnsPolicy: ClusterFirstWithHostNet | ||
| containers: | ||
| - name: packetbeat | ||
| securityContext: | ||
| runAsUser: 0 | ||
| capabilities: | ||
| add: | ||
| - NET_ADMIN | ||
| volumes: [] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
|
|
||
| 1. Check Beat status | ||
| $ kubectl get beat {{ include "beat.fullname" . }} -n {{ .Release.Namespace }} | ||
|
|
||
| 2. Check Beat pod status | ||
| $ kubectl get pods --namespace={{ .Release.Namespace }} -l beat.k8s.elastic.co/name={{ include "beat.fullname" . }} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same question here, how do maintain this version? (It is also mentioned in
deploy/eck-stack/README.md, I think it feels a bit redundant with what's in the mainREADME.md)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I updated this to include the reasoning behind this requirement, as I think it's important. This would need to be updated during the normal release process for the eck-operator, when the requirement changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If a user is using a version of the operator that still supports a previous k8s version (for example ECK
2.2which still supports K8S1.19), but the user needs to update the Helm Chart: does it prevent the user to use that new version of the Chart on1.19? I'm just trying to understand how the min. k8s version supported by the operator and the min. k8s version supported by a Chart are tied.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If a user was running eck
2.2on K8S1.19and attempted to use these Helm Charts, they would fail (helm would throw a failure) because of this K8S version requirement in the Chart.yaml.