ECK resources Helm chart - Beats

deploy/README.md

@@ -46,6 +46,37 @@ To see all resources installed by the helm chart
 kubectl get elastic -l "app.kubernetes.io/instance"=es-kb-quickstart -n elastic-stack
 ```
 
+## ECK Helm Chart Development
+
+### ECK Helm Chart test suite
+
+[Helm UnitTest Plugin](https://github.com/quintush/helm-unittest) is used to ensure Helm Charts render properly.
+
+#### Installation
+
+```
+helm plugin install https://github.com/quintush/helm-unittest --version 0.2.8
+```
+
+#### Running Test Suite
+
+The test suite can be run from the Makefile in the root of the project with the following command:
+
+```
+make helm-test
+```
+
+*Note* that the Makefile target runs the script in `{root}/hack/helm/test.sh`
+
+#### Manually invoking the Helm Unit Tests for a particular Chart
+
+The Helm unit tests can be manually invoked for any of the charts with the following command:
+
+```
+cd deploy/eck-stack
+helm unittest -3 -f 'templates/tests/*.yaml' .
+``````
+
 ## Licensing
 
 The ECK Helm Charts are licensed under the [Elastic License 2.0](https://www.elastic.co/licensing/elastic-license) like the operator, but require different subscription levels.

deploy/eck-beats/.helmignore

@@ -0,0 +1,24 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/
+templates/tests

deploy/eck-beats/Chart.yaml

@@ -0,0 +1,11 @@
+apiVersion: v2
+name: eck-beats
+description: A Helm chart to deploy Elastic Beats managed by the ECK Operator.
+# Requirement comes from minimum version supported for eck-operator (https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s_supported_versions.html)
+kubeVersion: ">= 1.20.0-0"
+type: application
+version: 0.1.0
+sources:
+  - https://github.com/elastic/cloud-on-k8s
+  - https://github.com/elastic/beats
+icon: https://helm.elastic.co/icons/beats.png

deploy/eck-beats/LICENSE

@@ -0,0 +1,93 @@
+Elastic License 2.0
+
+URL: https://www.elastic.co/licensing/elastic-license
+
+## Acceptance
+
+By using the software, you agree to all of the terms and conditions below.
+
+## Copyright License
+
+The licensor grants you a non-exclusive, royalty-free, worldwide,
+non-sublicensable, non-transferable license to use, copy, distribute, make
+available, and prepare derivative works of the software, in each case subject to
+the limitations and conditions below.
+
+## Limitations
+
+You may not provide the software to third parties as a hosted or managed
+service, where the service provides users with access to any substantial set of
+the features or functionality of the software.
+
+You may not move, change, disable, or circumvent the license key functionality
+in the software, and you may not remove or obscure any functionality in the
+software that is protected by the license key.
+
+You may not alter, remove, or obscure any licensing, copyright, or other notices
+of the licensor in the software. Any use of the licensor’s trademarks is subject
+to applicable law.
+
+## Patents
+
+The licensor grants you a license, under any patent claims the licensor can
+license, or becomes able to license, to make, have made, use, sell, offer for
+sale, import and have imported the software, in each case subject to the
+limitations and conditions in this license. This license does not cover any
+patent claims that you cause to be infringed by modifications or additions to
+the software. If you or your company make any written claim that the software
+infringes or contributes to infringement of any patent, your patent license for
+the software granted under these terms ends immediately. If your company makes
+such a claim, your patent license ends immediately for work on behalf of your
+company.
+
+## Notices
+
+You must ensure that anyone who gets a copy of any part of the software from you
+also gets a copy of these terms.
+
+If you modify the software, you must include in any modified copies of the
+software prominent notices stating that you have modified the software.
+
+## No Other Rights
+
+These terms do not imply any licenses other than those expressly granted in
+these terms.
+
+## Termination
+
+If you use the software in violation of these terms, such use is not licensed,
+and your licenses will automatically terminate. If the licensor provides you
+with a notice of your violation, and you cease all violation of this license no
+later than 30 days after you receive that notice, your licenses will be
+reinstated retroactively. However, if you violate these terms after such
+reinstatement, any additional violation of these terms will cause your licenses
+to terminate automatically and permanently.
+
+## No Liability
+
+*As far as the law allows, the software comes as is, without any warranty or
+condition, and the licensor will not be liable to you for any damages arising
+out of these terms or the use or nature of the software, under any kind of
+legal claim.*
+
+## Definitions
+
+The **licensor** is the entity offering these terms, and the **software** is the
+software the licensor makes available under these terms, including any portion
+of it.
+
+**you** refers to the individual or entity agreeing to these terms.
+
+**your company** is any legal entity, sole proprietorship, or other kind of
+organization that you work for, plus all organizations that have control over,
+are under the control of, or are under common control with that
+organization. **control** means ownership of substantially all the assets of an
+entity, or the power to direct its management and policies by vote, contract, or
+otherwise. Control can be direct or indirect.
+
+**your licenses** are all the licenses granted to you for the software under
+these terms.
+
+**use** means anything you do with the software requiring one of your licenses.
+
+**trademark** means trademarks, service marks, and similar rights.

deploy/eck-beats/examples/auditbeat_hosts.yaml

@@ -0,0 +1,111 @@
+name: auditbeat
+version: 8.3.3
+spec:
+  type: auditbeat
+  elasticsearchRef:
+    name: elasticsearch
+  kibanaRef:
+    name: kibana
+  config:
+    auditbeat.modules:
+    - module: file_integrity
+      paths:
+      - /hostfs/bin
+      - /hostfs/usr/bin
+      - /hostfs/sbin
+      - /hostfs/usr/sbin
+      - /hostfs/etc
+      exclude_files:
+      - '(?i)\.sw[nop]$'
+      - '~$'
+      - '/\.git($|/)'
+      scan_at_start: true
+      scan_rate_per_sec: 50 MiB
+      max_file_size: 100 MiB
+      hash_types: [sha1]
+      recursive: true
+    - module: auditd
+      audit_rules: |
+        # Executions
+        -a always,exit -F arch=b64 -S execve,execveat -k exec
+
+        # Unauthorized access attempts (amd64 only)
+        -a always,exit -F arch=b64 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EACCES -k access
+        -a always,exit -F arch=b64 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EPERM -k access
+
+    processors:
+    - add_cloud_metadata: {}
+    - add_host_metadata: {}
+    - add_process_metadata:
+        match_pids: ['process.pid']
+  daemonSet:
+    podTemplate:
+      spec:
+        hostPID: true  # Required by auditd module
+        dnsPolicy: ClusterFirstWithHostNet
+        hostNetwork: true # Allows to provide richer host metadata
+        automountServiceAccountToken: true # some older Beat versions are depending on this settings presence in k8s context
+        securityContext:
+          runAsUser: 0
+        volumes:
+        - name: bin
+          hostPath:
+            path: /bin
+        - name: usrbin
+          hostPath:
+            path: /usr/bin
+        - name: sbin
+          hostPath:
+            path: /sbin
+        - name: usrsbin
+          hostPath:
+            path: /usr/sbin
+        - name: etc
+          hostPath:
+            path: /etc
+        - name: run-containerd
+          hostPath:
+            path: /run/containerd
+            type: DirectoryOrCreate
+        # Uncomment the below when running on GKE. See https://github.com/elastic/beats/issues/8523 for more context.
+        #- name: run
+        #  hostPath:
+        #    path: /run
+        #initContainers:
+        #- name: cos-init
+        #  image: docker.elastic.co/beats/auditbeat:8.3.3
+        #  volumeMounts:
+        #  - name: run
+        #    mountPath: /run
+        #  command: ['sh', '-c', 'export SYSTEMD_IGNORE_CHROOT=1 && systemctl stop systemd-journald-audit.socket && systemctl mask systemd-journald-audit.socket && systemctl restart systemd-journald']
+        containers:
+        - name: auditbeat
+          securityContext:
+            capabilities:
+              add:
+              # Capabilities needed for auditd module
+              - 'AUDIT_READ'
+              - 'AUDIT_WRITE'
+              - 'AUDIT_CONTROL'
+          volumeMounts:
+          - name: bin
+            mountPath: /hostfs/bin
+            readOnly: true
+          - name: sbin
+            mountPath: /hostfs/sbin
+            readOnly: true
+          - name: usrbin
+            mountPath: /hostfs/usr/bin
+            readOnly: true
+          - name: usrsbin
+            mountPath: /hostfs/usr/sbin
+            readOnly: true
+          - name: etc
+            mountPath: /hostfs/etc
+            readOnly: true
+          # Directory with root filesystems of containers executed with containerd, this can be
+          # different with other runtimes. This volume is needed to monitor the file integrity
+          # of files in containers.
+          - name: run-containerd
+            mountPath: /run/containerd
+            readOnly: true

deploy/eck-beats/examples/filebeat_no_autodiscover.yaml

@@ -0,0 +1,49 @@
+apiVersion: beat.k8s.elastic.co/v1beta1
+kind: Beat
+metadata:
+  name: filebeat
+spec:
+  type: filebeat
+  version: 8.3.3
+  elasticsearchRef:
+    name: elasticsearch
+  kibanaRef:
+    name: kibana
+  config:
+    filebeat.inputs:
+    - type: container
+      paths:
+      - /var/log/containers/*.log
+    processors:
+    - add_host_metadata: {}
+    - add_cloud_metadata: {}
+  daemonSet:
+    podTemplate:
+      spec:
+        automountServiceAccountToken: true
+        terminationGracePeriodSeconds: 30
+        dnsPolicy: ClusterFirstWithHostNet
+        hostNetwork: true # Allows to provide richer host metadata
+        containers:
+        - name: filebeat
+          securityContext:
+            runAsUser: 0
+            # If using Red Hat OpenShift uncomment this:
+            #privileged: true
+          volumeMounts:
+          - name: varlogcontainers
+            mountPath: /var/log/containers
+          - name: varlogpods
+            mountPath: /var/log/pods
+          - name: varlibdockercontainers
+            mountPath: /var/lib/docker/containers
+        volumes:
+        - name: varlogcontainers
+          hostPath:
+            path: /var/log/containers
+        - name: varlogpods
+          hostPath:
+            path: /var/log/pods
+        - name: varlibdockercontainers
+          hostPath:
+            path: /var/lib/docker/containers

deploy/eck-beats/examples/heartbeat_es_kb_health.yaml

@@ -0,0 +1,24 @@
+name: heartbeat
+version: 8.3.3
+spec:
+  type: heartbeat
+  elasticsearchRef:
+    name: elasticsearch
+  config:
+    heartbeat.monitors:
+    - type: tcp
+      schedule: '@every 5s'
+      # This should directly match the name of the Elasticsearch instance
+      # with "-es-http" appended to the name.
+      hosts: ["elasticsearch-es-http.default.svc:9200"]
+    - type: tcp
+      schedule: '@every 5s'
+      # This should directly match the names of the Kibana instance
+      # with "-kb-http" appended to the name.
+      hosts: ["eck-kibana-kb-http.default.svc:5601"]
+  deployment:
+    replicas: 1
+    podTemplate:
+      spec:
+        securityContext:
+          runAsUser: 0