Add k8s keystore backend #18096

ChrsMark · 2020-04-29T14:38:24Z

What does this PR do?

This PR implements a new Keystore Backend, which is used to retrieve k8s secrets and use them in configurations provided by hints-based autodiscover feature. Any hints based configuration has only access to the secrets of the same namespace of the pod by which was triggered.

We still need to decide if this keystone should be used along with the already used FileKeystore in static autodiscover configurations.

Why is it important?

This is important for the users so as to avoid having sensitive data on hints' configurations and being able to leverage k8s secrets for that purpose.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Test k8s keystore backend with hints autodiscover

Create the k8s secret:

cat <<EOF | kubectl apply -f -                                                                                                                   
apiVersion: v1
kind: Secret
metadata:
  name: somesecret
type: Opaque
data:
  value: $(echo -n "passpass" | base64)
EOF

Start the Redis sever as hints autodiscover target:

cat <<EOF | kubectl apply -f -                                                                                                                   
apiVersion: v1
kind: Service
metadata:
  name: redis
  labels:
    app: redis
spec:
  clusterIP: None
  ports:
  - name: web
    port: 6379
    protocol: TCP
  selector:
    app: redis
  type: ClusterIP
---
apiVersion: v1
kind: Pod
metadata:
  name: redis
  labels:
    role: main
    app: redis
  annotations:
    co.elastic.metrics/module: redis
    co.elastic.metrics/hosts: '${data.host}:6379'
    co.elastic.metrics/password: "${kubernetes.default.somesecret.value}"
spec:
  containers:
    - name: redis
      image: redis
      command:
        - redis-server
        - "--requirepass 'passpass'"
EOF

Configure Metricbeat with hints autodiscover enabled:

metricbeat.autodiscover:
  providers:
    - type: kubernetes
      node: ${NODE_NAME}
      hints.enabled: true

Start metricbeat and ensure that Redis module is properly started and collecting metrics from the Redis server using the password provided via the k8s secret.
Use https://github.com/elastic/beats/blob/master/deploy/kubernetes/metricbeat-kubernetes.yaml but make sure that secrets is added in the list of resources in ClusterRole.
Redeploy redis Pod with a different password to check that it will fail to authenticate. Change: --requirepass 'passpass2'

Test k8s keystore backend with static autodiscover templates

For Secret creation and Redis target pod use the steps from the previous scenario.

Configure Metricbeat with static autodiscover templates:

metricbeat.autodiscover:
  providers:
    - type: kubernetes
      node: ${NODE_NAME}
      templates:
        - condition:
            contains:
              kubernetes.pod.name: "redis"
          config:
            - module: redis
              hosts: "${data.host}:6379"
              password: "${kubernetes.default.somesecret.value}"

Start Metricbeat and check that Redis module successfully collects metrics.
Deploy redis pod with a different password to make metricset fail to authenticate.

Test basic keystore with static autodiscover templates

Create the File Keystore. pod start Redis with passpass:

./metricbeat keystore create                                                                                 
Created metricbeat keystore
./metricbeat keystore add REDIS_PASSWORD                                                                      
Enter value for REDIS_PASSWORD: 
Successfully updated the keystore

Metricbeat Configuration

metricbeat.autodiscover:
  providers:
    - type: kubernetes
      node: ${NODE_NAME}
      templates:
        - condition:
            contains:
              kubernetes.pod.name: "redis"
          config:
            - module: redis
              hosts: "${data.host}:6379"
              password: "${REDIS_PASSWORD}"

Start Metricbeat and expect to see the module to collect metrics
Try to start Redis pod with a different password so as to make metricset fail to collect metrics.

Related issues

cc: @exekias this one is still in progress and needs cleanups, docs and error handling but the approach is not expected to change.

TODOs:

Add docs
Add tests
Remove sensitive logging

Signed-off-by: chrismark <chrismarkou92@gmail.com>

elasticmachine · 2020-04-29T14:45:31Z

❕ Build Aborted

Either there was a build timeout or someone aborted the build.'}

Expand to view the summary

Build stats

Build Cause: [Branch indexing]
Start Time: 2020-05-13T15:26:45.671+0000
Duration: 124 min 9 sec (7449330)

Steps errors

Expand to view the steps failures

Name: Report to Codecov
- Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done
- Result: FAILURE
- Duration: 2 min 14 sec
- Start Time: 2020-05-13T16:44:33.494+0000
- log

Log output

Expand to view the last 100 lines of log output

[2020-05-13T16:47:07.757Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:07.760Z] Still waiting to schedule task
[2020-05-13T16:47:07.761Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:07.764Z] Still waiting to schedule task
[2020-05-13T16:47:07.764Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:07.849Z] Still waiting to schedule task
[2020-05-13T16:47:07.849Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:07.853Z] Still waiting to schedule task
[2020-05-13T16:47:07.853Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:07.860Z] Still waiting to schedule task
[2020-05-13T16:47:07.860Z] Waiting for next available executor on ‘macosx’
[2020-05-13T16:47:07.933Z] Still waiting to schedule task
[2020-05-13T16:47:07.934Z] All nodes of label ‘windows-immutable&&windows-2019’ are offline
[2020-05-13T16:47:07.937Z] Still waiting to schedule task
[2020-05-13T16:47:07.937Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:07.941Z] Still waiting to schedule task
[2020-05-13T16:47:07.941Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:08.010Z] Still waiting to schedule task
[2020-05-13T16:47:08.011Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:08.015Z] Still waiting to schedule task
[2020-05-13T16:47:08.015Z] All nodes of label ‘windows-immutable&&windows-2019’ are offline
[2020-05-13T16:47:08.151Z] Still waiting to schedule task
[2020-05-13T16:47:08.151Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:08.155Z] Still waiting to schedule task
[2020-05-13T16:47:08.156Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:08.158Z] Still waiting to schedule task
[2020-05-13T16:47:08.159Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T16:47:08.161Z] Still waiting to schedule task
[2020-05-13T16:47:08.161Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T17:30:22.539Z] Cancelling nested steps due to timeout
[2020-05-13T17:30:30.016Z] Failed in branch Elastic Agent x-pack
[2020-05-13T17:30:30.018Z] Failed in branch Elastic Agent x-pack Windows
[2020-05-13T17:30:30.019Z] Failed in branch Elastic Agent Mac OS X
[2020-05-13T17:30:30.020Z] Failed in branch Filebeat oss
[2020-05-13T17:30:30.021Z] Failed in branch Filebeat x-pack
[2020-05-13T17:30:30.023Z] Failed in branch Filebeat Mac OS X
[2020-05-13T17:30:30.025Z] Failed in branch Filebeat Windows
[2020-05-13T17:30:30.027Z] Failed in branch Heartbeat
[2020-05-13T17:30:30.028Z] Failed in branch Auditbeat oss
[2020-05-13T17:30:30.041Z] Failed in branch Auditbeat x-pack
[2020-05-13T17:30:30.048Z] Failed in branch Libbeat
[2020-05-13T17:30:30.059Z] Failed in branch Libbeat x-pack
[2020-05-13T17:30:30.060Z] Failed in branch Metricbeat OSS Unit tests
[2020-05-13T17:30:30.061Z] Failed in branch Metricbeat OSS Integration tests
[2020-05-13T17:30:30.064Z] Failed in branch Metricbeat Python integration tests
[2020-05-13T17:30:30.067Z] Failed in branch Metricbeat x-pack
[2020-05-13T17:30:30.072Z] Failed in branch Metricbeat crosscompile
[2020-05-13T17:30:30.074Z] Failed in branch Metricbeat Mac OS X
[2020-05-13T17:30:30.080Z] Failed in branch Metricbeat Windows
[2020-05-13T17:30:30.085Z] Failed in branch Packetbeat
[2020-05-13T17:30:30.090Z] Failed in branch dockerlogbeat
[2020-05-13T17:30:30.091Z] Failed in branch Winlogbeat
[2020-05-13T17:30:30.098Z] Failed in branch Winlogbeat Windows x-pack
[2020-05-13T17:30:30.105Z] Failed in branch Functionbeat
[2020-05-13T17:30:30.109Z] Failed in branch Journalbeat
[2020-05-13T17:30:30.113Z] Failed in branch Generators
[2020-05-13T17:30:30.114Z] Failed in branch Kubernetes
[2020-05-13T17:30:30.567Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats
[2020-05-13T17:30:31.122Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-05-13T17:30:31.207Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Lint
[2020-05-13T17:30:32.100Z] + cat
[2020-05-13T17:30:32.100Z] + /usr/local/bin/runbld ./runbld-script
[2020-05-13T17:30:32.100Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-05-13T17:30:38.586Z] runbld>>> runbld started
[2020-05-13T17:30:38.586Z] runbld>>> 1.6.11/a66728ff8f4356963772e6e6d2069392fa06acbe
[2020-05-13T17:30:41.219Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-18096' in order of occurrence in the config (last value wins).
[2020-05-13T17:30:42.031Z] runbld>>> Debug logging enabled.
[2020-05-13T17:30:42.031Z] runbld>>> Storing result
[2020-05-13T17:30:42.436Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-05-13T17:30:42.436Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200513173041-B7033A4C
[2020-05-13T17:30:42.436Z] runbld>>> Adding system facts.
[2020-05-13T17:30:43.337Z] runbld>>> Adding vcs info for the latest commit:  2f0b2dab5b9b83e06f702200fa01ccf12940ba37
[2020-05-13T17:30:43.337Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-05-13T17:30:43.337Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-05-13T17:30:43.337Z] Processing JUnit reports with runbld...
[2020-05-13T17:30:43.337Z] + echo 'Processing JUnit reports with runbld...'
[2020-05-13T17:30:43.799Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-05-13T17:30:43.799Z] runbld>>> DURATION: 14ms
[2020-05-13T17:30:43.799Z] runbld>>> STDOUT: 40 bytes
[2020-05-13T17:30:43.799Z] runbld>>> STDERR: 49 bytes
[2020-05-13T17:30:43.799Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-05-13T17:30:43.799Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats
[2020-05-13T17:30:45.202Z] runbld>>> Storing build metadata: 
[2020-05-13T17:30:45.202Z] runbld>>> Adding test report.
[2020-05-13T17:30:45.202Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats
[2020-05-13T17:30:45.997Z] runbld>>> Found 0 test output files
[2020-05-13T17:30:45.997Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 0 Skipped: 0
[2020-05-13T17:30:46.405Z] runbld>>> Storing result
[2020-05-13T17:30:46.405Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-05-13T17:30:46.405Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200513173041-B7033A4C
[2020-05-13T17:30:46.405Z] runbld>>> Email notification disabled by environment variable.
[2020-05-13T17:30:46.405Z] runbld>>> Slack notification disabled by environment variable.
[2020-05-13T17:30:53.404Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096
[2020-05-13T17:30:53.795Z] [INFO] getVaultSecret: Getting secrets
[2020-05-13T17:30:53.979Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-05-13T17:30:55.337Z] + chmod 755 generate-build-data.sh
[2020-05-13T17:30:55.337Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18096/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18096/runs/16 ABORTED 7449330
[2020-05-13T17:30:55.887Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18096/runs/16/steps/?limit=10000 -o steps-info.json
[2020-05-13T17:30:59.166Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18096/runs/16/tests/?status=FAILED -o tests-errors.json
[2020-05-13T17:30:59.166Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18096/runs/16/log/ -o pipeline-log.txt

Signed-off-by: chrismark <chrismarkou92@gmail.com>

elasticmachine · 2020-04-30T16:14:42Z

Pinging @elastic/integrations-platforms (Team:Platforms)

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark · 2020-05-26T11:56:31Z

@urso thanks for reviewing here! I ve pushed changes in your comments' direction. Let me know what you think.

Also, documentation have been added/updated along with some basic tests. I plan to add more tests about this feature in follow-up PR since I need to update k8s.io/client-go/kubernetes dependency( I prefer to not include this in this PR) in order to make use of the fake client in order to interact with "actual" k8s secrets and test the Retrieve method of the k8s keystore.

heartbeat/autodiscover/builder/hints/monitors_test.go

libbeat/autodiscover/providers/kubernetes/kubernetes.go

libbeat/k8skeystore/kubernetes_keystore.go

libbeat/keystore/keystore.go

Signed-off-by: chrismark <chrismarkou92@gmail.com>

urso

LGTM, really cool feature.

Please check CI.

elasticmachine · 2020-05-26T21:22:50Z

💔 Tests Failed

Expand to view the summary

Build stats

Build Cause: [Pull request #18096 updated]
Start Time: 2020-05-27T08:31:09.537+0000
Duration: 77 min 11 sec

Test stats 🧪

Test	Results
Failed	10
Passed	9152
Skipped	1526
Total	10688

Test errors

Expand to view the tests failures

Name: Build and Test / Filebeat Mac OS X / test_file_disappear – test_crawler.Test
- Age: 1
- Duration: 1.703
- Error Details:
Name: Build and Test / Filebeat Mac OS X / test_clean_removed_with_clean_inactive – test_registrar.Test
- Age: 1
- Duration: 1.879
- Error Details:
  -------------------- >> begin captured stdout << ---------------------
  registry size: 2
  registry size after remove: 2

--------------------- >> end captured stdout << ----------------------

Name: Build and Test / Filebeat Mac OS X / test_rotating_file_with_restart – test_registrar.Test
- Age: 1
- Duration: 5.936
- Error Details:
Name: Build and Test / Filebeat Mac OS X / test_upgrade_from_6_3_0 – test_registrar_upgrade.Test
- Age: 1
- Duration: 1.769
- Error Details:
Name: Build and Test / Filebeat Mac OS X / test_upgrade_from_6_3_1 – test_registrar_upgrade.Test
- Age: 1
- Duration: 1.866
- Error Details:
Name: Build and Test / Filebeat Mac OS X / test_upgrade_from_faulty_6_3_1 – test_registrar_upgrade.Test
- Age: 1
- Duration: 1.863
- Error Details:
Name: Build and Test / Filebeat Mac OS X / test_upgrade_from_latest – test_registrar_upgrade.Test
- Age: 1
- Duration: 1.854
- Error Details:
Name: Build and Test / Auditbeat oss / Auditbeat Mac OS X / TestEventReader – file_integrity
- Age: 1
- Duration: 2.29
- Error Details: Failed
Name: Build and Test / Auditbeat oss / Auditbeat Mac OS X / TestRaces – file_integrity
- Age: 1
- Duration: 1.25
- Error Details: Failed
Name: Build and Test / Auditbeat oss / Auditbeat Mac OS X / TestEventReader/chown – file_integrity
- Age: 4
- Duration: 1.1
- Error Details: Failed

Steps errors

Expand to view the steps failures

Name: Mage build unitTest
- Description: mage build unitTest
- Duration: 18 min 42 sec
- Start Time: 2020-05-27T08:56:06.236+0000
- log
Name: Recursively delete the current directory from the workspace
- Description: script returned exit code 1
- Duration: 1 min 59 sec
- Start Time: 2020-05-27T09:14:02.725+0000
- log
Name: Mage build unitTest
- Description: mage build unitTest
- Duration: 2 min 8 sec
- Start Time: 2020-05-27T09:12:07.140+0000
- log
Name: Recursively delete the current directory from the workspace
- Description: script returned exit code 1
- Duration: 0 min 10 sec
- Start Time: 2020-05-27T09:14:21.129+0000
- log
Name: Integration tests
- Description: MODULE=kubernetes make -C metricbeat integration-tests
- Duration: 1 min 59 sec
- Start Time: 2020-05-27T08:53:52.632+0000
- log

Log output

Expand to view the last 100 lines of log output

[2020-05-27T09:47:51.908Z] + FILE=packetbeat/build/coverage/full.cov
[2020-05-27T09:47:51.909Z] + [ -f packetbeat/build/coverage/full.cov ]
[2020-05-27T09:47:51.909Z] + FILE=winlogbeat/build/coverage/full.cov
[2020-05-27T09:47:51.909Z] + [ -f winlogbeat/build/coverage/full.cov ]
[2020-05-27T09:47:51.909Z] + FILE=journalbeat/build/coverage/full.cov
[2020-05-27T09:47:51.909Z] + [ -f journalbeat/build/coverage/full.cov ]
[2020-05-27T09:47:52.420Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats
[2020-05-27T09:47:52.733Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-05-27T09:47:52.744Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Lint
[2020-05-27T09:47:52.827Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-05-27T09:47:52.902Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Winlogbeat-oss
[2020-05-27T09:47:52.972Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-05-27T09:47:53.054Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Dockerlogbeat
[2020-05-27T09:47:53.135Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Journalbeat-oss
[2020-05-27T09:47:53.204Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-05-27T09:47:53.273Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-05-27T09:47:53.344Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-05-27T09:47:53.414Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-05-27T09:47:53.482Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-05-27T09:47:53.551Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-05-27T09:47:53.620Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-05-27T09:47:53.692Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Heartbeat-oss
[2020-05-27T09:47:53.757Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-05-27T09:47:53.825Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-05-27T09:47:53.893Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-05-27T09:47:53.962Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Libbeat-x-pack
[2020-05-27T09:47:54.029Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-05-27T09:47:54.103Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Auditbeat-Linux
[2020-05-27T09:47:54.177Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Filebeat-Windows
[2020-05-27T09:47:54.250Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-05-27T09:47:54.325Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-05-27T09:47:54.389Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Packetbeat-oss
[2020-05-27T09:47:54.455Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-Windows
[2020-05-27T09:47:54.521Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-05-27T09:47:54.595Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-05-27T09:47:54.667Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-05-27T09:47:54.735Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-05-27T09:47:54.798Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-05-27T09:47:54.863Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Auditbeat-Mac-OS-X
[2020-05-27T09:47:54.934Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Filebeat-x-pack
[2020-05-27T09:47:55.000Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Functionbeat-Windows
[2020-05-27T09:47:55.079Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Heartbeat-Windows
[2020-05-27T09:47:55.155Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Filebeat-oss
[2020-05-27T09:47:55.226Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-05-27T09:47:55.292Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-05-27T09:47:55.439Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-05-27T09:47:55.519Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Libbeat-oss
[2020-05-27T09:47:55.600Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-05-27T09:47:55.671Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-05-27T09:47:55.739Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-05-27T09:47:55.806Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-05-27T09:47:56.168Z] + cat
[2020-05-27T09:47:56.168Z] + /usr/local/bin/runbld ./runbld-script
[2020-05-27T09:47:56.169Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-05-27T09:48:02.754Z] runbld>>> runbld started
[2020-05-27T09:48:02.754Z] runbld>>> 1.6.11/a66728ff8f4356963772e6e6d2069392fa06acbe
[2020-05-27T09:48:03.694Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-18096' in order of occurrence in the config (last value wins).
[2020-05-27T09:48:05.093Z] runbld>>> Debug logging enabled.
[2020-05-27T09:48:05.093Z] runbld>>> Storing result
[2020-05-27T09:48:05.093Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-05-27T09:48:05.093Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200527094804-561A8009
[2020-05-27T09:48:05.093Z] runbld>>> Adding system facts.
[2020-05-27T09:48:06.031Z] runbld>>> Adding vcs info for the latest commit:  5a7e385af2b23c4b6518f01642b96b29097525be
[2020-05-27T09:48:06.031Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-05-27T09:48:06.031Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-05-27T09:48:06.031Z] Processing JUnit reports with runbld...
[2020-05-27T09:48:06.031Z] + echo 'Processing JUnit reports with runbld...'
[2020-05-27T09:48:06.600Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-05-27T09:48:06.600Z] runbld>>> DURATION: 12ms
[2020-05-27T09:48:06.600Z] runbld>>> STDOUT: 40 bytes
[2020-05-27T09:48:06.600Z] runbld>>> STDERR: 49 bytes
[2020-05-27T09:48:06.600Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-05-27T09:48:06.600Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats
[2020-05-27T09:48:07.977Z] runbld>>> Storing build metadata: 
[2020-05-27T09:48:07.977Z] runbld>>> Adding test report.
[2020-05-27T09:48:07.977Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats
[2020-05-27T09:48:08.914Z] runbld>>> Found 108 test output files
[2020-05-27T09:48:09.173Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-05-27T09:48:09.173Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-05-27T09:48:10.554Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-05-27T09:48:10.554Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-05-27T09:48:10.554Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-05-27T09:48:10.554Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-cloudfoundry.xml
[2020-05-27T09:48:10.554Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-05-27T09:48:10.554Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-05-27T09:48:10.813Z] runbld>>> Test output logs contained: Errors: 0 Failures: 10 Tests: 10538 Skipped: 1292
[2020-05-27T09:48:11.072Z] runbld>>> Storing result
[2020-05-27T09:48:11.072Z] runbld>>> FAILURES: 10
[2020-05-27T09:48:13.604Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-05-27T09:48:13.604Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200527094804-561A8009
[2020-05-27T09:48:13.604Z] runbld>>> Email notification disabled by environment variable.
[2020-05-27T09:48:13.604Z] runbld>>> Slack notification disabled by environment variable.
[2020-05-27T09:48:19.593Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18096
[2020-05-27T09:48:19.834Z] [INFO] getVaultSecret: Getting secrets
[2020-05-27T09:48:19.870Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-05-27T09:48:20.613Z] + chmod 755 generate-build-data.sh
[2020-05-27T09:48:20.614Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18096/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18096/runs/3 FAILURE 4630813
[2020-05-27T09:48:20.614Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18096/runs/3/steps/?limit=10000 -o steps-info.json
[2020-05-27T09:48:23.097Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18096/runs/3/tests/?status=FAILED -o tests-errors.json
[2020-05-27T09:48:23.347Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18096/runs/3/log/ -o pipeline-log.txt

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark · 2020-05-27T13:33:26Z

Failing jobs seem to be unrelated since they have been failing on master already.
More specifically k8s job is being investigated on #18766.
I will merge this based on the successful Travis.

(cherry picked from commit b772f2a)

ChrsMark added 3 commits April 29, 2020 13:52

Move filekeystore to Mapper

654fc4b

Signed-off-by: chrismark <chrismarkou92@gmail.com>

Add k8s Keystore

33a1ff5

Signed-off-by: chrismark <chrismarkou92@gmail.com>

Keystore access per namespace

4ddb15c

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark added enhancement in progress Pull request is currently in progress. [zube]: In Progress autodiscovery labels Apr 29, 2020

ChrsMark self-assigned this Apr 29, 2020

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 29, 2020

ChrsMark added 2 commits April 30, 2020 13:29

Add k8s Keystore for static configs too

c68fe18

Signed-off-by: chrismark <chrismarkou92@gmail.com>

working poc

d0f3629

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark marked this pull request as ready for review April 30, 2020 16:13

ChrsMark requested a review from a team as a code owner April 30, 2020 16:13

ChrsMark added the Team:Platforms Label for the Integrations - Platforms team label Apr 30, 2020

zube bot added [zube]: Inbox and removed [zube]: In Progress labels Apr 30, 2020

andresrc added [zube]: In Review and removed needs_team Indicates that the issue/PR needs a Team:* label [zube]: Inbox labels May 2, 2020

Merge remote-tracking branch 'upstream/master' into k8s-keystore-backend

a0d1181

ChrsMark force-pushed the k8s-keystore-backend branch from 63b3095 to a0d1181 Compare May 6, 2020 10:55

Add error handling

c92b6e5

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark added the v7.9.0 label May 6, 2020

ChrsMark requested review from a team May 6, 2020 12:11

ChrsMark added 3 commits May 6, 2020 16:14

Fix keystore override issue

c5fd9e8

Signed-off-by: chrismark <chrismarkou92@gmail.com>

Fix tests

1c4166f

Signed-off-by: chrismark <chrismarkou92@gmail.com>

Fix monitors

a069dd7

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark added 4 commits May 25, 2020 18:24

Add docs

5e76432

Signed-off-by: chrismark <chrismarkou92@gmail.com>

review fixes

a975065

Signed-off-by: chrismark <chrismarkou92@gmail.com>

Add provider test

2e139c6

Signed-off-by: chrismark <chrismarkou92@gmail.com>

k8s Keystore tests

e02314d

Signed-off-by: chrismark <chrismarkou92@gmail.com>

urso reviewed May 26, 2020

View reviewed changes

heartbeat/autodiscover/builder/hints/monitors_test.go Outdated Show resolved Hide resolved

urso reviewed May 26, 2020

View reviewed changes

libbeat/autodiscover/providers/kubernetes/kubernetes.go Outdated Show resolved Hide resolved

urso reviewed May 26, 2020

View reviewed changes

libbeat/k8skeystore/kubernetes_keystore.go Outdated Show resolved Hide resolved

urso reviewed May 26, 2020

View reviewed changes

libbeat/keystore/keystore.go Outdated Show resolved Hide resolved

ChrsMark added 2 commits May 26, 2020 15:50

Review changes

55070ef

Signed-off-by: chrismark <chrismarkou92@gmail.com>

fix imports

8659570

Signed-off-by: chrismark <chrismarkou92@gmail.com>

urso approved these changes May 26, 2020

View reviewed changes

ChrsMark added 2 commits May 27, 2020 10:07

Merge remote-tracking branch 'upstream/master' into k8s-keystore-backend

1867f26

Improve logging message and key check

5a7e385

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark merged commit b772f2a into elastic:master May 27, 2020

zube bot added [zube]: Done and removed [zube]: In Review labels May 27, 2020

ChrsMark added a commit to ChrsMark/beats that referenced this pull request May 27, 2020

Add k8s keystore backend (elastic#18096)

9caa470

(cherry picked from commit b772f2a)

ChrsMark mentioned this pull request May 27, 2020

Cherry-pick #18096 to 7.x: Add k8s keystore backend #18774

Merged

9 tasks

ChrsMark added test-plan Add this PR to be manual test plan and removed in progress Pull request is currently in progress. labels May 27, 2020

ChrsMark added a commit that referenced this pull request May 28, 2020

Cherry-pick #18096 to 7.x: Add k8s keystore backend (#18774)

a6dd728

This was referenced May 28, 2020

Upgrade k8s.io/client-go and k8s keystore tests #18817

Merged

Cherry-pick #18817 to 7.x: Upgrade k8s.io/client-go and k8s keystore tests #18879

Merged

Mask password discovered via module autodiscover hint #15616

Merged

andresrc removed the [zube]: Done label Jun 2, 2020

andresrc added the test-plan-added This PR has been added to the test plan label Jul 14, 2020

ChrsMark mentioned this pull request Jan 27, 2023

Support of secrets for autodiscovery scenarios in kubernetes environments elastic/elastic-agent#2200

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add k8s keystore backend #18096

Add k8s keystore backend #18096

ChrsMark commented Apr 29, 2020 •

edited

Loading

elasticmachine commented Apr 29, 2020 •

edited

Loading

Build stats

elasticmachine commented Apr 30, 2020

ChrsMark commented May 26, 2020 •

edited

Loading

urso left a comment

elasticmachine commented May 26, 2020 •

edited

Loading

Build stats

Test stats 🧪

ChrsMark commented May 27, 2020 •

edited

Loading

Add k8s keystore backend #18096

Add k8s keystore backend #18096

Conversation

ChrsMark commented Apr 29, 2020 • edited Loading

What does this PR do?

Why is it important?

Checklist

How to test this PR locally

Test k8s keystore backend with hints autodiscover

Test k8s keystore backend with static autodiscover templates

Test basic keystore with static autodiscover templates

Related issues

TODOs:

elasticmachine commented Apr 29, 2020 • edited Loading

❕ Build Aborted

Build stats

Steps errors

Log output

elasticmachine commented Apr 30, 2020

ChrsMark commented May 26, 2020 • edited Loading

urso left a comment

Choose a reason for hiding this comment

elasticmachine commented May 26, 2020 • edited Loading

💔 Tests Failed

Build stats

Test stats 🧪

Test errors

Steps errors

Log output

ChrsMark commented May 27, 2020 • edited Loading

ChrsMark commented Apr 29, 2020 •

edited

Loading

elasticmachine commented Apr 29, 2020 •

edited

Loading

ChrsMark commented May 26, 2020 •

edited

Loading

elasticmachine commented May 26, 2020 •

edited

Loading

ChrsMark commented May 27, 2020 •

edited

Loading