Cherry-pick #20898 to 7.9: Avoid generating incomplete configurations in autodiscover #20920

jsoriano · 2020-09-02T11:16:47Z

Cherry-pick of PR #20898 to 7.9 branch. Original message:

What does this PR do?

Handle errors when configuration unpacking fails. In principle this can
only happen when some variable is missing, because configuration has
been previously parsed as YAML. Errors on unpacking were previously
ignored.

When a variable is missing, this is clearly logged at the debug level.

This changes the behaviour, previously an incomplete configuration was
generated on this case.

Why is it important?

To detect problems with incomplete configuration earlier and improve feedback in logs
when a variable is missing in event.

Since #18979 this error is more frequent, as a legit configuration for containers produces
errors with pod events that don't contain container fields.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~I have made corresponding changes to the documentation~~
~~I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Logs

For example the following configuration includes an incorrect field container.id.

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/lib/docker/containers/${container.id}/*.log

Before this change, an incomplete configuration is generated, and errors like these ones are logged:

2020-09-01T16:52:50.094+0200	ERROR	[autodiscover]	autodiscover/autodiscover.go:209	Auto discover config check failed for config '{
  "docker-json": {
    "cri_flags": true,
    "format": "auto",
    "partial": true,
    "stream": "all"
  },
  "symlinks": true,
  "type": "container"
}', won't start runner: each input must have at least one path defined

After this change, the cause of the problem is more clear, and it is only logged at the debug level:

2020-09-02T11:57:08.225+0200	DEBUG	[autodiscover]	template/config.go:156	Configuration template cannot be resolved: field 'container.id' not available in event or environment accessing 'paths' (source:'/home/jaime/tmp/filebeat-autodiscover-docker-hints.yml')

…0898) Handle errors when configuration unpacking fails. In principle this can only happen when some variable is missing, because configuration has been previously parsed as YAML. Errors on unpacking were previously ignored. When a variable is missing, this is clearly logged at the debug level. This changes the behaviour, previously an incomplete configuration was generated on this case. (cherry picked from commit 99fd545)

elasticmachine · 2020-09-02T11:16:50Z

Pinging @elastic/integrations-platforms (Team:Platforms)

CHANGELOG.next.asciidoc

elasticmachine · 2020-09-02T11:36:30Z

💔 Tests Failed

Expand to view the summary

Build stats

Build Cause: [Pull request #20920 updated]
Start Time: 2020-09-03T16:51:48.688+0000
Duration: 75 min 57 sec

Test stats 🧪

Test	Results
Failed	1
Passed	19389
Skipped	1841
Total	21231

Test errors

Expand to view the tests failures

Name: Build and Test / Metricbeat Python integration tests / test_remote_write – test_prometheus.TestRemoteWrite
- Age: 1
- Duration: 60.367
- Error Details: Timeout waiting for 'cond' to be true. Waited 60 seconds.
  -------------------- >> begin captured logging << --------------------
  compose.config.config: DEBUG: Using configuration files: /go/src/github.com/elastic/beats/metricbeat/module/prometheus/docker-compose.yml
  docker.utils.config: DEBUG: Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
  docker.utils.config: DEBUG: No config file found
  docker.utils.config: DEBUG: Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
  docker.utils.config: DEBUG: No config file found
  compose.parallel: DEBUG: Pending: {<Container: prometheus_7918a246f6f8_prometheus_1 (30d7f5)>}
  compose.parallel: DEBUG: Starting producer thread for <Container: prometheus_7918a246f6f8_prometheus_1 (30d7f5)>
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Finished processing: <Container: prometheus_7918a246f6f8_prometheus_1 (30d7f5)>
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: {<Container: prometheus_7918a246f6f8_prometheus_1 (30d7f5)>}
  compose.parallel: DEBUG: Starting producer thread for <Container: prometheus_7918a246f6f8_prometheus_1 (30d7f5)>
  compose.parallel: DEBUG: Finished processing: <Container: prometheus_7918a246f6f8_prometheus_1 (30d7f5)>
  compose.parallel: DEBUG: Pending: set()
  compose.network: INFO: Removing network prometheus_7918a246f6f8_default
  compose.config.config: DEBUG: Using configuration files: /go/src/github.com/elastic/beats/metricbeat/module/prometheus/docker-compose.yml
  docker.utils.config: DEBUG: Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
  docker.utils.config: DEBUG: No config file found
  docker.utils.config: DEBUG: Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
  docker.utils.config: DEBUG: No config file found
  compose.service: INFO: Pulling prometheus-host-network (docker.elastic.co/integrations-ci/beats-prometheus:2.6.0-2)...
  docker.auth: DEBUG: Looking for auth config
  docker.auth: DEBUG: No auth config in memory - loading from filesystem
  docker.utils.config: DEBUG: Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
  docker.utils.config: DEBUG: No config file found
  docker.auth: DEBUG: Looking for auth entry for 'docker.elastic.co'
  docker.auth: DEBUG: No entry found
  docker.auth: DEBUG: No auth config found
  compose.service: ERROR: 404 Client Error: Not Found ("manifest for docker.elastic.co/integrations-ci/beats-prometheus:2.6.0-2 not found: manifest unknown: manifest unknown")
  compose.network: INFO: Creating network "prometheus_7918a246f6f8_default" with the default driver
  compose.parallel: DEBUG: Pending: {<Service: prometheus-host-network>}
  compose.parallel: DEBUG: Starting producer thread for <Service: prometheus-host-network>
  compose.parallel: DEBUG: Pending: {ServiceName(project='prometheus_7918a246f6f8', service='prometheus-host-network', number=1)}
  compose.parallel: DEBUG: Starting producer thread for ServiceName(project='prometheus_7918a246f6f8', service='prometheus-host-network', number=1)
  compose.service: DEBUG: Added config hash: db0571ac919ec4c3e1c77e89bd584ad4eaa726f6a9c68094a7ea3adf1a4aa187
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Finished processing: ServiceName(project='prometheus_7918a246f6f8', service='prometheus-host-network', number=1)
  compose.parallel: DEBUG: Pending: set()
  compose.parallel: DEBUG: Finished processing: <Service: prometheus-host-network>
  compose.parallel: DEBUG: Pending: set()
  --------------------- >> end captured logging << ---------------------

Steps errors

Expand to view the steps failures

Name: Install Go 1.14.7
- Description: .ci/scripts/install-go.sh
- Duration: 1 min 57 sec
- Start Time: 2020-09-03T17:15:25.235+0000
- log
Name: Mage pythonIntegTest
- Description: mage pythonIntegTest
- Duration: 35 min 3 sec
- Start Time: 2020-09-03T17:15:46.443+0000
- log

Log output

Expand to view the last 100 lines of log output

[2020-09-03T18:06:17.674Z] + rm source.tgz
[2020-09-03T18:06:17.688Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats
[2020-09-03T18:06:17.712Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Lint
[2020-09-03T18:06:17.798Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Winlogbeat-oss
[2020-09-03T18:06:17.877Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-09-03T18:06:17.955Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-09-03T18:06:18.033Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/dockerlogbeat
[2020-09-03T18:06:18.111Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-09-03T18:06:18.193Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Journalbeat
[2020-09-03T18:06:18.274Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-09-03T18:06:18.352Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Packetbeat-Linux
[2020-09-03T18:06:18.429Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-09-03T18:06:18.514Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-09-03T18:06:18.595Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Auditbeat-oss-Windows
[2020-09-03T18:06:18.673Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Heartbeat-oss
[2020-09-03T18:06:18.756Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Auditbeat-x-pack-Windows
[2020-09-03T18:06:18.837Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-09-03T18:06:18.916Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Heartbeat-Windows
[2020-09-03T18:06:18.995Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Auditbeat-oss-Linux
[2020-09-03T18:06:19.075Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-09-03T18:06:19.158Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Libbeat-x-pack
[2020-09-03T18:06:19.238Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-09-03T18:06:19.317Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-09-03T18:06:19.396Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Filebeat-Windows
[2020-09-03T18:06:19.473Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Functionbeat-Windows
[2020-09-03T18:06:19.551Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-09-03T18:06:19.629Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Packetbeat-Windows
[2020-09-03T18:06:19.709Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-Windows
[2020-09-03T18:06:19.786Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-09-03T18:06:19.873Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-09-03T18:06:19.950Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-09-03T18:06:20.030Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-09-03T18:06:20.108Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-09-03T18:06:20.186Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-09-03T18:06:20.264Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-09-03T18:06:20.345Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Filebeat-oss
[2020-09-03T18:06:20.430Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Filebeat-x-pack
[2020-09-03T18:06:20.511Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-09-03T18:06:20.595Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Libbeat-oss
[2020-09-03T18:06:20.682Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-09-03T18:06:20.768Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-09-03T18:06:20.846Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-09-03T18:06:20.926Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-09-03T18:06:21.005Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-09-03T18:06:21.084Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-09-03T18:06:21.162Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-09-03T18:06:21.241Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-09-03T18:06:21.320Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-09-03T18:06:21.400Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-09-03T18:06:21.478Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Packetbeat-Mac-OS-X
[2020-09-03T18:06:21.877Z] + cat
[2020-09-03T18:06:21.877Z] + /usr/local/bin/runbld ./runbld-script --job-name elastic+beats+pull-request
[2020-09-03T18:06:21.877Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-09-03T18:06:28.474Z] runbld>>> runbld started
[2020-09-03T18:06:28.474Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-09-03T18:06:30.395Z] runbld>>> The following profiles matched the job 'elastic+beats+pull-request' in order of occurrence in the config (last value wins).
[2020-09-03T18:06:30.395Z] runbld>>> Matches in the system config:
[2020-09-03T18:06:30.395Z] runbld>>> - Matched ^elastic\+beats
[2020-09-03T18:06:30.395Z] runbld>>> - Matched ^elastic\+beats\+pull-request
[2020-09-03T18:06:31.782Z] runbld>>> Debug logging enabled.
[2020-09-03T18:06:31.782Z] runbld>>> Storing result
[2020-09-03T18:06:31.782Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-09-03T18:06:31.782Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200903180631-3604803E
[2020-09-03T18:06:31.782Z] runbld>>> Adding system facts.
[2020-09-03T18:06:32.726Z] runbld>>> Adding vcs info for the latest commit:  fc73050956f28897b383705d29be1573fc7916b3
[2020-09-03T18:06:32.727Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-09-03T18:06:32.727Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-09-03T18:06:32.988Z] Processing JUnit reports with runbld...
[2020-09-03T18:06:32.988Z] + echo 'Processing JUnit reports with runbld...'
[2020-09-03T18:06:33.250Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-09-03T18:06:33.250Z] runbld>>> DURATION: 21ms
[2020-09-03T18:06:33.250Z] runbld>>> STDOUT: 40 bytes
[2020-09-03T18:06:33.250Z] runbld>>> STDERR: 49 bytes
[2020-09-03T18:06:33.250Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-09-03T18:06:33.250Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats_PR-20920
[2020-09-03T18:06:34.195Z] runbld>>> Storing build metadata: 
[2020-09-03T18:06:34.195Z] runbld>>> Adding test report.
[2020-09-03T18:06:34.195Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats
[2020-09-03T18:06:35.143Z] runbld>>> Found 140 test output files
[2020-09-03T18:06:37.059Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-09-03T18:06:37.059Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-09-03T18:06:37.059Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-09-03T18:06:37.059Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-09-03T18:06:37.320Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-09-03T18:06:37.320Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-09-03T18:06:37.321Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20920/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-09-03T18:06:38.265Z] runbld>>> Test output logs contained: Errors: 1 Failures: 0 Tests: 21080 Skipped: 1560
[2020-09-03T18:06:38.532Z] runbld>>> Storing result
[2020-09-03T18:06:38.532Z] runbld>>> FAILURES: 1
[2020-09-03T18:06:38.810Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-09-03T18:06:38.810Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200903180631-3604803E
[2020-09-03T18:06:39.075Z] runbld>>> Email notification disabled by environment variable.
[2020-09-03T18:06:39.076Z] runbld>>> Slack notification disabled by environment variable.
[2020-09-03T18:06:44.559Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-20920
[2020-09-03T18:06:44.669Z] [INFO] getVaultSecret: Getting secrets
[2020-09-03T18:06:44.742Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-09-03T18:06:45.537Z] + chmod 755 generate-build-data.sh
[2020-09-03T18:06:45.537Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20920/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20920/runs/5 FAILURE 4496589
[2020-09-03T18:06:45.537Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20920/runs/5/steps/?limit=10000 -o steps-info.json
[2020-09-03T18:06:48.020Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20920/runs/5/tests/?status=FAILED -o tests-errors.json

jsoriano · 2020-09-02T13:38:33Z

jenkins run the tests please

…rations in autodiscover (elastic#20920) Handle errors when configuration unpacking fails. In principle this can only happen when some variable is missing, because configuration has been previously parsed as YAML. Errors on unpacking were previously ignored. When a variable is missing, this is clearly logged at the debug level. This changes the behaviour, previously an incomplete configuration was generated on this case. (cherry picked from commit ff60b11)

jsoriano added [zube]: In Review backport Team:Platforms Label for the Integrations - Platforms team labels Sep 2, 2020

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 2, 2020

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Sep 2, 2020

narph reviewed Sep 2, 2020

View reviewed changes

CHANGELOG.next.asciidoc Show resolved Hide resolved

Fix changelog

ecec6f5

ChrsMark approved these changes Sep 2, 2020

View reviewed changes

jsoriano added 2 commits September 3, 2020 18:47

Merge branch '7.9' into backport_20898_7.9

2c74e47

Fix changelog

fc73050

kaiyan-sheng approved these changes Sep 3, 2020

View reviewed changes

jsoriano merged commit 5be9bcd into elastic:7.9 Sep 4, 2020

jsoriano deleted the backport_20898_7.9 branch September 4, 2020 09:45

zube bot added [zube]: Done and removed [zube]: In Review labels Sep 4, 2020

zube bot removed the [zube]: Done label Dec 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cherry-pick #20898 to 7.9: Avoid generating incomplete configurations in autodiscover #20920

Cherry-pick #20898 to 7.9: Avoid generating incomplete configurations in autodiscover #20920

jsoriano commented Sep 2, 2020 •

edited by zube bot

Loading

elasticmachine commented Sep 2, 2020

elasticmachine commented Sep 2, 2020 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

jsoriano commented Sep 2, 2020

Cherry-pick #20898 to 7.9: Avoid generating incomplete configurations in autodiscover #20920

Cherry-pick #20898 to 7.9: Avoid generating incomplete configurations in autodiscover #20920

Conversation

jsoriano commented Sep 2, 2020 • edited by zube bot Loading

What does this PR do?

Why is it important?

Checklist

Related issues

Logs

elasticmachine commented Sep 2, 2020

elasticmachine commented Sep 2, 2020 • edited by jenkins-beats-ci bot Loading

💔 Tests Failed

Build stats

Test stats 🧪

Test errors

Steps errors

Log output

jsoriano commented Sep 2, 2020

jsoriano commented Sep 2, 2020 •

edited by zube bot

Loading

elasticmachine commented Sep 2, 2020 •

edited by jenkins-beats-ci bot

Loading