Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] install docker-compose with retry #24069

Merged

Conversation

v1v
Copy link
Member

@v1v v1v commented Feb 16, 2021

What does this PR do?

Remove corrupted binary if it failed to download it.

Why is it important?

When installing docker-compose if it failed the very first attempt then the binary it might be corrupted and the retries will fail over and over

Fixes

[2021-02-16T14:17:03.541Z] + echo 'Found docker-compose. Checking version..'
[2021-02-16T14:17:03.541Z] Found docker-compose. Checking version..
[2021-02-16T14:17:03.541Z] ++ docker-compose --version
[2021-02-16T14:17:03.541Z] ++ awk '{print $3}'
[2021-02-16T14:17:03.541Z] ++ sed s/,//
[2021-02-16T14:17:03.541Z] [2802] Cannot open self /var/lib/jenkins/workspace/Beats_beats_PR-24041/bin/docker-compose or archive /var/lib/jenkins/workspace/Beats_beats_PR-24041/bin/docker-compose.pkg
[2021-02-16T14:17:03.541Z] + FOUND_DOCKER_COMPOSE_VERSION=
script returned exit code 255

image

@v1v v1v added automation Team:Automation Label for the Observability productivity team labels Feb 16, 2021
@v1v v1v requested a review from a team February 16, 2021 17:59
@v1v v1v self-assigned this Feb 16, 2021
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Feb 16, 2021
Comment on lines 26 to 33
if ! curl -sSLo "${DC_CMD}" "https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-$(uname -s)-$(uname -m)" ; then
if [ -e "${DC_CMD}" ] ; then
rm "${DC_CMD}"
fi
exit 1
else
chmod +x "${DC_CMD}"
fi
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was tempted to use the trap design, but somehow I think if/else might be easier to follow

@elasticmachine
Copy link
Collaborator

elasticmachine commented Feb 16, 2021

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #24069 updated

  • Start Time: 2021-02-16T18:01:58.734+0000

  • Duration: 68 min 40 sec

  • Commit: f3d3fc9

Test stats 🧪

Test Results
Failed 0
Passed 46543
Skipped 4754
Total 51297

Trends 🧪

Image of Build Times

Image of Tests

Steps errors 3

Expand to view the steps failures

x-pack/metricbeat-windows-7-32-windows-7-32-bit - mage build unitTest
  • Took 4 min 20 sec . View more details on here
  • Description: mage build unitTest
authenticate
  • Took 0 min 1 sec . View more details on here
  • Description: gcloud auth activate-service-account --key-file %FILE_CREDENTIAL%
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-02-16T19:08:12.175Z] 11fb4d3fb711: Layer already exists
[2021-02-16T19:08:12.175Z] be835c3c7702: Layer already exists
[2021-02-16T19:08:12.433Z] bdaea450dcb0: Layer already exists
[2021-02-16T19:08:12.433Z] 17e8a33b10c2: Layer already exists
[2021-02-16T19:08:12.433Z] 00af10937683: Layer already exists
[2021-02-16T19:08:12.433Z] 3aa55ff7bca1: Layer already exists
[2021-02-16T19:08:12.433Z] da10bfee9635: Layer already exists
[2021-02-16T19:08:12.433Z] ab12229ad1d6: Layer already exists
[2021-02-16T19:08:12.691Z] c85339c56349: Layer already exists
[2021-02-16T19:08:13.626Z] f3d3fc92b95fad5641d27966cc6800b3fea4dc1a: digest: sha256:f494d1991b0d7489f38a39245c68fb364d2d48de7dd57efbb958115960d4fcc4 size: 3034
[2021-02-16T19:08:14.018Z] Client: Docker Engine - Community
[2021-02-16T19:08:14.018Z]  Version:           20.10.3
[2021-02-16T19:08:14.018Z]  API version:       1.41
[2021-02-16T19:08:14.018Z]  Go version:        go1.13.15
[2021-02-16T19:08:14.018Z]  Git commit:        48d30b5
[2021-02-16T19:08:14.018Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-02-16T19:08:14.018Z]  OS/Arch:           linux/amd64
[2021-02-16T19:08:14.018Z]  Context:           default
[2021-02-16T19:08:14.018Z]  Experimental:      true
[2021-02-16T19:08:14.018Z] 
[2021-02-16T19:08:14.018Z] Server: Docker Engine - Community
[2021-02-16T19:08:14.018Z]  Engine:
[2021-02-16T19:08:14.018Z]   Version:          20.10.3
[2021-02-16T19:08:14.018Z]   API version:      1.41 (minimum version 1.12)
[2021-02-16T19:08:14.018Z]   Go version:       go1.13.15
[2021-02-16T19:08:14.018Z]   Git commit:       46229ca
[2021-02-16T19:08:14.018Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-16T19:08:14.018Z]   OS/Arch:          linux/amd64
[2021-02-16T19:08:14.018Z]   Experimental:     false
[2021-02-16T19:08:14.018Z]  containerd:
[2021-02-16T19:08:14.018Z]   Version:          1.4.3
[2021-02-16T19:08:14.018Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-16T19:08:14.018Z]  runc:
[2021-02-16T19:08:14.018Z]   Version:          1.0.0-rc92
[2021-02-16T19:08:14.018Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-16T19:08:14.018Z]  docker-init:
[2021-02-16T19:08:14.018Z]   Version:          0.19.0
[2021-02-16T19:08:14.018Z]   GitCommit:        de40ad0
[2021-02-16T19:08:14.018Z] Unable to find image 'alpine:3.4' locally
[2021-02-16T19:08:14.585Z] 3.4: Pulling from library/alpine
[2021-02-16T19:08:14.844Z] c1e54eec4b57: Pulling fs layer
[2021-02-16T19:08:15.102Z] c1e54eec4b57: Download complete
[2021-02-16T19:08:15.361Z] c1e54eec4b57: Pull complete
[2021-02-16T19:08:15.361Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-02-16T19:08:15.361Z] Status: Downloaded newer image for alpine:3.4
[2021-02-16T19:08:17.570Z] + python .ci/scripts/pre_archive_test.py
[2021-02-16T19:08:27.594Z] Copy ./x-pack/metricbeat/build into build/x-pack/metricbeat/build
[2021-02-16T19:08:27.594Z] Copy ./x-pack/metricbeat/build/golang-crossbuild into build/x-pack/metricbeat/build/golang-crossbuild
[2021-02-16T19:08:27.594Z] Copy ./x-pack/elastic-agent/build into build/x-pack/elastic-agent/build
[2021-02-16T19:08:27.594Z] Copy ./x-pack/elastic-agent/build/golang-crossbuild into build/x-pack/elastic-agent/build/golang-crossbuild
[2021-02-16T19:08:27.594Z] Copy ./x-pack/elastic-agent/build/package/elastic-agent/elastic-agent-linux-amd64.docker/docker-build into build/x-pack/elastic-agent/build/package/elastic-agent/elastic-agent-linux-amd64.docker/docker-build
[2021-02-16T19:08:27.594Z] Copy ./x-pack/elastic-agent/build/package/elastic-agent-ubi8/elastic-agent-linux-amd64.docker/docker-build into build/x-pack/elastic-agent/build/package/elastic-agent-ubi8/elastic-agent-linux-amd64.docker/docker-build
[2021-02-16T19:08:27.594Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-02-16T19:08:27.594Z] Copy ./x-pack/filebeat/build/golang-crossbuild into build/x-pack/filebeat/build/golang-crossbuild
[2021-02-16T19:08:27.594Z] Copy ./x-pack/heartbeat/build into build/x-pack/heartbeat/build
[2021-02-16T19:08:27.594Z] Copy ./x-pack/heartbeat/build/golang-crossbuild into build/x-pack/heartbeat/build/golang-crossbuild
[2021-02-16T19:08:27.615Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24069/src/github.com/elastic/beats/build
[2021-02-16T19:08:28.185Z] + rm -rf ve
[2021-02-16T19:08:28.185Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-02-16T19:08:28.197Z] Recording test results
[2021-02-16T19:08:30.800Z] None of the test reports contained any result
[2021-02-16T19:08:30.808Z] [Checks API] No suitable checks publisher found.
[2021-02-16T19:08:31.168Z] Client: Docker Engine - Community
[2021-02-16T19:08:31.168Z]  Version:           20.10.3
[2021-02-16T19:08:31.168Z]  API version:       1.41
[2021-02-16T19:08:31.168Z]  Go version:        go1.13.15
[2021-02-16T19:08:31.168Z]  Git commit:        48d30b5
[2021-02-16T19:08:31.168Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-02-16T19:08:31.168Z]  OS/Arch:           linux/amd64
[2021-02-16T19:08:31.168Z]  Context:           default
[2021-02-16T19:08:31.168Z]  Experimental:      true
[2021-02-16T19:08:31.168Z] 
[2021-02-16T19:08:31.168Z] Server: Docker Engine - Community
[2021-02-16T19:08:31.168Z]  Engine:
[2021-02-16T19:08:31.168Z]   Version:          20.10.3
[2021-02-16T19:08:31.168Z]   API version:      1.41 (minimum version 1.12)
[2021-02-16T19:08:31.168Z]   Go version:       go1.13.15
[2021-02-16T19:08:31.168Z]   Git commit:       46229ca
[2021-02-16T19:08:31.168Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-16T19:08:31.168Z]   OS/Arch:          linux/amd64
[2021-02-16T19:08:31.168Z]   Experimental:     false
[2021-02-16T19:08:31.168Z]  containerd:
[2021-02-16T19:08:31.168Z]   Version:          1.4.3
[2021-02-16T19:08:31.168Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-16T19:08:31.168Z]  runc:
[2021-02-16T19:08:31.168Z]   Version:          1.0.0-rc92
[2021-02-16T19:08:31.168Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-16T19:08:31.168Z]  docker-init:
[2021-02-16T19:08:31.168Z]   Version:          0.19.0
[2021-02-16T19:08:31.168Z]   GitCommit:        de40ad0
[2021-02-16T19:09:32.022Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-02-16T19:09:32.107Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24069/src/github.com/elastic/beats
[2021-02-16T19:09:37.829Z] Running on beats-ci-immutable-ubuntu-1604-1613501383818793863 in /var/lib/jenkins/workspace/Beats_beats_PR-24069
[2021-02-16T19:09:37.890Z] [INFO] getVaultSecret: Getting secrets
[2021-02-16T19:09:37.990Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-02-16T19:09:38.834Z] + chmod 755 generate-build-data.sh
[2021-02-16T19:09:38.834Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24069/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24069/runs/2 FAILURE 4059807
[2021-02-16T19:09:38.834Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24069/runs/2/steps/?limit=10000 -o steps-info.json
[2021-02-16T19:09:48.893Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24069/runs/2/tests/?status=FAILED -o tests-errors.json
[2021-02-16T19:09:49.829Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24069/runs/2/log/ -o pipeline-log.txt

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 46543
Skipped 4754
Total 51297

@v1v
Copy link
Member Author

v1v commented Feb 17, 2021

Go build for x-pack/metricbeats in windows-7-32 got a resources limitation of 2gb:

[2021-02-16T18:43:58.114Z] panic: Not enough storage is available to process this command.

No an environmental issue but a build system issue with windows-7

@v1v v1v merged commit 465750c into elastic:master Feb 17, 2021
@v1v v1v deleted the feature/docker-compose-environemntal-issue-with-retry branch February 17, 2021 09:45
v1v added a commit to v1v/beats that referenced this pull request Feb 17, 2021
v1v added a commit to v1v/beats that referenced this pull request Feb 17, 2021
v1v added a commit to v1v/beats that referenced this pull request Feb 17, 2021
…-arm

* upstream/master:
  [CI] install docker-compose with retry (elastic#24069)
  Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052)
  updating manifest files for filebeat threatintel module (elastic#24074)
  Add Zeek Signatures (elastic#23772)
  Update Beats to ECS 1.8.0 (elastic#23465)
  Support running Docker logging plugin on ARM64 (elastic#24034)
  Fix ec2 metricset fields.yml and add integration test (elastic#23726)
  Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060)
  [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773)
  [Elastic Agent] Enroll with Fleet Server (elastic#23865)
  [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944)
  [Ingest Management] Fix reloading of log level for services (elastic#24055)
  Add Agent standalone k8s manifest (elastic#23679)
v1v added a commit to v1v/beats that referenced this pull request Feb 17, 2021
…dows-7

* upstream/master: (332 commits)
  Use ECS v1.8.0 (elastic#24086)
  Add support for postgresql csv logs (elastic#23334)
  [Heartbeat] Refactor config system (elastic#23467)
  [CI] install docker-compose with retry (elastic#24069)
  Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052)
  updating manifest files for filebeat threatintel module (elastic#24074)
  Add Zeek Signatures (elastic#23772)
  Update Beats to ECS 1.8.0 (elastic#23465)
  Support running Docker logging plugin on ARM64 (elastic#24034)
  Fix ec2 metricset fields.yml and add integration test (elastic#23726)
  Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060)
  [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773)
  [Elastic Agent] Enroll with Fleet Server (elastic#23865)
  [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944)
  [Ingest Management] Fix reloading of log level for services (elastic#24055)
  Add Agent standalone k8s manifest (elastic#23679)
  [Metricbeat][Kubernetes] Extend state_node with more conditions (elastic#23905)
  [CI] googleStorageUploadExt step (elastic#24048)
  Check fields are documented for aws metricsets (elastic#23887)
  Update go-concert to 0.1.0 (elastic#23770)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automation Team:Automation Label for the Observability productivity team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants