Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] reuse configuration from the workers if possible #24184

Merged
merged 5 commits into from
Feb 24, 2021

Conversation

v1v
Copy link
Member

@v1v v1v commented Feb 23, 2021

What does this PR do?

Reuse if possible the installed tools in the worker

Why is it important?

Reduce the environmental issues when tooling is installed on the fly.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 23, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Feb 23, 2021

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Started by user Victor Martinez

  • Start Time: 2021-02-23T13:38:12.658+0000

  • Duration: 182 min 42 sec

  • Commit: 67998d1

Test stats 🧪

Test Results
Failed 6
Passed 46863
Skipped 4975
Total 51844

Trends 🧪

Image of Build Times

Image of Tests

Test errors 6

Expand to view the tests failures

Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/cloudwatch
    Expand to view the error details

     Failed 
    

    Expand to view the stacktrace

     === RUN   TestFetch
        testdata.go:183: field missing 'aws.rds.metrics.ReadLatency.max': check if fields are documented in `metricbeat/aws/cloudwatch/_meta/fields.yml` file or run 'make update' on Metricbeat folder to update fields in `metricbeat/fields.yml`
    --- FAIL: TestFetch (20.44s)
     
    

Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/ec2
    Expand to view the error details

     Failed 
    

    Expand to view the stacktrace

     === RUN   TestFetch
        testdata.go:183: field missing 'host.network.out.packets': check if fields are documented in `metricbeat/aws/ec2/_meta/fields.yml` file or run 'make update' on Metricbeat folder to update fields in `metricbeat/fields.yml`
    --- FAIL: TestFetch (17.53s)
     
    

Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/rds
    Expand to view the error details

     Failed 
    

    Expand to view the stacktrace

     === RUN   TestFetch
        testdata.go:183: field missing 'cloud.account.name': check if fields are documented in `metricbeat/aws/rds/_meta/fields.yml` file or run 'make update' on Metricbeat folder to update fields in `metricbeat/fields.yml`
    --- FAIL: TestFetch (17.37s)
     
    

Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/s3_daily_storage
    Expand to view the error details

     Failed 
    

    Expand to view the stacktrace

     === RUN   TestFetch
        testdata.go:183: field missing 'cloud.account.name': check if fields are documented in `metricbeat/aws/s3_daily_storage/_meta/fields.yml` file or run 'make update' on Metricbeat folder to update fields in `metricbeat/fields.yml`
    --- FAIL: TestFetch (9.30s)
     
    

Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/s3_request
    Expand to view the error details

     Failed 
    

    Expand to view the stacktrace

     === RUN   TestFetch
        s3_request_integration_test.go:29: 
            	Error Trace:	s3_request_integration_test.go:29
            	Error:      	Should NOT be empty, but was []
            	Test:       	TestFetch
    --- FAIL: TestFetch (8.16s)
     
    

Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/sqs
    Expand to view the error details

     Failed 
    

    Expand to view the stacktrace

     === RUN   TestFetch
        sqs_integration_test.go:29: 
            	Error Trace:	sqs_integration_test.go:29
            	Error:      	Should NOT be empty, but was []
            	Test:       	TestFetch
    --- FAIL: TestFetch (9.45s)
     
    

Steps errors 4

Expand to view the steps failures

Terraform Apply on x-pack/metricbeat/module/aws
  • Took 0 min 19 sec . View more details on here
  • Description: terraform apply -auto-approve
Terraform Apply on x-pack/metricbeat/module/aws
  • Took 0 min 17 sec . View more details on here
  • Description: terraform apply -auto-approve
x-pack/metricbeat-cloud - mage build test
  • Took 17 min 27 sec . View more details on here
  • Description: mage build test
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-02-23T14:42:18.656Z] 
[2021-02-23T14:42:18.656Z] - generated xml file: /go/src/github.com/elastic/beats/x-pack/filebeat/build/TEST-python-integration.xml -
[2021-02-23T14:42:18.656Z] ============================= slowest 20 durations =============================
[2021-02-23T14:42:18.656Z] 40.76s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_171_checkpoint
[2021-02-23T14:42:18.656Z] 31.55s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_000_suricata
[2021-02-23T14:42:18.656Z] 18.65s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_110_o365
[2021-02-23T14:42:18.656Z] 17.10s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_140_snort
[2021-02-23T14:42:18.656Z] 15.27s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_109_o365
[2021-02-23T14:42:18.656Z] 15.03s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_212_threatintel
[2021-02-23T14:42:18.656Z] 11.48s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_178_cisco
[2021-02-23T14:42:18.656Z] 11.13s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_242_ibmmq
[2021-02-23T14:42:18.656Z] 11.10s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_213_threatintel
[2021-02-23T14:42:18.656Z] 11.00s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_196_cisco
[2021-02-23T14:42:18.656Z] 10.84s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_226_panw
[2021-02-23T14:42:18.656Z] 10.76s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_186_cisco
[2021-02-23T14:42:18.656Z] 10.75s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_230_panw
[2021-02-23T14:42:18.656Z] 10.68s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_243_ibmmq
[2021-02-23T14:42:18.656Z] 10.61s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_229_panw
[2021-02-23T14:42:18.656Z] 10.13s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_141_juniper
[2021-02-23T14:42:18.656Z] 10.04s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_255_oracle
[2021-02-23T14:42:18.656Z] 10.02s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_154_cyberark
[2021-02-23T14:42:18.656Z] 10.01s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_215_imperva
[2021-02-23T14:42:18.656Z] 10.01s call     x-pack/filebeat/tests/system/test_xpack_modules.py::XPackTest::test_fileset_file_174_squid
[2021-02-23T14:42:18.657Z] ======================= 317 passed in 1553.55s (0:25:53) =======================
[2021-02-23T14:42:18.657Z] >> python test: Integration Testing Complete
[2021-02-23T14:42:20.950Z] Client: Docker Engine - Community
[2021-02-23T14:42:20.950Z]  Version:           20.10.3
[2021-02-23T14:42:20.950Z]  API version:       1.41
[2021-02-23T14:42:20.950Z]  Go version:        go1.13.15
[2021-02-23T14:42:20.950Z]  Git commit:        48d30b5
[2021-02-23T14:42:20.950Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-02-23T14:42:20.951Z]  OS/Arch:           linux/amd64
[2021-02-23T14:42:20.951Z]  Context:           default
[2021-02-23T14:42:20.951Z]  Experimental:      true
[2021-02-23T14:42:20.951Z] 
[2021-02-23T14:42:20.951Z] Server: Docker Engine - Community
[2021-02-23T14:42:20.951Z]  Engine:
[2021-02-23T14:42:20.951Z]   Version:          20.10.3
[2021-02-23T14:42:20.951Z]   API version:      1.41 (minimum version 1.12)
[2021-02-23T14:42:20.951Z]   Go version:       go1.13.15
[2021-02-23T14:42:20.951Z]   Git commit:       46229ca
[2021-02-23T14:42:20.951Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-23T14:42:20.951Z]   OS/Arch:          linux/amd64
[2021-02-23T14:42:20.951Z]   Experimental:     false
[2021-02-23T14:42:20.951Z]  containerd:
[2021-02-23T14:42:20.951Z]   Version:          1.4.3
[2021-02-23T14:42:20.951Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-23T14:42:20.951Z]  runc:
[2021-02-23T14:42:20.951Z]   Version:          1.0.0-rc92
[2021-02-23T14:42:20.951Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-23T14:42:20.951Z]  docker-init:
[2021-02-23T14:42:20.951Z]   Version:          0.19.0
[2021-02-23T14:42:20.951Z]   GitCommit:        de40ad0
[2021-02-23T14:42:23.263Z] + python .ci/scripts/pre_archive_test.py
[2021-02-23T14:42:25.172Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-02-23T14:42:25.187Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24184/src/github.com/elastic/beats/build
[2021-02-23T14:42:25.492Z] + rm -rf ve
[2021-02-23T14:42:25.492Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-02-23T14:42:25.506Z] Recording test results
[2021-02-23T14:42:28.004Z] [Checks API] No suitable checks publisher found.
[2021-02-23T14:42:28.353Z] Client: Docker Engine - Community
[2021-02-23T14:42:28.353Z]  Version:           20.10.3
[2021-02-23T14:42:28.353Z]  API version:       1.41
[2021-02-23T14:42:28.353Z]  Go version:        go1.13.15
[2021-02-23T14:42:28.353Z]  Git commit:        48d30b5
[2021-02-23T14:42:28.353Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-02-23T14:42:28.353Z]  OS/Arch:           linux/amd64
[2021-02-23T14:42:28.353Z]  Context:           default
[2021-02-23T14:42:28.353Z]  Experimental:      true
[2021-02-23T14:42:28.353Z] 
[2021-02-23T14:42:28.353Z] Server: Docker Engine - Community
[2021-02-23T14:42:28.353Z]  Engine:
[2021-02-23T14:42:28.353Z]   Version:          20.10.3
[2021-02-23T14:42:28.353Z]   API version:      1.41 (minimum version 1.12)
[2021-02-23T14:42:28.353Z]   Go version:       go1.13.15
[2021-02-23T14:42:28.353Z]   Git commit:       46229ca
[2021-02-23T14:42:28.353Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-02-23T14:42:28.353Z]   OS/Arch:          linux/amd64
[2021-02-23T14:42:28.353Z]   Experimental:     false
[2021-02-23T14:42:28.353Z]  containerd:
[2021-02-23T14:42:28.353Z]   Version:          1.4.3
[2021-02-23T14:42:28.353Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-02-23T14:42:28.353Z]  runc:
[2021-02-23T14:42:28.353Z]   Version:          1.0.0-rc92
[2021-02-23T14:42:28.353Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-02-23T14:42:28.353Z]  docker-init:
[2021-02-23T14:42:28.353Z]   Version:          0.19.0
[2021-02-23T14:42:28.353Z]   GitCommit:        de40ad0
[2021-02-23T16:39:52.630Z] Cancelling nested steps due to timeout
[2021-02-23T16:39:52.680Z] Failed in branch x-pack/auditbeat-packaging-linux
[2021-02-23T16:39:52.843Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-02-23T16:39:52.912Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24184/src/github.com/elastic/beats
[2021-02-23T16:39:53.272Z] Running on worker-1244230 in /var/lib/jenkins/workspace/Beats_beats_PR-24184
[2021-02-23T16:39:53.331Z] [INFO] getVaultSecret: Getting secrets
[2021-02-23T16:39:53.450Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-02-23T16:39:56.002Z] + chmod 755 generate-build-data.sh
[2021-02-23T16:39:56.002Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24184/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24184/runs/3 FAILURE 10901948
[2021-02-23T16:39:56.002Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24184/runs/3/steps/?limit=10000 -o steps-info.json
[2021-02-23T16:40:03.249Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24184/runs/3/tests/?status=FAILED -o tests-errors.json
[2021-02-23T16:40:07.674Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24184/runs/3/log/ -o pipeline-log.txt

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test Results
Failed 6
Passed 46863
Skipped 4975
Total 51844

Genuine test errors 6

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/cloudwatch
  • Name: Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/ec2
  • Name: Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/rds
  • Name: Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/s3_daily_storage
  • Name: Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/s3_request
  • Name: Build&Test / x-pack/metricbeat-cloud / TestFetch – github.com/elastic/beats/v7/x-pack/metricbeat/module/aws/sqs

@@ -11,7 +11,7 @@ if command -v kind
then
echo "Found Kind. Checking version.."
FOUND_KIND_VERSION=$(kind --version 2>&1 >/dev/null | awk '{print $3}')
if [ $FOUND_KIND_VERSION == $KIND_VERSION ]
if [ "$FOUND_KIND_VERSION" == "$KIND_VERSION" ]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linting

if command -v kubectl
then
echo "Found kubectl. Checking version.."
FOUND_KUBECTL_VERSION=$(kubectl version --client --short 2>&1 >/dev/null | awk '{print $3}')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$ kubectl version --client --short
Client Version: v1.20.2
$ kubectl version --client --short 2>&1 >/dev/null | awk '{print $3}'
v1.20.2

if command -v terraform
then
echo "Found Terraform. Checking version.."
FOUND_TERRAFORM_VERSION=$(terraform --version | awk '{print $2}' | sed s/v//)
if [ $FOUND_TERRAFORM_VERSION == $TERRAFORM_VERSION ]
if [ "$FOUND_TERRAFORM_VERSION" == "$TERRAFORM_VERSION" ]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linting

ARCH_SUFFIX=arm
fi

if curl -sSLo - "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_${OS}_${ARCH_SUFFIX}.zip" > "${TERRAFORM_CMD}.zip" ; then
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andresrc andresrc added the Team:Automation Label for the Observability productivity team label Feb 23, 2021
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 23, 2021
@v1v v1v marked this pull request as ready for review February 24, 2021 11:00
@v1v v1v requested a review from a team as a code owner February 24, 2021 11:00
@v1v v1v self-assigned this Feb 24, 2021
@v1v v1v added the automation label Feb 24, 2021
@v1v v1v merged commit 6c0470c into elastic:master Feb 24, 2021
v1v added a commit to v1v/beats that referenced this pull request Feb 24, 2021
v1v added a commit to v1v/beats that referenced this pull request Feb 24, 2021
v1v added a commit that referenced this pull request Feb 24, 2021
…raform (#24184)(#24214)(#24209)

* [CI] reuse configuration from the workers if possible (#24184)
* Bump terraform dependency with arm64 support (#24214)
v1v added a commit that referenced this pull request Feb 24, 2021
…rraform (#24184) (#24214) (#24211)

* [CI] reuse configuration from the workers if possible (#24184)
* Bump terraform dependency with arm64 support (#24214)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automation Team:Automation Label for the Observability productivity team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants