ironbank: validate heartbeat docker context #32982

v1v · 2022-09-05T17:32:12Z

What does this PR do?

It validates the DoD docker context for x-pack/heartbeat contains the required dependencies and the docker image can be built.

Why is it important?

As agreed, docker context generation does not validate the changes, hence this is the approach to verify the artifacts to be added to the DoD docker context are built correctly.

Test

CI

See this build

NOTE: The existing pipeline will not run on PR basis, but on a time basis for the active branches (see #32977), though users can trigger a build manually using the Jenkins UI.

Manually

hub pr checkout 32982
cd x-pack/heartbeat
make -C ironbank package
mage ironbank
make -C ironbank validate-ironbank
echo $?

produced:

curl -sSfL -o ../build/heartbeat-ironbank-8.5.0-docker-build-context/tinit https://github.com/krallin/tini/releases/download/v0.19.0/tini-amd64
curl -sSfL -o ../build/heartbeat-ironbank-8.5.0-docker-build-context/jq https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64
...
...
docker build \
		--build-arg BASE_REGISTRY=docker.elastic.co \
		--build-arg BASE_IMAGE=ubi8/ubi \
		--build-arg BASE_TAG=latest \
		../build/heartbeat-ironbank-8.5.0-docker-build-context
 => exporting to image                                                     0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:acd39886d84caaacc713628341380cbeb430866f758ba  0.0s
...
...
0

Follow ups

Potentially it could be also tested with some integration testing hence the docker heartbeat works with the Elastic Stack. Although to do so, it requires to store the docker-compose for the Elastic Stack and some users/roles (similar to https://github.com/jenkinsci/opentelemetry-plugin/tree/master/src/test/resources).
For the time being, I'll say to keep it simple, otherwise we can add certain complexity that might not be needed

Issues

Requires #32976

elasticmachine · 2022-09-05T18:19:16Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-09-06T11:14:03.967+0000
Duration: 46 min 53 sec

Test stats 🧪

Test	Results
Failed	0
Passed	178
Skipped	0
Total	178

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

…eartbeat/hardening_manifest.yaml as the source of truth hence no need to do any duplication of entries

v1v · 2022-09-06T11:02:46Z

.ci/ironbank-validation.groovy

+        failure {
+          notifyStatus(slackStatus: 'danger', subject: "[${env.REPO}@${BRANCH_NAME}] package for ${env.BEATS_FOLDER}", body: "Contact the heartbeats team. (<${env.RUN_DISPLAY_URL}|Open>)")
+        }


This failure might be relevant to the product team, since it's related to the packaging

v1v · 2022-09-06T11:03:24Z

.ci/ironbank-validation.groovy

+        failure {
+          notifyStatus(slackStatus: 'danger', subject: "[${env.REPO}@${BRANCH_NAME}] Ironbank docker context for ${env.BEATS_FOLDER}", body: "Contact the @observablt-robots-team team. (<${env.RUN_DISPLAY_URL}|Open>)")
+        }


Ironbank packages could fail, if so, likely an environmental issue, since the DRA will also fail. In any case we will redirect the message to the robots team

v1v · 2022-09-06T11:04:09Z

.ci/ironbank-validation.groovy

+        failure {
+          notifyStatus(slackStatus: 'danger', subject: "[${env.REPO}@${BRANCH_NAME}] Ironbank validation failed", body: "Contact the @observablt-robots-team team. (<${env.RUN_DISPLAY_URL}|Open>)")
+        }


This is the new validation, so if the docker image cannot be built then something is broken in the list of dependencies, if that happens then we will need to fix it.

x-pack/heartbeat/ironbank/Makefile

amannocci

LGTM

amannocci · 2022-09-06T15:02:26Z

x-pack/heartbeat/ironbank/Makefile

+
+## @help:setup-yq: Install yq in ../build/yq.
+.PHONY: setup-yq
+setup-yq:


We could try to detect if yq is installed at the OS level and if not then download it.

I'd prefer to keep the logic simpler and agnostic to the worker, we can revisit this in the future if needed

(cherry picked from commit 7cbb828)

(cherry picked from commit 7cbb828) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

v1v added 2 commits September 5, 2022 13:34

ci: ironbank validation pipeline

d0acded

ironbank: validate heartbeat docker context

f2baf0a

v1v added automation Team:Automation Label for the Observability productivity team backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.4.0 Automated backport with mergify labels Sep 5, 2022

v1v self-assigned this Sep 5, 2022

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Sep 5, 2022

v1v mentioned this pull request Sep 5, 2022

ci: run daily the ironbank validations #32977

Merged

v1v added 7 commits September 6, 2022 09:54

ironbank validation: use the dev-tools/packaging/templates/ironbank/h…

6cc419b

…eartbeat/hardening_manifest.yaml as the source of truth hence no need to do any duplication of entries

use yq to parse the hardening and get the filename and url

3ecb23b

run mage instead

dbc3060

improve failure notifications and ownership

4cb4f15

change the notifications and who to contact

b46a1a8

ci: change timeout to 1 hour

15b2fcd

use curl

af590e5

v1v commented Sep 6, 2022

View reviewed changes

x-pack/heartbeat/ironbank/Makefile Outdated Show resolved Hide resolved

support 64 arm and amd

35257ca

v1v requested review from kuisathaverat and a team September 6, 2022 11:14

v1v marked this pull request as ready for review September 6, 2022 11:14

v1v requested review from a team as code owners September 6, 2022 11:14

v1v requested review from fearful-symmetry and faec and removed request for a team September 6, 2022 11:14

cachedout approved these changes Sep 6, 2022

View reviewed changes

kuisathaverat approved these changes Sep 6, 2022

View reviewed changes

amannocci self-requested a review September 6, 2022 16:52

amannocci approved these changes Sep 6, 2022

View reviewed changes

v1v merged commit 7cbb828 into elastic:main Sep 7, 2022

mergify bot mentioned this pull request Sep 7, 2022

[7.17](backport #32982) ironbank: validate heartbeat docker context #33004

Merged

mergify bot pushed a commit that referenced this pull request Sep 7, 2022

ironbank: validate heartbeat docker context (#32982)

ba2d77e

(cherry picked from commit 7cbb828)

mergify bot mentioned this pull request Sep 7, 2022

[8.4](backport #32982) ironbank: validate heartbeat docker context #33005

Merged

mergify bot pushed a commit that referenced this pull request Sep 7, 2022

ironbank: validate heartbeat docker context (#32982)

2c8e979

(cherry picked from commit 7cbb828)

v1v added a commit that referenced this pull request Sep 7, 2022

ironbank: validate heartbeat docker context (#32982) (#33004)

8ea55e4

(cherry picked from commit 7cbb828) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

v1v added a commit that referenced this pull request Sep 7, 2022

ironbank: validate heartbeat docker context (#32982) (#33005)

b418e04

(cherry picked from commit 7cbb828) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

v1v mentioned this pull request May 24, 2023

[Heartbeat] Remove synthetics support for ironbank #35551

Merged

4 tasks

chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023

ironbank: validate heartbeat docker context (#32982)

5cf7315

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ironbank: validate heartbeat docker context #32982

ironbank: validate heartbeat docker context #32982

v1v commented Sep 5, 2022 •

edited

Loading

elasticmachine commented Sep 5, 2022 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

v1v Sep 6, 2022

v1v Sep 6, 2022

v1v Sep 6, 2022

amannocci left a comment

amannocci Sep 6, 2022

v1v Sep 7, 2022

ironbank: validate heartbeat docker context #32982

ironbank: validate heartbeat docker context #32982

Conversation

v1v commented Sep 5, 2022 • edited Loading

What does this PR do?

Why is it important?

Test

CI

Manually

Follow ups

Issues

elasticmachine commented Sep 5, 2022 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

v1v Sep 6, 2022

Choose a reason for hiding this comment

v1v Sep 6, 2022

Choose a reason for hiding this comment

v1v Sep 6, 2022

Choose a reason for hiding this comment

amannocci left a comment

Choose a reason for hiding this comment

amannocci Sep 6, 2022

Choose a reason for hiding this comment

v1v Sep 7, 2022

Choose a reason for hiding this comment

v1v commented Sep 5, 2022 •

edited

Loading

elasticmachine commented Sep 5, 2022 •

edited by jenkins-beats-ci bot

Loading