Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ironbank: support for heartbeat #32954

Merged
merged 11 commits into from
Sep 1, 2022

Conversation

v1v
Copy link
Member

@v1v v1v commented Sep 1, 2022

Motivation/summary

Automate the docker context generation for the Heartbeat IronBank releases, this will allow us to move away from creating those docker context manually in a different repository, since the Platform Release team already provide the automation.

Implementation details

  1. Support for mage ironbank

DoD build context content

As required the build context contains the following files:

  • Dockerfile with specific requirements for DoD (registry args, no internet dependencies, healthcheck, ...)
  • hardening_manifest.yaml with all required dependencies
  • LICENSE specific for DoD
  • README.md with specific content for DoD
  • config/ (optional) directory containing all config files to include into the Docker image
  • scripts/ directory containing all scripts to include into the Docker image (example: entrypoint...)

Why is it important?

No more manual post-release steps to generate the ironbank releases for heartbeat

Actions

  • Generate the ironbank tar.gz docker context following the Unified Release requirements
  • Generate hardening_manifest.yaml that includes all the dependencies.

Test

$ mage ironbank
>> prepareIronbankBuild: prepare the IronBank container context.
>> saveIronbank: save the IronBank container context.
>> creating TAR file from directory: build/heartbeat-ironbank-8.5.0-docker-build-context, target: build/distributions/heartbeat-ironbank-8.5.0-docker-build-context.tar.gz
>> skipping root directory
ironbank ran for 22.143145ms

produced

$ tree build/heartbeat-ironbank-8.5.0-docker-build-context 
build/heartbeat-ironbank-8.5.0-docker-build-context
├── Dockerfile
├── LICENSE
├── README.md
├── hardening_manifest.yaml
└── signatures
    ├── RPM-GPG-KEY-CentOS-Official
    ├── RPM-GPG-KEY-rockyofficial
    └── linux_signing_key.pub

1 directory, 7 files
$ grep -R 8.5.0 build/heartbeat-ironbank-8.5.0-docker-build-context
build/heartbeat-ironbank-8.5.0-docker-build-context/Dockerfile:ARG ELASTIC_STACK=8.5.0
build/heartbeat-ironbank-8.5.0-docker-build-context/README.md:[here](https://www.elastic.co/guide/en/beats/libbeat/current/release-notes-8.5.0.html).
build/heartbeat-ironbank-8.5.0-docker-build-context/hardening_manifest.yaml:  - "8.5.0"
build/heartbeat-ironbank-8.5.0-docker-build-context/hardening_manifest.yaml:  ELASTIC_STACK: "8.5.0"
build/heartbeat-ironbank-8.5.0-docker-build-context/hardening_manifest.yaml:  org.opencontainers.image.version: "8.5.0"
build/heartbeat-ironbank-8.5.0-docker-build-context/hardening_manifest.yaml:  - filename: "heartbeat-8.5.0-linux-x86_64.tar.gz"
build/heartbeat-ironbank-8.5.0-docker-build-context/hardening_manifest.yaml:    url: "<artifact_path>/heartbeat-8.5.0-linux-x86_64.tar.gz"
build/heartbeat-ironbank-8.5.0-docker-build-context/hardening_manifest.yaml:      value: e327d8469bd71ea0d747673bc362240e3ddad672a0eb7fe815a06d1317e74598

Issues

A similar approach was done for other beats.

It also contains the fix put in place in #32867 but for heartbeats

v1v added 11 commits July 26, 2022 10:01
since it requires the tar.gz from the mage package
…ackaging

* upstream/main: (109 commits)
  Add cap_net_raw requirements to heartbeat docs (elastic#32816)
  apply a quick hotfix for having main working properly (elastic#32934)
  action: checks for x-pack/libbeat and libbeat (elastic#32754)
  Update to Go 1.18 in go.mod. (elastic#32940)
  [heartbeat] disable browser code on windows via build tags (elastic#32939)
  action: checks for heartbeat and x-pack/heartbeat (elastic#32749)
  Make event acknowledgment asynchronous in shipper output (elastic#32785)
  [Automation] Update elastic stack version to 8.5.0-fedc3e60 for testing (elastic#32930)
  Preallocate memory to reduce GC load (elastic#32905)
  [Automation] Update elastic stack version to 8.5.0-440e0896 for testing (elastic#32919)
  Skip broken ceph tests. (elastic#32912)
  Use non-deprecated docker image for testing jolokia (elastic#32885)
  update ironbank image product name (elastic#32867)
  ci: pre-commit stage within Jenkins (elastic#32839)
  Fix a couple of bugs in the logic for how AWS metric periods are calculated (elastic#32724)
  [Filebeat] [httpjson] Add support for single string containing multiple relation-types in getRFC5988Link (elastic#32811)
  [Heartbeat] Update HB k8s template to use <Mi> metric (elastic#32801)
  action: checks for metricbeat and x-pack/metricbeat (elastic#32748)
  action: checks for filebeat and x-pack/filebeat (elastic#32746)
  allow for json/ndjson content type with charset (elastic#32767)
  ...
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 1, 2022
@mergify
Copy link
Contributor

mergify bot commented Sep 1, 2022

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @v1v? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@mergify mergify bot assigned v1v Sep 1, 2022
@v1v v1v added automation Team:Automation Label for the Observability productivity team backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.4.0 Automated backport with mergify labels Sep 1, 2022
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Sep 1, 2022
@v1v v1v requested review from a team and kuisathaverat September 1, 2022 10:59
@v1v v1v marked this pull request as ready for review September 1, 2022 10:59
@v1v v1v requested review from a team as code owners September 1, 2022 10:59
@v1v v1v requested review from belimawr, rdner and cmacknz and removed request for a team September 1, 2022 10:59
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-09-01T10:57:00.212+0000

  • Duration: 142 min 4 sec

Test stats 🧪

Test Results
Failed 0
Passed 22747
Skipped 1947
Total 24694

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@v1v
Copy link
Member Author

v1v commented Sep 1, 2022

➜  /tmp gsutil cp gs://beats-ci-artifacts/beats/pull-requests/pr-32954/heartbeat/heartbeat-ironbank-8.5.0-SNAPSHOT-docker-build-context.tar.gz . 
Copying gs://beats-ci-artifacts/beats/pull-requests/pr-32954/heartbeat/heartbeat-ironbank-8.5.0-SNAPSHOT-docker-build-context.tar.gz...
/ [1 files][ 25.7 KiB/ 25.7 KiB]                                                
Operation completed over 1 objects/25.7 KiB.                                     
➜  /tmp tar -xvf heartbeat-ironbank-8.5.0-SNAPSHOT-docker-build-context.tar.gz 
x Dockerfile
x LICENSE
x README.md
x hardening_manifest.yaml
x signatures
x signatures/RPM-GPG-KEY-CentOS-Official
x signatures/RPM-GPG-KEY-rockyofficial
x signatures/linux_signing_key.pub
➜  /tmp grep 8.5.0 *
Dockerfile:ARG ELASTIC_STACK=8.5.0
README.md:[here](https://www.elastic.co/guide/en/beats/libbeat/current/release-notes-8.5.0.html).
hardening_manifest.yaml:  - "8.5.0"
hardening_manifest.yaml:  ELASTIC_STACK: "8.5.0"
hardening_manifest.yaml:  org.opencontainers.image.version: "8.5.0"
hardening_manifest.yaml:  - filename: "heartbeat-8.5.0-linux-x86_64.tar.gz"
hardening_manifest.yaml:    url: "<artifact_path>/heartbeat-8.5.0-linux-x86_64.tar.gz"
hardening_manifest.yaml:      value: e327d8469bd71ea0d747673bc362240e3ddad672a0eb7fe815a06d1317e74598

@v1v v1v merged commit 544cfd6 into elastic:main Sep 1, 2022
@v1v v1v deleted the feature/heartbeat-packaging-simpler branch September 1, 2022 14:39
mergify bot pushed a commit that referenced this pull request Sep 1, 2022
mergify bot pushed a commit that referenced this pull request Sep 1, 2022
v1v added a commit that referenced this pull request Sep 1, 2022
(cherry picked from commit 544cfd6)

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
elasticmachine pushed a commit to emilioalvap/beats that referenced this pull request May 24, 2023
elasticmachine pushed a commit to emilioalvap/beats that referenced this pull request May 24, 2023
emilioalvap added a commit that referenced this pull request May 26, 2023
* [Heartbeat] Remove synthetics support for ironbank

* ci: remove heartbeat ironbank validation

* ci: remove heartbeat ironbank validation scheduler

* remove heartbeat ironbank

* avoid running it

* Revert "ironbank: support for heartbeat (#32954)"

This reverts commit 544cfd6.

* Revert "Revert "ironbank: support for heartbeat (#32954)""

This reverts commit 1e6b816.

* Revert "avoid running it"

This reverts commit 0c533e7.

* Revert "remove heartbeat ironbank"

This reverts commit afa51b7.

* Revert "ci: remove heartbeat ironbank validation scheduler"

This reverts commit e34fa74.

* Revert "ci: remove heartbeat ironbank validation"

This reverts commit b0e0ad0.

* update version

---------

Co-authored-by: Victor Martinez <VictorMartinezRubio@gmail.com>
mergify bot pushed a commit that referenced this pull request May 26, 2023
* [Heartbeat] Remove synthetics support for ironbank

* ci: remove heartbeat ironbank validation

* ci: remove heartbeat ironbank validation scheduler

* remove heartbeat ironbank

* avoid running it

* Revert "ironbank: support for heartbeat (#32954)"

This reverts commit 544cfd6.

* Revert "Revert "ironbank: support for heartbeat (#32954)""

This reverts commit 1e6b816.

* Revert "avoid running it"

This reverts commit 0c533e7.

* Revert "remove heartbeat ironbank"

This reverts commit afa51b7.

* Revert "ci: remove heartbeat ironbank validation scheduler"

This reverts commit e34fa74.

* Revert "ci: remove heartbeat ironbank validation"

This reverts commit b0e0ad0.

* update version

---------

Co-authored-by: Victor Martinez <VictorMartinezRubio@gmail.com>
(cherry picked from commit 3f5afa5)

# Conflicts:
#	dev-tools/packaging/templates/ironbank/heartbeat/hardening_manifest.yaml
mergify bot pushed a commit that referenced this pull request May 26, 2023
* [Heartbeat] Remove synthetics support for ironbank

* ci: remove heartbeat ironbank validation

* ci: remove heartbeat ironbank validation scheduler

* remove heartbeat ironbank

* avoid running it

* Revert "ironbank: support for heartbeat (#32954)"

This reverts commit 544cfd6.

* Revert "Revert "ironbank: support for heartbeat (#32954)""

This reverts commit 1e6b816.

* Revert "avoid running it"

This reverts commit 0c533e7.

* Revert "remove heartbeat ironbank"

This reverts commit afa51b7.

* Revert "ci: remove heartbeat ironbank validation scheduler"

This reverts commit e34fa74.

* Revert "ci: remove heartbeat ironbank validation"

This reverts commit b0e0ad0.

* update version

---------

Co-authored-by: Victor Martinez <VictorMartinezRubio@gmail.com>
(cherry picked from commit 3f5afa5)
mergify bot pushed a commit that referenced this pull request May 26, 2023
* [Heartbeat] Remove synthetics support for ironbank

* ci: remove heartbeat ironbank validation

* ci: remove heartbeat ironbank validation scheduler

* remove heartbeat ironbank

* avoid running it

* Revert "ironbank: support for heartbeat (#32954)"

This reverts commit 544cfd6.

* Revert "Revert "ironbank: support for heartbeat (#32954)""

This reverts commit 1e6b816.

* Revert "avoid running it"

This reverts commit 0c533e7.

* Revert "remove heartbeat ironbank"

This reverts commit afa51b7.

* Revert "ci: remove heartbeat ironbank validation scheduler"

This reverts commit e34fa74.

* Revert "ci: remove heartbeat ironbank validation"

This reverts commit b0e0ad0.

* update version

---------

Co-authored-by: Victor Martinez <VictorMartinezRubio@gmail.com>
(cherry picked from commit 3f5afa5)
emilioalvap added a commit that referenced this pull request May 26, 2023
* [Heartbeat] Remove synthetics support for ironbank

* ci: remove heartbeat ironbank validation

* ci: remove heartbeat ironbank validation scheduler

* remove heartbeat ironbank

* avoid running it

* Revert "ironbank: support for heartbeat (#32954)"

This reverts commit 544cfd6.

* Revert "Revert "ironbank: support for heartbeat (#32954)""

This reverts commit 1e6b816.

* Revert "avoid running it"

This reverts commit 0c533e7.

* Revert "remove heartbeat ironbank"

This reverts commit afa51b7.

* Revert "ci: remove heartbeat ironbank validation scheduler"

This reverts commit e34fa74.

* Revert "ci: remove heartbeat ironbank validation"

This reverts commit b0e0ad0.

* update version

---------

Co-authored-by: Victor Martinez <VictorMartinezRubio@gmail.com>
(cherry picked from commit 3f5afa5)

Co-authored-by: Emilio Alvarez Piñeiro <95703246+emilioalvap@users.noreply.github.com>
emilioalvap added a commit that referenced this pull request May 26, 2023
* [Heartbeat] Remove synthetics support for ironbank

* ci: remove heartbeat ironbank validation

* ci: remove heartbeat ironbank validation scheduler

* remove heartbeat ironbank

* avoid running it

* Revert "ironbank: support for heartbeat (#32954)"

This reverts commit 544cfd6.

* Revert "Revert "ironbank: support for heartbeat (#32954)""

This reverts commit 1e6b816.

* Revert "avoid running it"

This reverts commit 0c533e7.

* Revert "remove heartbeat ironbank"

This reverts commit afa51b7.

* Revert "ci: remove heartbeat ironbank validation scheduler"

This reverts commit e34fa74.

* Revert "ci: remove heartbeat ironbank validation"

This reverts commit b0e0ad0.

* update version

---------

Co-authored-by: Victor Martinez <VictorMartinezRubio@gmail.com>
(cherry picked from commit 3f5afa5)

Co-authored-by: Emilio Alvarez Piñeiro <95703246+emilioalvap@users.noreply.github.com>
chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
* [Heartbeat] Remove synthetics support for ironbank

* ci: remove heartbeat ironbank validation

* ci: remove heartbeat ironbank validation scheduler

* remove heartbeat ironbank

* avoid running it

* Revert "ironbank: support for heartbeat (#32954)"

This reverts commit 544cfd6.

* Revert "Revert "ironbank: support for heartbeat (#32954)""

This reverts commit 1e6b816.

* Revert "avoid running it"

This reverts commit 0c533e7.

* Revert "remove heartbeat ironbank"

This reverts commit afa51b7.

* Revert "ci: remove heartbeat ironbank validation scheduler"

This reverts commit e34fa74.

* Revert "ci: remove heartbeat ironbank validation"

This reverts commit b0e0ad0.

* update version

---------

Co-authored-by: Victor Martinez <VictorMartinezRubio@gmail.com>
emilioalvap added a commit that referenced this pull request Jun 14, 2023
…nbank (#35588)

* [Heartbeat] Remove synthetics support for ironbank (#35551)

* [Heartbeat] Remove synthetics support for ironbank

* ci: remove heartbeat ironbank validation

* ci: remove heartbeat ironbank validation scheduler

* remove heartbeat ironbank

* avoid running it

* Revert "ironbank: support for heartbeat (#32954)"

This reverts commit 544cfd6.

* Revert "Revert "ironbank: support for heartbeat (#32954)""

This reverts commit 1e6b816.

* Revert "avoid running it"

This reverts commit 0c533e7.

* Revert "remove heartbeat ironbank"

This reverts commit afa51b7.

* Revert "ci: remove heartbeat ironbank validation scheduler"

This reverts commit e34fa74.

* Revert "ci: remove heartbeat ironbank validation"

This reverts commit b0e0ad0.

* update version

---------

Co-authored-by: Victor Martinez <VictorMartinezRubio@gmail.com>
(cherry picked from commit 3f5afa5)

# Conflicts:
#	dev-tools/packaging/templates/ironbank/heartbeat/hardening_manifest.yaml

* Manual merge

---------

Co-authored-by: Emilio Alvarez Piñeiro <95703246+emilioalvap@users.noreply.github.com>
Co-authored-by: emilioalvap <emilio.alvarezpineiro@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automation backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.4.0 Automated backport with mergify Team:Automation Label for the Observability productivity team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants