-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: create CI artifacts for DEV usage #21645
chore: create CI artifacts for DEV usage #21645
Conversation
It will create the artifacts with some requirements related to integrity
]) { | ||
dir("${env.BEATS_FOLDER}") { | ||
sh(label: "Release ${env.BEATS_FOLDER} ${env.PLATFORMS}", script: 'mage package') | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not completely familiar with the CI here, would that run DEV=true mage package
for all the beats, quick grep through the code I don't think it would have any impact.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The release()
method is called for linux/Mac release: https://github.com/elastic/beats/pull/21645/files#diff-bdaa5a61de14efda6eb2bbb2a38fa9a8R93-R148
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DEV
doesn't affect mage package
AFAIK. Does it affect something else?
You can produce "snapshot" artifacts from packaging with SNAPSHOT=true
if that's what you're looking for.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ph are we ok if we use SNAPSHOT
instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DEV is used by agent so it skips verification of the artifacts, agent does verify both normal builds and snapshot artifacts. without this artifacts from PRs will fail on these verifications as they wont be signed and 8.0.0-SNAPSHOT artifact published is different
it is a new flag so it should not alter other behavior
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any other concern about @michalpristas's comment? We'd like to merge this to fix some of the errors we see at the e2e tests
@michalpristas Can you review this too to make sure we don't have side effect? |
@andrewkroh @urso do you see any other implication/concern about this change? We see it super simple to implement, but we also know that the artifacts in the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if this wont affect release builds then it's a ✅ from me
@mdelapenya LGTM. |
Merging as per #21645 (comment) |
…dependencies-goals * upstream/master: (46 commits) Use badger code from upstream repository (elastic#21705) Disable writes sync in persistent cache (elastic#21754) Make API address and Shard ID required in Cloud Foundry settings (elastic#21759) [CI] Support skip-ci label (elastic#21377) Increase recommended memory when deploying in Cloud foundry (elastic#21755) typofix for dns timeout configuration option (elastic#21069) chore: create CI artifacts for DEV usage (elastic#21645) [Ingest Manager] Atomic installed forgotten changelog elastic#21780 [Ingest Manager] Agent atomic installer (elastic#21745) Add missing configuration annotations (elastic#21736) [Filebeat] Add check for context.DeadlineExceeded error (elastic#21732) Remove kafka partition ISR from Metricbeat docs (elastic#21709) Skip flaky test with oauth2 config in httpjson input (elastic#21749) Fix for azure retrieve resource by ids (elastic#21711) Update obs app links (elastic#21682) fix: update fleet test suite name (elastic#21738) Remove dot from file.extension value in Auditbeat FIM (elastic#21644) Fix leaks with metadata processors (elastic#16349) Add istiod metricset (elastic#21519) [Ingest Manager] Change Sync/Close call order (elastic#21735) ...
…ne-2.0-arm * upstream/master: (93 commits) Fix non-windows fields on system/filesystem (elastic#21758) disable TestReceiveEventsAndMetadata/TestSocketCleanup/TestReceiveNewEventsConcurrently in Windows (elastic#21750) Use badger code from upstream repository (elastic#21705) Disable writes sync in persistent cache (elastic#21754) Make API address and Shard ID required in Cloud Foundry settings (elastic#21759) [CI] Support skip-ci label (elastic#21377) Increase recommended memory when deploying in Cloud foundry (elastic#21755) typofix for dns timeout configuration option (elastic#21069) chore: create CI artifacts for DEV usage (elastic#21645) [Ingest Manager] Atomic installed forgotten changelog elastic#21780 [Ingest Manager] Agent atomic installer (elastic#21745) Add missing configuration annotations (elastic#21736) [Filebeat] Add check for context.DeadlineExceeded error (elastic#21732) Remove kafka partition ISR from Metricbeat docs (elastic#21709) Skip flaky test with oauth2 config in httpjson input (elastic#21749) Fix for azure retrieve resource by ids (elastic#21711) Update obs app links (elastic#21682) fix: update fleet test suite name (elastic#21738) Remove dot from file.extension value in Auditbeat FIM (elastic#21644) Fix leaks with metadata processors (elastic#16349) ...
It will create the artifacts with some requirements related to integrity
It will create the artifacts with some requirements related to integrity
* feat: add a new step to run the e2e tests for certain parts of Beats (#21100) * feat: add a new step to run the e2e tests for certain parts of Beats We are going to trigger the tests for those parts affected by the elastic-agent, filebeat, or metricbeat, because those are the ones we verify in the e2e-testing suite * chore: do not include heartbeat * feat: trigger the e2e tests * fix: use relative path * chore: use proper target branch name for PRs * chore: use different tag * fix: use proper env variable * chore: pass github checks context to downstream job * chore: revert shared lib version Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: add BASE_DIR env variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove duplicated env * ffix: add param comma separator * fix: wrong copy&paste * chore: move e2e GH check out of the release context * chore: simplify conditional logic * chore: refine execution of test suites * fix: use proper parameter name * chore: set metricbeat version * chore: remove slack notifications on PRs * chore: update parameter * chore: run multiple test suites per beat type Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [E2E Tests] fix: set versions ony for PRs (#21608) * fix: set versions ony for PRs We want to use default versions per branch when running after a merge * fix: add trailing comma Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [CI: Packaging] fix: push ubi8 images too (#21621) * fix: push ubi8 images too * chore: enhance retries Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: use variables in log * chore: add "-oss" images Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * fix: remove extra curly brace in script (#21692) * fix: remove extra curly brace * chore: proper indent * fix: update fleet test suite name (#21738) * chore: create CI artifacts for DEV usage (#21645) It will create the artifacts with some requirements related to integrity * chore: simplify triggering the E2E tests for Beats (#21790) * chore: pass beat as a method argument (no side-effects) * chore: run tests in a separate stage * fix: use parenthesis * chore: update comment * chore: do not trigger E2E tests if no suite was added * fix: use missing curly brackets * fix: wrong closure wrapping * fix: condition was not set * chore: delegate variant pushes to the right method (#21861) * fix: delegate pushes to variants * chore: group conditions for x-pack * chore: simplify with endsWith Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * feat: package aliases for snapshots (#21960) * feat: push aliases for docker images * feat: build alias for snapshots * fix: only update alias on snapshots Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * fix: wrong image name for alias * fix: reuse variable as groovy does not hide variables by scope * chore: extract common logic to a method * Revert "fix: only update alias on snapshots" This reverts commit cff2cef. * Revert "feat: build alias for snapshots" This reverts commit 707e0d7. * chore: do not push aliases for PRs Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * chore: Use third number as x Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
* feat: add a new step to run the e2e tests for certain parts of Beats (#21100) * feat: add a new step to run the e2e tests for certain parts of Beats We are going to trigger the tests for those parts affected by the elastic-agent, filebeat, or metricbeat, because those are the ones we verify in the e2e-testing suite * chore: do not include heartbeat * feat: trigger the e2e tests * fix: use relative path * chore: use proper target branch name for PRs * chore: use different tag * fix: use proper env variable * chore: pass github checks context to downstream job * chore: revert shared lib version Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: add BASE_DIR env variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove duplicated env * ffix: add param comma separator * fix: wrong copy&paste * chore: move e2e GH check out of the release context * chore: simplify conditional logic * chore: refine execution of test suites * fix: use proper parameter name * chore: set metricbeat version * chore: remove slack notifications on PRs * chore: update parameter * chore: run multiple test suites per beat type Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [E2E Tests] fix: set versions ony for PRs (#21608) * fix: set versions ony for PRs We want to use default versions per branch when running after a merge * fix: add trailing comma Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [CI: Packaging] fix: push ubi8 images too (#21621) * fix: push ubi8 images too * chore: enhance retries Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: use variables in log * chore: add "-oss" images Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * fix: remove extra curly brace in script (#21692) * fix: remove extra curly brace * chore: proper indent * fix: update fleet test suite name (#21738) * chore: create CI artifacts for DEV usage (#21645) It will create the artifacts with some requirements related to integrity * chore: simplify triggering the E2E tests for Beats (#21790) * chore: pass beat as a method argument (no side-effects) * chore: run tests in a separate stage * fix: use parenthesis * chore: update comment * chore: do not trigger E2E tests if no suite was added * fix: use missing curly brackets * fix: wrong closure wrapping * fix: condition was not set * chore: delegate variant pushes to the right method (#21861) * fix: delegate pushes to variants * chore: group conditions for x-pack * chore: simplify with endsWith Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> # Conflicts: # .ci/packaging.groovy * feat: package aliases for snapshots (#21960) * feat: push aliases for docker images * feat: build alias for snapshots * fix: only update alias on snapshots Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * fix: wrong image name for alias * fix: reuse variable as groovy does not hide variables by scope * chore: extract common logic to a method * Revert "fix: only update alias on snapshots" This reverts commit cff2cef. * Revert "feat: build alias for snapshots" This reverts commit 707e0d7. * chore: do not push aliases for PRs Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * chore: Use third number as x Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
* feat: add a new step to run the e2e tests for certain parts of Beats (elastic#21100) * feat: add a new step to run the e2e tests for certain parts of Beats We are going to trigger the tests for those parts affected by the elastic-agent, filebeat, or metricbeat, because those are the ones we verify in the e2e-testing suite * chore: do not include heartbeat * feat: trigger the e2e tests * fix: use relative path * chore: use proper target branch name for PRs * chore: use different tag * fix: use proper env variable * chore: pass github checks context to downstream job * chore: revert shared lib version Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: add BASE_DIR env variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove duplicated env * ffix: add param comma separator * fix: wrong copy&paste * chore: move e2e GH check out of the release context * chore: simplify conditional logic * chore: refine execution of test suites * fix: use proper parameter name * chore: set metricbeat version * chore: remove slack notifications on PRs * chore: update parameter * chore: run multiple test suites per beat type Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [E2E Tests] fix: set versions ony for PRs (elastic#21608) * fix: set versions ony for PRs We want to use default versions per branch when running after a merge * fix: add trailing comma Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [CI: Packaging] fix: push ubi8 images too (elastic#21621) * fix: push ubi8 images too * chore: enhance retries Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: use variables in log * chore: add "-oss" images Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * fix: remove extra curly brace in script (elastic#21692) * fix: remove extra curly brace * chore: proper indent * fix: update fleet test suite name (elastic#21738) * chore: create CI artifacts for DEV usage (elastic#21645) It will create the artifacts with some requirements related to integrity * chore: simplify triggering the E2E tests for Beats (elastic#21790) * chore: pass beat as a method argument (no side-effects) * chore: run tests in a separate stage * fix: use parenthesis * chore: update comment * chore: do not trigger E2E tests if no suite was added * fix: use missing curly brackets * fix: wrong closure wrapping * fix: condition was not set * chore: delegate variant pushes to the right method (elastic#21861) * fix: delegate pushes to variants * chore: group conditions for x-pack * chore: simplify with endsWith Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * feat: package aliases for snapshots (elastic#21960) * feat: push aliases for docker images * feat: build alias for snapshots * fix: only update alias on snapshots Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * fix: wrong image name for alias * fix: reuse variable as groovy does not hide variables by scope * chore: extract common logic to a method * Revert "fix: only update alias on snapshots" This reverts commit cff2cef. * Revert "feat: build alias for snapshots" This reverts commit 707e0d7. * chore: do not push aliases for PRs Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * chore: Use third number as x Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> (cherry picked from commit 624c459)
…23142) Backports the following commits to 7.x: * feat: add a new step to run the e2e tests for certain parts of Beats (#21100) * [E2E Tests] fix: set versions ony for PRs (#21608) * [CI: Packaging] fix: push ubi8 images too (#21621) * fix: remove extra curly brace in script (#21692) * fix: update fleet test suite name (#21738) * chore: create CI artifacts for DEV usage (#21645) * chore: simplify triggering the E2E tests for Beats (#21790) * chore: delegate variant pushes to the right method (#21861) * feat: package aliases for snapshots (#21960) * fix: use proper param name for e2e tests (#22836) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
…ports (elastic#23142) Backports the following commits to 7.x: * feat: add a new step to run the e2e tests for certain parts of Beats (elastic#21100) * [E2E Tests] fix: set versions ony for PRs (elastic#21608) * [CI: Packaging] fix: push ubi8 images too (elastic#21621) * fix: remove extra curly brace in script (elastic#21692) * fix: update fleet test suite name (elastic#21738) * chore: create CI artifacts for DEV usage (elastic#21645) * chore: simplify triggering the E2E tests for Beats (elastic#21790) * chore: delegate variant pushes to the right method (elastic#21861) * feat: package aliases for snapshots (elastic#21960) * fix: use proper param name for e2e tests (elastic#22836) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 1006bd9)
…23142) (#23396) Backports the following commits to 7.x: * feat: add a new step to run the e2e tests for certain parts of Beats (#21100) * [E2E Tests] fix: set versions ony for PRs (#21608) * [CI: Packaging] fix: push ubi8 images too (#21621) * fix: remove extra curly brace in script (#21692) * fix: update fleet test suite name (#21738) * chore: create CI artifacts for DEV usage (#21645) * chore: simplify triggering the E2E tests for Beats (#21790) * chore: delegate variant pushes to the right method (#21861) * feat: package aliases for snapshots (#21960) * fix: use proper param name for e2e tests (#22836) (cherry picked from commit 1006bd9) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
* feat: add a new step to run the e2e tests for certain parts of Beats (elastic#21100) * feat: add a new step to run the e2e tests for certain parts of Beats We are going to trigger the tests for those parts affected by the elastic-agent, filebeat, or metricbeat, because those are the ones we verify in the e2e-testing suite * chore: do not include heartbeat * feat: trigger the e2e tests * fix: use relative path * chore: use proper target branch name for PRs * chore: use different tag * fix: use proper env variable * chore: pass github checks context to downstream job * chore: revert shared lib version Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: add BASE_DIR env variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove duplicated env * ffix: add param comma separator * fix: wrong copy&paste * chore: move e2e GH check out of the release context * chore: simplify conditional logic * chore: refine execution of test suites * fix: use proper parameter name * chore: set metricbeat version * chore: remove slack notifications on PRs * chore: update parameter * chore: run multiple test suites per beat type Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [E2E Tests] fix: set versions ony for PRs (elastic#21608) * fix: set versions ony for PRs We want to use default versions per branch when running after a merge * fix: add trailing comma Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [CI: Packaging] fix: push ubi8 images too (elastic#21621) * fix: push ubi8 images too * chore: enhance retries Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: use variables in log * chore: add "-oss" images Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * fix: remove extra curly brace in script (elastic#21692) * fix: remove extra curly brace * chore: proper indent * fix: update fleet test suite name (elastic#21738) * chore: create CI artifacts for DEV usage (elastic#21645) It will create the artifacts with some requirements related to integrity * chore: simplify triggering the E2E tests for Beats (elastic#21790) * chore: pass beat as a method argument (no side-effects) * chore: run tests in a separate stage * fix: use parenthesis * chore: update comment * chore: do not trigger E2E tests if no suite was added * fix: use missing curly brackets * fix: wrong closure wrapping * fix: condition was not set * chore: delegate variant pushes to the right method (elastic#21861) * fix: delegate pushes to variants * chore: group conditions for x-pack * chore: simplify with endsWith Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> # Conflicts: # .ci/packaging.groovy * feat: package aliases for snapshots (elastic#21960) * feat: push aliases for docker images * feat: build alias for snapshots * fix: only update alias on snapshots Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * fix: wrong image name for alias * fix: reuse variable as groovy does not hide variables by scope * chore: extract common logic to a method * Revert "fix: only update alias on snapshots" This reverts commit cff2cef. * Revert "feat: build alias for snapshots" This reverts commit 707e0d7. * chore: do not push aliases for PRs Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * chore: Use third number as x Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
What does this PR do?
It adds the DEV=true env variable when packaging the artifacts, so that they are created in dev mode.
Why is it important?
When using the
observability-ci
images, which are consumed by different CI projects, the artifacts come with a constraint to check integrity from a .ASC file. Setting this variable will disable that constraint, making it possible to consume the artifacts in a smooth manner on CI time.Checklist
CHANGELOG.next.asciidoc
orCHANGELOG-developer.next.asciidoc
.