Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use our build-repo for Maven downloads & other updates #953

Merged
merged 8 commits into from
Dec 12, 2024

Conversation

lfrancke
Copy link
Member

@lfrancke lfrancke commented Dec 11, 2024

Description

This PR bundles a few (hopefully) minor changes to our workflows:

  • This uses the Stackable build repo which mirrors maven central and a few other repositories needed by our products
    • We are hoping this is more stable than upstream Maven central where we see frequent connection errors in our builds
  • Adds aCI change to prevent matrix jobs from canceling when one job fails. This should mean there are less jobs to rerun on failure
  • Adds a patch to Druid to not depend on fmpp 0.9.14 anymore which does depend on a version range for Freemarker which would resolve to a SNAPSHOT dependency
  • Adds a patch to Kafka to update the repositories used by Gradle
  • Hopefully consistently adds --batch-mode and --no-transfer-progress to all Maven based builds

Definition of Done Checklist

Make sure all these products build and use the Maven pull-through cache in Nexus:

Products

Tasks

TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

bake --product <product> --image-version <stackable-image-version>
kind load docker-image <image-tagged-with-the-major-version> --name=<name-of-your-test-cluster>

See the output of bake to retrieve the image tag for <image-tagged-with-the-major-version>.

@lfrancke lfrancke force-pushed the push-wtnvozywmlzk branch 3 times, most recently from 285ffac to 9fb6446 Compare December 12, 2024 05:40
lfrancke and others added 5 commits December 12, 2024 06:44
This is because FMPP Maven Plugin depends on FMPP in version 0.9.14
which itself depends on a Freemarker version that has not been pinned.
Instead it specifies a "range" which resolves to a SNAPSHOT version
which we don't want.
NickLarsenNZ
NickLarsenNZ previously approved these changes Dec 12, 2024
Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lfrancke lfrancke marked this pull request as ready for review December 12, 2024 09:24
@lfrancke lfrancke changed the title Use our build-repo for Maven downloads Use our build-repo for Maven downloads & other updates Dec 12, 2024
Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one suggestion

CHANGELOG.md Outdated Show resolved Hide resolved
Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>
@lfrancke lfrancke enabled auto-merge December 12, 2024 09:34
@NickLarsenNZ NickLarsenNZ self-requested a review December 12, 2024 09:36
Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lfrancke lfrancke added this pull request to the merge queue Dec 12, 2024
Merged via the queue into main with commit efd9b9d Dec 12, 2024
3 checks passed
@lfrancke lfrancke deleted the push-wtnvozywmlzk branch December 12, 2024 09:37
NickLarsenNZ added a commit that referenced this pull request Dec 13, 2024
* Use our build-repo for Maven downloads

* ci(build): Continue matrix builds on failure so there is less jobs to rerun when there are transient failures

* Update Spark Mvn build as well

* Hopefully silence Spark build

* Update FMPP version

This is because FMPP Maven Plugin depends on FMPP in version 0.9.14
which itself depends on a Freemarker version that has not been pinned.
Instead it specifies a "range" which resolves to a SNAPSHOT version
which we don't want.

* Add Stackable repo to Kafka build as well

* Update changelog

* Update CHANGELOG.md

Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>

---------

Co-authored-by: Nick Larsen <nick.larsen@stackable.tech>
Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>
NickLarsenNZ added a commit that referenced this pull request Dec 13, 2024
* ci: remove redundant usages of the free-disk-space action

Co-authored-by: Techassi <sascha.lautenschlaeger@stackable.tech>

* ci: Add attempt number to the workflow run name, and URL in the slack message (#941)

Co-authored-by: Techassi <sascha.lautenschlaeger@stackable.tech>

* ci: register a dummy test_build_images workflow which will be completed in a separate PR (#947)

* Use our build-repo for Maven downloads & other updates (#953)

* Use our build-repo for Maven downloads

* ci(build): Continue matrix builds on failure so there is less jobs to rerun when there are transient failures

* Update Spark Mvn build as well

* Hopefully silence Spark build

* Update FMPP version

This is because FMPP Maven Plugin depends on FMPP in version 0.9.14
which itself depends on a Freemarker version that has not been pinned.
Instead it specifies a "range" which resolves to a SNAPSHOT version
which we don't want.

* Add Stackable repo to Kafka build as well

* Update changelog

* Update CHANGELOG.md

Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>

---------

Co-authored-by: Nick Larsen <nick.larsen@stackable.tech>
Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>

* ci: Fix static analysis errors (#955)

* ci: Fix high severity errors

* ci: Set persist-credentials to false

* ci: Move step output to env var

* chore: Add changelog entry

* ci(build): stagger the scheduled builds to ease load on external systems (#954)

---------

Co-authored-by: Techassi <sascha.lautenschlaeger@stackable.tech>
Co-authored-by: Lars Francke <lars.francke@stackable.tech>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants