Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow independent distribution releases #429

Closed
TylerHelmuth opened this issue Nov 21, 2023 · 10 comments
Closed

Allow independent distribution releases #429

TylerHelmuth opened this issue Nov 21, 2023 · 10 comments
Labels
github_actions Pull requests that update GitHub Actions code

Comments

@TylerHelmuth
Copy link
Member

Our current release problem has a couple problems:

  1. We cannot release distributions independently. If we want to release a patch for the Contrib Distribution we have to also release a patch for the Core distribution.
    • Related, if the job fails after 1 distribution is published but not the other, we can't rerun the job, we have to push new tags.
  2. We have run into diskspace issues in the past where the job fills up the allotted amount. The more we add to contrib and the more distributions we create the more this will be an issue.

As we look to support more distributions the connectedness of our CI will be a roadblock.

I propose we split the release of each distribution into it's own workflow with their own tags, such as v0.90.0-contrib or v0.90.0-core. The logic of each workflow can reuse a shared action, but I believe we'd need a goreleaser.yaml per distribution.

If each distribution has its own workflow we can:

  1. Release each distribution independently via their own tags. If necessary we can add a make target to simplify the Release Manager's steps.
  2. Each distribution will have more disk space (or any resource, including time) to work with.

Based on #357 (comment).

@TylerHelmuth TylerHelmuth added the github_actions Pull requests that update GitHub Actions code label Nov 21, 2023
@jpkrohling
Copy link
Member

I agree with that. I've been experimenting with splitting the distributions on their own workflows as well on a personal repository, and I prefer that approach than what I originally built in this repo here.

While I think it's great to have the ability to release different distributions separately, I don't think it's a good idea to do so: I think it might be confusing for users to have, say, otelcol-contrib-v1.0.1 and not an otelcol-core-v1.0.1 (and vice-versa).

Related, if the job fails after 1 distribution is published but not the other, we can't rerun the job, we have to push new tags.

Care to expand on this part? Is this only about the step where tags are pushed, or is it something else? I think the release should already be idempotent for most steps.

@codeboten
Copy link
Contributor

When there's an issue with a release, we delete the tags and push them again. There's a note about tags in the README:

⚠️ Important note: Git tags in this repository may change at any time to fix any issues found during a release. They are only meant to trigger Github releases and should not be relied upon.

I agree that there needs to be separate workflows for the distros, otherwise we're limited in what we can release in the future. We've already seen this in the past.

Re. different versions for different distros, I'm not sure I agree that we should always keep the different versions in sync. I'm thinking of the world of the future where most of the components in core are stable and no longer need to release on a frequent cadence. This would mean that we could end up in a scenario where core and contrib are at different versions and that should be perfectly acceptable.

I understand that's not where we are today, but I also don't see this as a dealbreaker when it comes to how releases should be made.

@TylerHelmuth
Copy link
Member Author

Care to expand on this part? Is this only about the step where tags are pushed, or is it something else? I think the release should already be idempotent for most steps.

I've not looked into it in depth, but if the action goreleaser/goreleaser-action@v5 we use for publishing has an error it may be possible to push 1 image but not others. This is currently a theoretic issue.

It sounds like we are in agreement to split up the release jobs. I could get started on that as we continue to discuss tagging.

@TylerHelmuth
Copy link
Member Author

Spent some time on this today and ran into this problem: https://github.com/orgs/goreleaser/discussions/4469. A workaround would be to use multiple goreleaser configuration files, but I'd like to avoid that if possible.

@TylerHelmuth
Copy link
Member Author

@jpkrohling looking at your repo I do like how there are individual gorelease configs. This will be really useful for certain distributions, like k8s or sidecar, that don't need to build specific images/binaries.

If we went that route I think we'd no longer auto-generate the file, are you ok with that?

@jpkrohling
Copy link
Member

I'm fine with that, but I'd do it as part of a next step, so that we have smaller changes applied at each iteration.

@jpkrohling
Copy link
Member

If we went that route I think we'd no longer auto-generate the file, are you ok with that?

Can you clarify this part? I see benefit in still generating this, but perhaps you found limitations around that?

@TylerHelmuth
Copy link
Member Author

I was wrong, we can still generate the files like you do in your repo.

@TylerHelmuth
Copy link
Member Author

@jpkrohling here is a PR for what the generated files can look like: #486.

I think the auto-generated files will break down if we ever make any specific distributions, like a Windows distribution, but it is possible that we can adjust the code for those cases. For now, generating the files ensures the distributions are the same.

@TylerHelmuth
Copy link
Member Author

We now have a CI process that separates the builds!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
github_actions Pull requests that update GitHub Actions code
Projects
None yet
Development

No branches or pull requests

3 participants