Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRT Onboarding #11564

Merged
merged 30 commits into from
Feb 25, 2022
Merged

CRT Onboarding #11564

merged 30 commits into from
Feb 25, 2022

Conversation

mdeggies
Copy link
Member

@mdeggies mdeggies commented Feb 12, 2022

Onboard Packer OSS to Common Release Tooling

There are a few important changes to call out in this PR:

  • GitHub Actions is being used as the CI provider to build the go binary artifacts, docker image, and linux packages. This logic lives in .github/workflows/build.yml and workflow runs are available under https://github.com/hashicorp/packer/actions.
  • Artifact builds will continue to run on PR's and merges to master. This behavior can be changed by updating the on: logic in the .github/workflows/build.yml file.
  • The .release/ci.hcl file contains some metadata for our release service. The only part you'll likely ever need to change is the release_branches array -- when you want to release off of a new release branch, update this value to the release branch name and be sure to update the branches array in .github/workflows/build.yml to the same value. This should be set to either main, the release branch that we'll be releasing off of (once release branches are adopted), or a test branch that you are working off of if you need to validate every part of the pipeline before merging.
  • The netcgo tag has been added for darwin binaries to avoid issues that customers have faced connecting over a VPN. More detail in this RFC. After this PR is merged, the next Packer release will close Packer 1.5.6+ does not use macOS system DNS resolver and cannot connect to host #9710

Things to call out in the changelog for the next release using CRT:

  • The dockerfile that is used to build the packer docker image available at hashicorp/packer now lives in the root of this repo. The README that describes how to build the docker targets defined in the Dockerfile are available under ./release/docker/README.md.
  • Docker images are now available for all supported architectures that the linux binaries are built for including arm, arm64, 386, and amd64
  • Packer's linux package service configs and pre/post install scripts are now available under .release/linux
  • Packer's linux packages are now available for all supported linux architectures including arm, arm64, 386, and amd64

Other odds and ends to note:

Still TODO:

  • Add website-docker-image job secrets to GitHub Actions
  • Add acceptance test secrets to GitHub Actions and change the workflow to run on either merges to master or at a scheduled time (e.g. nightly)
  • Port over nightly github prereleases to GHA
  • Sync with Packer team to validate built artifacts and align on naming convention for docker images pushed to DockerHub and AWS ECR

@mdeggies mdeggies requested a review from a team as a code owner February 12, 2022 04:24
@mdeggies mdeggies requested review from a team, alvin-huang and claire-labry and removed request for a team February 15, 2022 04:04
Copy link

@samsalisbury samsalisbury left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mdeggies I only reviewed the docker parts, I hope this is helpful, let me know if I can help with addressing any of the issues or improving documentation around them, thanks :)

.github/workflows/build.yml Outdated Show resolved Hide resolved
.github/workflows/build.yml Outdated Show resolved Hide resolved
Dockerfile Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
Dockerfile Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
@nywilken
Copy link
Contributor

nywilken commented Feb 15, 2022

Artifact builds will now only run on merges to the release branches or to main

Are you referring to the Artifact builds that get built from PRs? For example https://app.circleci.com/pipelines/github/hashicorp/packer/11432/workflows/f32c1dc5-9e34-4e48-b267-7d2f6e46b698/jobs/140848

Copy link
Contributor

@nywilken nywilken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking really good. Thank you for opening up this PR and helping with the onboarding to CRT. I left a couple of questions so far.

I'll continue reviewing and look to test the built artifacts.

.github/workflows/build.yml Outdated Show resolved Hide resolved
.github/workflows/acceptance-test.yml Outdated Show resolved Hide resolved

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Co-authored-by: Wilken Rivera <wilken@hashicorp.com>
Dockerfile Outdated Show resolved Hide resolved
.release/ci.hcl Outdated Show resolved Hide resolved
.release/ci.hcl Outdated Show resolved Hide resolved
@alvin-huang
Copy link
Contributor

@kpenfound would you mind double checking the Linux packaging files?

Makefile Show resolved Hide resolved
Copy link
Contributor

@nywilken nywilken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good to me. I left two questions.

Previously, upon a successful release the CHANGELOG would be updated to reflect the latest release version. Is that handled in the CRT? Or do we need to add the upcoming version information ourselves after a release. It is okay if we do. I'm just asking so that I can update our docs.

@@ -1,11 +1,14 @@
name: "Acceptance Test"
#
# This GitHub action runs Packer's acceptance tests every night.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 this is going to be awesome.

@@ -0,0 +1 @@
1.17.5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to update this file when upgrading to a new version of Go?

Copy link
Member Author

@mdeggies mdeggies Feb 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! If the version of go changes that is used to build packer in CI, this will need to be updated.

build template.json
```

For the [manual installation](https://www.packer.io/docs/plugins#installing-plugins) of third-party plugins, we recommended that plugin binaries are placed under a sub-directory under the working directory. Add `-e PACKER_PLUGIN_PATH=/workspace/<subdirectory_plugin_path>` to the command above to tell Packer where the plugins are.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to @hc-github-team-packer we need to update this README once #11598 is complete

@@ -70,3 +70,8 @@
/post-processor/yandex-export/ @GennadySpb
/post-processor/yandex-import/ @GennadySpb
/post-processor/vsphere-template/ nelson@bennu.cl

# release configuration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed an update to fix the issue. This may cause a conflict here that needs to be pulled in.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great. TY!


# This is used for release builds by .github/workflows/build.yml
version:
@$(CURDIR)/scripts/version.sh version/version.go
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@mdeggies
Copy link
Member Author

mdeggies commented Feb 25, 2022

Previously, upon a successful release the CHANGELOG would be updated to reflect the latest release version. Is that handled in the CRT? Or do we need to add the upcoming version information ourselves after a release. It is okay if we do. I'm just asking so that I can update our docs.

We don't have this automated in CRT yet, but it's on our short term roadmap, so hopefully we'll be able to get to it soon. We post updates about new / changing functionality over in #feed-crt-newsletter, so feel free to hop in there to subscribe

Copy link
Contributor

@nywilken nywilken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are going to officially release next week. So we will walk through the whole process. If there are any kinks we can address them in the actual release workflow. Does that sound good to you?

If so, please feel free to merge when ready.

@mdeggies mdeggies changed the title WIP: CRT Onboarding CRT Onboarding Feb 25, 2022
@mdeggies mdeggies merged commit b63a9c9 into master Feb 25, 2022
@mdeggies mdeggies deleted the crt-onboarding branch February 25, 2022 23:56
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Packer 1.5.6+ does not use macOS system DNS resolver and cannot connect to host
5 participants