From f904964e7f212a7e2398f2289c96182dd087fd65 Mon Sep 17 00:00:00 2001 From: milldr Date: Wed, 28 Aug 2024 14:50:02 -0400 Subject: [PATCH 1/4] Update webhook setup instructions in philips-labs-github-runners --- docs/layers/github-actions/philips-labs-github-runners.mdx | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/layers/github-actions/philips-labs-github-runners.mdx b/docs/layers/github-actions/philips-labs-github-runners.mdx index 9a31d1760..666492881 100644 --- a/docs/layers/github-actions/philips-labs-github-runners.mdx +++ b/docs/layers/github-actions/philips-labs-github-runners.mdx @@ -136,8 +136,7 @@ Follow the guide with the upstream module, [philips-labs/terraform-aws-github-ru 1. Provide the webhook secret (`terraform output -raw `). 1. In the _"Permissions & Events"_ section and then _"Subscribe to Events"_ subsection, check _"Workflow Job"_. - 1. Ensure the webhook for the GitHub app is enabled and pointing to the output of the module. - - The endpoint can be found from `atmos terraform output philips-labs-github-runners -s core-use1-auto 'webhook'` + 1. Ensure the webhook for the GitHub app is enabled and pointing to the output of the module. The endpoint can be found from `atmos terraform output philips-labs-github-runners -s core-use1-auto 'webhook'` From 334eeec43020ea7f7ea328ae2a881de53f1cb18f Mon Sep 17 00:00:00 2001 From: Dan Miller Date: Wed, 28 Aug 2024 12:58:34 -0700 Subject: [PATCH 2/4] Update docs/layers/github-actions/philips-labs-github-runners.mdx Co-authored-by: Erik Osterman (CEO @ Cloud Posse) --- docs/layers/github-actions/philips-labs-github-runners.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/github-actions/philips-labs-github-runners.mdx b/docs/layers/github-actions/philips-labs-github-runners.mdx index 666492881..734437444 100644 --- a/docs/layers/github-actions/philips-labs-github-runners.mdx +++ b/docs/layers/github-actions/philips-labs-github-runners.mdx @@ -136,7 +136,7 @@ Follow the guide with the upstream module, [philips-labs/terraform-aws-github-ru 1. Provide the webhook secret (`terraform output -raw `). 1. In the _"Permissions & Events"_ section and then _"Subscribe to Events"_ subsection, check _"Workflow Job"_. - 1. Ensure the webhook for the GitHub app is enabled and pointing to the output of the module. The endpoint can be found from `atmos terraform output philips-labs-github-runners -s core-use1-auto 'webhook'` + 1. Ensure the webhook for the GitHub app is enabled and pointing to the output of the module. The endpoint can be found by running `atmos terraform output philips-labs-github-runners -s core-use1-auto 'webhook'` From a95e6a003ba4d0c306e04243bd85a86932f3c6ba Mon Sep 17 00:00:00 2001 From: milldr Date: Thu, 29 Aug 2024 14:51:55 -0400 Subject: [PATCH 3/4] refactor philips labs docs --- .../philips-labs-github-runners.mdx | 180 ++++++++---------- 1 file changed, 83 insertions(+), 97 deletions(-) diff --git a/docs/layers/github-actions/philips-labs-github-runners.mdx b/docs/layers/github-actions/philips-labs-github-runners.mdx index 666492881..44cb5bbf3 100644 --- a/docs/layers/github-actions/philips-labs-github-runners.mdx +++ b/docs/layers/github-actions/philips-labs-github-runners.mdx @@ -2,12 +2,16 @@ title: Philips Labs GitHub Action Runners sidebar_label: Philips Labs Action Runners --- -import Intro from '@site/src/components/Intro'; -import KeyPoints from '@site/src/components/KeyPoints'; -import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; -import Steps from '@site/src/components/Steps'; -import Step from '@site/src/components/Step'; -import StepNumber from '@site/src/components/StepNumber'; + +import Intro from "@site/src/components/Intro"; +import KeyPoints from "@site/src/components/KeyPoints"; +import AtmosWorkflow from "@site/src/components/AtmosWorkflow"; +import Steps from "@site/src/components/Steps"; +import Step from "@site/src/components/Step"; +import StepNumber from "@site/src/components/StepNumber"; +import TaskList from "@site/src/components/TaskList"; +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; If we are not deploying EKS, it's not worth the additional effort to set up Self-Hosted runners on EKS. Instead, we @@ -18,13 +22,13 @@ import StepNumber from '@site/src/components/StepNumber'; ## Quick Start -| Steps | Actions | Calling Workflow | -| :------------------------------------------- | :------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | -| 1. Generate GitHub App Private Key | Set SSM Param `"/pl-github-runners/key"` to App private Key base64 encoded | `atmos workflow deploy/pl-github-runners -f github` | -| 2. Note GitHub ID | Set SSM Param `"/pl-github-runners/id"` to the GitHub App ID | `atmos workflow deploy/pl-github-runners -f github` | -| 3. Deploy GitHub OIDC Provider | Deploy GitHub OIDC to every needed account | `atmos workflow deploy/github-oidc-provider -f github` | -| 4. Deploy GitHub Runners | Deploy the GitHub runners | `atmos workflow deploy/pl-github-runners -f github` | -| 5. Update Webhook (if changed or redeployed) | Update the GitHub App Webhook | (if `enable_update_github_app_webhook: true`) `atmos workflow deploy/pl-github-runners -f github`. Otherwise Manual | +| Steps | Actions | +| :------------------------------------------------- | :-------------------------------------------------------------------------------------- | +| 1. Create GitHub App | ClickOps | +| 2. Upload GitHub App ID and Private Key to AWS SSM | Set SSM Param `"/pl-github-runners/id"` and `"/pl-github-runners/key"` (base64 encoded) | +| 3. Deploy GitHub OIDC Provider | Deploy GitHub OIDC to every needed account | +| 4. Deploy GitHub Runners | `atmos terraform deploy philips-labs-github-runners -s core-use1-auto` | +| 5. Update Webhook (if changed or redeployed) | ClickOps | ## Deploy @@ -35,9 +39,20 @@ hand the setup back to Cloud Posse. Then Cloud Posse can deploy the component an customer will then need to add the webhook to the GitHub App and ensure the App is installed to all relevant GitHub repositories. -Follow the guide with the upstream module, [philips-labs/terraform-aws-github-runner](https://github.com/philips-labs/terraform-aws-github-runner#setup-github-app-part-1), or follow the steps below. +Follow the guide with the upstream module, +[philips-labs/terraform-aws-github-runner](https://github.com/philips-labs/terraform-aws-github-runner#setup-github-app-part-1), +or follow the steps below. + + ### Vendor Components + + Vendor in the necessary components with the following workflow: + + + + + ### Create the GitHub App @@ -53,33 +68,41 @@ Follow the guide with the upstream module, [philips-labs/terraform-aws-github-ru 1. Choose a name 1. Choose a website (mandatory, not required for the module). 1. Disable the webhook for now (we will configure this later or create an alternative webhook). - 1. Add the following permission: - ```diff - # Required Permissions for Repository Runners: - ## Permissions for all runners: - ### Repository: - + Actions: Read-only (check for queued jobs) - + Checks: Read-only (receive events for new builds) - + Metadata: Read-only (default/required) - - ## Permissions for repository-level runners only: - ### Repository: - + Administration: Read & write (to register runner) - - ## Permissions for organization-level runners only: - ### Organization - + Self-hosted runners: Read & write (to register runner) - ``` + 1. Add the following permission for your chosen runner scope: + + + + #### Repository Permissions + + - Actions: Read-only (check for queued jobs) + - Checks: Read-only (receive events for new builds) + - Metadata: Read-only (default/required) + - Administration: Read & write (to register runner) + + + + + #### Repository Permissions + + - Actions: Read-only (check for queued jobs) + - Checks: Read-only (receive events for new builds) + - Metadata: Read-only (default/required) + + #### Organization Permissions + + - Self-hosted runners: Read & write (to register runner) + + + 1. Generate a Private Key + 1. If you are working with Cloud Posse, upload this Private Key and GitHub App ID to 1Password and inform Cloud Posse. Otherwise, continue to the next step. - If you are working with Cloud Posse, upload this Private Key and GitHub App ID to 1Password and inform Cloud Posse. - Otherwise, continue with the component deployment in `core-use1-auto`. - ### Deploy the `philips-labs-github-runner` Component + ### Upload AWS SSM Parameters :::tip @@ -88,30 +111,36 @@ Follow the guide with the upstream module, [philips-labs/terraform-aws-github-ru ::: - Run the `deploy/pl-github-runners` workflow with `atmos` to write the GitHub App information to the `core-use1-auto` - SSM account and deploy the component. - - - - This is the same as the following steps: + Now that the GitHub App has been created, upload the Private Key and GitHub App ID to AWS SSM Parameter Store in `core-use1-auto` (or your chosen region). - 1. Upload the PEM file key to the specified ssm path, `/pl-github-runners/key`, in `core-use1-auto` as a base64 - encoded string. + 1. Upload the PEM file key to the specified ssm path, `/pl-github-runners/key`, in `core-use1-auto` as a base64 encoded string. 2. Upload the GitHub App ID to the specified ssm path, `/pl-github-runners/id`, in `core-use1-auto`. - 3. Deploy the `philips-labs-github-runners` component to `core-use1-auto`. Run this with the following: + - ```console - atmos terraform apply philips-labs-github-runners -s core-use1-auto - ``` + Or run the `upload/pl-secrets` workflow with `atmos` to write the GitHub App information to the `core-use1-auto` SSM account and deploy the component. - Once the component is deployed, save the webhook URL and secret in 1Password. The endpoint can be found with the - following: + + + + + + ### Deploy GitHub OIDC Providers + + First deploy the GitHub OIDC provider to all accounts where we want to grant GitHub access. The typical list of accounts + is included with the `deploy/github-oidc-provider` workflow; run the following with `SuperAdmin`: + + + + + + + ### Deploy the Philips Labs GitHub Runners + + Now that the GitHub App has been created and the SSM parameters have been uploaded, deploy the `philips-labs-github-runners` component. + + - ```console - atmos terraform output philips-labs-github-runners -s core-use1-auto 'webhook' - ``` - @@ -135,53 +164,10 @@ Follow the guide with the upstream module, [philips-labs/terraform-aws-github-ru 1. Provide the webhook url, should be part of the output of terraform. 1. Provide the webhook secret (`terraform output -raw `). 1. In the _"Permissions & Events"_ section and then _"Subscribe to Events"_ subsection, check _"Workflow Job"_. - 1. Ensure the webhook for the GitHub app is enabled and pointing to the output of the module. The endpoint can be found from `atmos terraform output philips-labs-github-runners -s core-use1-auto 'webhook'` - - - - ## Vendor Components - - Vendor in the necessary components with the following workflow: - - - - ## Deploy the Philips Labs Runners - - :::info - - You can run all workflows at once in the right order with - - - - ::: - - - - ### GitHub OIDC Provider - - First deploy the GitHub OIDC provider to all accounts where we want to grant GitHub access. The typical list of accounts - is included with the `deploy/github-oidc-provider` workflow; run the following with `SuperAdmin`: - - - - - - ### GitHub Runners - - To deploy the self-hosted runners themselves, first verify [the GitHub App requirement](#Requirements) is complete - including both SSM parameters. Next we will deploy the token rotator and then the runners themselves. - - - - If all goes well, you should now see self-hosted runners registered to your infrastructure repository in GitHub. If not, - read through the attached [FAQs](#FAQ). - - - ## Usage @@ -212,9 +198,9 @@ Remove the `terraform` label from the default runner set and add the `terraform` Since the workflows are all labeled with `terraform` already, they will automatically select the new runner set on their next run. -# FAQ +## FAQ -## I cannot assume the role from GitHub Actions after deploying +### I cannot assume the role from GitHub Actions after deploying The following error is very common if the GitHub workflow is missing proper permission. From 26a8226ef99ba062fdd256b45065089999e272be Mon Sep 17 00:00:00 2001 From: milldr Date: Thu, 29 Aug 2024 14:54:12 -0400 Subject: [PATCH 4/4] upload workflow names --- docs/layers/github-actions/eks-github-actions-controller.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/layers/github-actions/eks-github-actions-controller.mdx b/docs/layers/github-actions/eks-github-actions-controller.mdx index 70db5a200..c366bbc88 100644 --- a/docs/layers/github-actions/eks-github-actions-controller.mdx +++ b/docs/layers/github-actions/eks-github-actions-controller.mdx @@ -123,7 +123,7 @@ In order to deploy Self-Hosted GitHub Runners on EKS, follow the steps outlined The first three steps before are all included in the following workflow: - +