From 94051893a429475f3e5e8817f3d7beb988d20511 Mon Sep 17 00:00:00 2001 From: Niek Palm Date: Wed, 6 May 2020 20:50:25 +0200 Subject: [PATCH 1/5] Describe archtecture basics --- README.md | 27 +++++++++++++++++++++++++++ docs/architecture.drawio | 2 +- docs/component-overview.svg | 3 +++ 3 files changed, 31 insertions(+), 1 deletion(-) create mode 100644 docs/component-overview.svg diff --git a/README.md b/README.md index 86e90ec85b..9aadc030f6 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,33 @@ > WIP: Module is in development +This modules create the required infra structure needed to host GitHub Action self hosted runners on spot instances in AWS. All logic required to handle the life cycle fo an action runners are implemented in AWS Lambda functions. + +## Motivation +GitHub action runners `self hosted` runners provide you with a flexible option to run you build load on compute of your choice. Currently there is no option provide to automate the creation, and scaling of the runners. This module provide besides the logic of the AWS infrastructure for scaling action runners up and down. + +We choose for to run manage the life cycle in several lambda. This gives us the option to grant the most minimal permissions to each step in the process of handle an event, scale up, or scale down. On the other hand the choice for Lambda keeps the costs low at the moment nothing is happening. + + +## Overview + +The process of scaling runners on demand starts by registering a GitHub App which delivers via a webhook protected by a secret a check run event at the API Gateway. The Gateway will trigger a Lambda which will verify the messages and queue a message on a SWS queue. Messages on the queue are read with a delay from 30 seconds. In case the build is not started after this delay, and no limits are reach a new spot instances is created via a launch template. The lambda will store a registration token in SSM Parameter store from where the user data script of the EC2 instance will read the token and register the runner. Stopping the instances is at the moment brute forced, every x minutes a Lambda is checking if a runner (instance) is not busy. In case the runner is not busy it will be removed from GitHub and AWS. Finally a cache is implemented to avoid the runner distribution needs to download every time. The cache is managed by a lambda that checks base on a cron expression if the distribution require an update. + +![Architecture](docs/component-overview.svg) + +Permission are managed on several places. Below the most important ones. For details check the Terraform sources. +- The GitHub App requires access to actions and publish event to AWS. +- The scale up lambda should have access to EC2 to create instances and tag instances. +- The scale down lambda should have access to EC2 to terminate instances. + +Besides the permissions are required to S3, CloudWatch, SSM, and S3. + + +## Usages + +## Examples + + diff --git a/docs/architecture.drawio b/docs/architecture.drawio index 010994a4fa..16c9bbde1c 100644 --- a/docs/architecture.drawio +++ b/docs/architecture.drawio @@ -1 +1 @@ -7Vzbcts2EP0azbQP1ZAAb3qU5EvSOm1sJ+OmLx5IhCnGEMGCUCT76wtQoEQClE1HtCXVSuwJsQRx2d1zsFiQ6cDhdHHOUDr5RENMOsAKFx140gHAtnpA/CMlD0tJL+gtBRGLQ1VpLbiOH3HxpJLO4hBnlYqcUsLjtCoc0yTBY16RIcbovFrtjpJqrymKsCG4HiNiSm/ikE+W0gD4a/kHHEeTomfbU/OboqKymkk2QSGdl0TwtAOHjFK+vJouhphI5RV6WT53tuHuamAMJ7zJAxahX8nCu/zny+P4CnwLvqHL778BZZ4fiMzUjPs310IwJHQWqoHzh0IbKY0TnmvUHYgf0eHQ6rjizlCWusDVBHrZrwpssyTbqAr0sl8V2Hrztta/rQ+wJDBKleYtrX+rNEDxAwd0xkmc4OHK9ywhjBgKY2GTISWUCVlCE6G9wYRPiSjZ4nI+iTm+TtFYanUucCNkdzThyvttUJSV4mWrwntSeT1dRBJoXTTPnG7E6CzNu/wo/L/27q24vB3nxhSNcEbvcTGwDoDi75n0lsFdTIg24B+Y8VgAoU/iSLbNqewKqRLBd1y2KGYRJ9FFXjqBlhp5XRchyiY4VNMxnVf5s+wVL0oi5cznmE4xZw+iSnHXU8BSzFIU52uYeo67lE1KEO35qiJS1BCtml6jR1woAL0ATJ6BJQNAOBTsooqU8QmNaILI6Vo6EEZLwpWe1nUuqNR/7j/fMecPylnQjNOqd21UbUZnbIyfGH/Br4hFmD9Rz1nWk3N50lAME8TjH1UmbV3rtqH1/uePQnCOOJ6jB8MEtaDd5LQ6mMW9nuOenIHSvZOYiYbiHICJNJiGJ/GM24fWwK1D4F3+R4dHgb0LNMLkM81i1fyIck6nz4JzLEaFWdUtniMYlKVLddzFCxxuYhyGl1605JuBKNYxD0rj20ipvxWsw6CKdRvaXdeAux+YaC9krbuduXDe4NGE0vufwHzJTDgJ+zJskcYmdHwvRWSUlwub526EGC/qKb4WT57FpGjHcDTXcQeeYxDMz7MFbMgWdkO2KBnStpS5GzOIau6zjE/WbgN8zW2K4KxoYjlJ9VQ5dtIb8vSGgsL/iqaWejCayn1rNctG7vbno3fh8WDu3d7/yX+/v/lj7KSFtkvu9kH0LCyH2TTOMkEPWR7essNacmqcqHb6TZ1oS4fp+YVZlaFB4L2eoZ/SSMnQ5zGfzEZC1k/TDvCIjLtGws5exFe6L9m7spDULnYlyxkrBtFWnNWNn1yS9FVkjgWtRTTrRmpWW68NOjTrl4aaQNB3N7vHVkuDf+BxoNOQ2d29igOdxgvythHgmR+cWs7LIsATyx3a/ruJAAmajkLUTvDnwr0L/mzbcKq9RrTbNFaz9grS7iZIX86wkADr+lLmqsxF8ZcTMb4HHH7CWYYi/Gv7FAAdJ3jhJnAwtKHrvRsKyP7N2sG/p0fxu8c/MJf4IcNix5sdFi8UeH+WGIC3V8RQjLuk/zxnfzUTeGZZDSEIpI9nTGhoLEdpvwIjHIOCtwwKdFIAPTP9+7YhQWC45KmYjhzy3+J3Giezw6MHr2nc4O8VPUBzL1DYIpQpybP/qRkg3Csz2A0ORI6B2KvTbn76N0d8PGmHeu1gz6h3NYVDTbgU9Pk8z/baBnh9HhR6QcXETqAlzl85CWo3SKFtxRzqzKQZMajKe8cJ+dsYmImlbflShr2JJ7CsccvjKb4doSx/tgUe0A/lds8DZgj2RR6PJAe4Mes1XfKD/Vrye4YJhgSjZJautmbHfdch77v0ZOzuQW8emG1ytZqXnMxj8r43DLyyA9gb7avbTPPaVVMtqB1YVbX7NWp3g5pDrtWD7W+ygKF40BXlKxzFGc/1b7HcEodFvaDm+GuXZ9K2Znqov7/2yrEYMDfT/ZxfpYGvlIEt80DkSOyvv7Oj03TGcUsUA6uvPgTOrpndTB/UOB48Ot6BO56eyt2540Ezn2B3T+g8IRSF0vniJOOIkINb2ZpuKsB+veELzE2FTgTyaiA2eiyu2eltSwie1YfQfxkhAN+37feTY8zg6yxCHtw1F3iHhXHY9LVcuF9vb0FzG9cujo8Zv/Yz/w7YNTr9A0Nn0xexQOuJ/u30bL6J9TUNEcf5mtv+NzbHEPxNk3oQVHHtWt6OcW1m8mu2fi2vEEe/e/Otn55Nfsut31M5vbqvL/5iEUriR5SbS3e8d/7RBbRq7NbOJxeiuP5GfpnTXP9PA/D0Pw== \ No newline at end of file +7VtbU9s4FP41mdl9gLEt3/KYC7Dd0i2UdpnuS0aJFcfFsVJZaYBfv7ItO9bFIcRJaAvADNaxLMk63/nOOZLcAYP5/QWBi9kHHKC4YxnBfQcMO5Zlds0u+5dJHgqJb7uFICRRwCutBTfRI+JCg0uXUYBSoSLFOKbRQhROcJKgCRVkkBC8EqtNcSz2uoAhUgQ3Exir0tsooDP+Fpa3lv+FonBW9my6/IXnsKzM3ySdwQCvaiJw1gEDgjEtrub3AxRnk1fOS/HcecPdamAEJXSbB4wYf4nv3ev/Pj9OPllf/a/w+ttJ9kDWzA8YL/kb925vmGAQ42XAB04fytlY4Cih+Yw6ffbHOhwYHYfdGWSlU8uRBHLZEwWmWsraEAVy2RMFpty8KfVvygOsCZSS0Lwh9W/UBsj+QB8vaRwlaFBhz2DCkMAgYjoZ4BgTJktwwmavP6PzmJVMdrmaRRTdLOAkm9UVsxsmm+KEcvSbVlnmE5+1ytCzyK7n92FmaKdwldqnIcHLRd7lO4Z/7d0RuxxNcmWyRijBd6gcWMcC7Pc8Q0t/GsWxNOAfiNCIGUIvjsKsbYqzriAvxWhKsxbZW0RJeJmXhsDgI9d1EcB0hgL+Oip4OZ6zXtF9TcTBfIHwHFHywKqUd11uWJxZbF5crc3UtZ1CNquZqF9SC+TUEFZNr62HXXAD0hvTx74/hDMyHSXQHL2/6t2l7/496Sq2NMSrJMaQGZIxjhLI+6hbFAoY3fAiJnSGQ5zA+Gwt7TMtJkE1ces6lzhTSA6ob4jSB44euKRYhFvRZ9ZRI3NwUYqXZII20QXgFAxJiOimil297giKIY1+iCPRKYI/epURzlrnruUIOne7rthEMTD+1FqdPULgQ60a57HmfrpdsR8gcevz6rOLYgRrbFVzshXcNiqjhrdPS8ZEJFVQpjFvhQucnjvw3bqhmo0sILOTZPNVU3swc8cUzRxo7Nzi0y3YueVJ2NjFzrUT7yrzfvXx5nNmYz8Qx92RTTyGYxRf4TSiUe4HJmwciNTUdylVEMlcqV5qe4wpxfNNinySMsqg7SnGsLU01YJBWqnYVFTcu3rHBBeQohVUaVwbCTR5QjlCYPe6tjM8t2r3hhFhDRXKSjJESE46tzJg9B2dKU/zH9nnNmGhUvJmj1+BpIa7p6IWmC6K6ZhG9yhoCmMIKlBUBDF9VtSFM3ARjUI+/XthFmAAgVlMS2UWz1eJpZTtHXRqLH6LxjOM73aglJqSUBL0skwoU3WMJ3eZKB7n5VLjOYggoWU97iPYk+dRXLajegzb6bu2wl+7c8W20YW5JVfUFGkaXLktIxDLE91Rle+VTRQvqUQgakOu3JB/6mwVzOwrflDDh4uIzpZjJustFh3LjbMIf0zYVUgrvdZQKLCLlgFrKCxphBSGpLgsnlA0kdQ8CoJYl5ZUN2RqWSGG9hCnpyF/q9aE4cp8AVS+0CQcntMMslb5huWrASD6vkRpBjOyTH6miOQJ9YqupsTIE+HIXlIae0vOcVqGIzvlIXLwa/ldCTUHyCvsrd1Q26jn3PPPDPt5Uc/QcAam92qinhjOxwHcUyplSC7niAGPnsDUFZOX4Ko9somzbQRj6DV1WDqxLSmXNjYvayj1u0dY1nCa6Od6iZjEMm6uswVqNT75Y8gm7wEFH1CawhD9uX+6ArbtPzNJ6w9M4Livhq7S7+lhkrNqieeluMrUBVthlGbTnUdbSXbx8oHWgZZ+9sKOZTRTp8d/Ht1Ll/ord3T3D/377vb9xF6ctF370dOZaUs7BnITe1o9llM72wQbafaJ+iLNrp8uh4On0xQdJEMs9VXDfLE1WgKeTYKxXLzFhb92XAikuBD41tG4Vr8QpnLtZ0TmUQIpUjc3fh+S3YlTu1qCblT2iXFqAJ9PcEtCdQxPAI7jeVsR6oE4TO+31WWuISJvnns3lO3qudfgczzTage+I/g9NS8t/V61w2pke/tvnu/38nwO8F/Y86nI6+XqTwXs9bPDJJHGF7bFn2v0APCehz/L80zzFaW4YD/Yc6Wo3+2+cIZbZti/7OIb2Hr/sO3ppHbzbL6WeQZtN01acSlQI7/98iXfJd9uxY9X/umYMF/MQOQs2yhMeSNadsy3Ekc0mqPRGKb5swfwwMdkQT1qvJezzp2McdstB2vbRPE4LGiqR6y+LIqzsprpzy2koz1Q0JgHVScDmCOPHuG4sgHxFPuwmn4FtdVnBfzhTnWYXzh23wyjTYkQcIErQL9lWlQZlNDoiZSR7yVpas789MdJPpIQJkwFOb3J6n3lp0hAeTzkCKdINhKIYIgBpKivP7H+luL+Uimu7fxkaYa1Y1R2aAVr9AVO0wWmoyhJKUwmmljv3PEdYGtwzqO9Rr+iqLT50IbrC/qzNefffc3x91K2d/2px4MGBP3mC/MH2/3c8GFHIyJY6NAF4vcmVe7eNnjwJLDJTWx7utUzxAH6tgTHPW212p5+S7fxRMvm+k1brffSNMvTfvhtDNVJv7HmpiBLSittV42yjsqa6jdDb/rbGCRL+tNELXvSHyuuP8IuzHX9KTs4+x8= \ No newline at end of file diff --git a/docs/component-overview.svg b/docs/component-overview.svg new file mode 100644 index 0000000000..9d3465c053 --- /dev/null +++ b/docs/component-overview.svg @@ -0,0 +1,3 @@ + + +
AWS Cloud
AWS Cloud
Download binary
Download binaryRunners
POST event
POST event
API Gateway
API Gateway
Webhook
Webhook
Github App
Github App
Request run event
Request run event
Webhook
Webhook
WebhookQueue SQS
(DelayedMessage)
WebhookQueue...
Register runner
Register runner
Scale runners up
Scale runners...
Terminates
Terminates
Deregister runner
Deregister runner
Scale Runners Down
Scale Runners...
Actions Runners Binaries
Actions Runne...
Upload
Upload
Github Organization
Github Organi...
UpdateBinary
UpdateBinary
Creates
CreatesViewer does not support full SVG 1.1 \ No newline at end of file From ba8672d233097c79977522bed850f62e6e7f7c8f Mon Sep 17 00:00:00 2001 From: Niek Palm Date: Thu, 7 May 2020 17:50:29 +0200 Subject: [PATCH 2/5] Update docs --- README.md | 62 +++++++++++++++++++++++++++++++------ docs/component-overview.svg | 2 +- 2 files changed, 53 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 9aadc030f6..9715b1b697 100644 --- a/README.md +++ b/README.md @@ -2,35 +2,77 @@ > WIP: Module is in development -This modules create the required infra structure needed to host GitHub Action self hosted runners on spot instances in AWS. All logic required to handle the life cycle fo an action runners are implemented in AWS Lambda functions. +This modules create the required infra structure needed to host GitHub Action self hosted runners on AWS spot instances. All logic required to handle the life cycle for an action runners are implemented in AWS Lambda functions. ## Motivation -GitHub action runners `self hosted` runners provide you with a flexible option to run you build load on compute of your choice. Currently there is no option provide to automate the creation, and scaling of the runners. This module provide besides the logic of the AWS infrastructure for scaling action runners up and down. -We choose for to run manage the life cycle in several lambda. This gives us the option to grant the most minimal permissions to each step in the process of handle an event, scale up, or scale down. On the other hand the choice for Lambda keeps the costs low at the moment nothing is happening. +GitHub action runners `self hosted` runners provide you with a flexible option to run you build load on compute of your choice. Currently there is no option provided to automate the creation, and scaling of the runners. Besides required AWS infra structure definition the module provides the logic to handle GitHub `check_events` for queued builds and scale up and down runners on Org and Repo level. +Lambda is chooses as runtime for two major reasons. First is allows us to create small components with minimal access to AWS cloud and GitHub. Secondly is provides a scalable setup for minimal costs that could work on repo level and could scale to Org level. + +A logical question would why not Kubernetes? In the current approach we stay very close to the the way the GitHub action runners are available today. A agent will run on its own dedicated instances during a workflow run. Another logical choice could be AWS Auto Scaling groups. This choice would require typically much more permission on instance level to GitHub. And besides that scaling up and down is not that trivial which requires custom logic as well. ## Overview -The process of scaling runners on demand starts by registering a GitHub App which delivers via a webhook protected by a secret a check run event at the API Gateway. The Gateway will trigger a Lambda which will verify the messages and queue a message on a SWS queue. Messages on the queue are read with a delay from 30 seconds. In case the build is not started after this delay, and no limits are reach a new spot instances is created via a launch template. The lambda will store a registration token in SSM Parameter store from where the user data script of the EC2 instance will read the token and register the runner. Stopping the instances is at the moment brute forced, every x minutes a Lambda is checking if a runner (instance) is not busy. In case the runner is not busy it will be removed from GitHub and AWS. Finally a cache is implemented to avoid the runner distribution needs to download every time. The cache is managed by a lambda that checks base on a cron expression if the distribution require an update. +The process of scaling runners on demand starts by registering a GitHub App which delivers via a webhook protected a check run event at the API Gateway. The Gateway will trigger a Lambda which will verify the messages signature and filter for queued build events. Accepted events are posted on a queue. Messages on this queue will be delayed for x seconds to give available runners the time to take up the build event. + +In case the build is not started and yet, and no limits are reach the lambda will create a new spot instances is via a launch template. The lambda will request a registration token on GitHub and store this in SSM. The EC2 instances will install required software via a `user_data` script, fetch and delete the registration token from SSM. + +Stopping the instances is at the moment brute forced, every x minutes a Lambda is checking if a runner (instance) is not busy. In case the runner is not busy it will be removed from GitHub and AWS. + +Downloading the GitHub action distribution can be occasionally slow, up to 10 more minutes. Therefore a lambda is introduced that synchronize based on a schedule the binary from GitHub to a cache in S3. The EC2 will fetch the distribution form the cache instead of internet. ![Architecture](docs/component-overview.svg) Permission are managed on several places. Below the most important ones. For details check the Terraform sources. -- The GitHub App requires access to actions and publish event to AWS. -- The scale up lambda should have access to EC2 to create instances and tag instances. -- The scale down lambda should have access to EC2 to terminate instances. -Besides the permissions are required to S3, CloudWatch, SSM, and S3. +- The GitHub App requires access to actions and publish check events to AWS. +- The scale up lambda should have access to EC2 for creating and tagging instances. +- The scale down lambda should have access to EC2 to terminate instances. +Besides the permissions are required to S3, CloudWatch, SSM, and S3. ## Usages -## Examples +Examples are provided in [the example directory](examples/). Please ensure you have installed the following tools. + +- Terraform, or [tfenv](https://github.com/tfutils/tfenv). +- Bash shell or compatible. +- TODO: building lambda ? +- AWS cli +Before you start the setup you have to choose if you are planning to run the runners on repo level on or Org level. Running on repo level a runner will be dedicated to one repo only, no other repo can use the runner. On Org level you can use the runner(s) for all the org repo's. +The setup consists of running Terraform to create all AWS resources and configure the GitHub app. The problem here is that our Terraform module requires configuration parameters from the GitHub app and our GitHub app requires output parameters of Terraform. So first create the app, configure the basics, run terraform finalize the app configuration. +### Setup GitHub App (part 1) +The module requires a GitHub app. Go to GitHub and create a new app. Be-aware you can create apps your Org or for a user. For now we handle only the Org level app. + +1. Create app in Github +2. Choose a name +3. Choose a website (mandatory, not required for the module). +4. Disable the webhook (for now). +5. Repository permissions, enable `Checks` to receive vents for now builds. +6. _Only for repo level runner!_ - Repository permissions, `Administration` - Read and Write (to register runner) +7. _Only for Org level runner!_ - Organization permissions, `Administration` - Read and Write (to register runner) +8. Save the new app. +9. Next generate a private key on the General page. +10. Make a note of the following app parameters: App Id and Client ID + +### Setup terraform module + +TODO + +### Setup GitHub App (part 12 + +Go back to the GitHub App and update the following settings. + +1. Enable the webhook. +2. Provide the webhook secret. +3. Enable the `Check` event for the webhook. + +## Examples ## Philips Forest @@ -41,7 +83,7 @@ This module is part of the Philips Forest. / __\__ _ __ ___ ___| |_ / _\/ _ \| '__/ _ \/ __| __| / / | (_) | | | __/\__ \ |_ - \/ \___/|_| \___||___/\__| + \/ \___/|_| \___||___/\__| Infrastructure ``` diff --git a/docs/component-overview.svg b/docs/component-overview.svg index 9d3465c053..433ac07387 100644 --- a/docs/component-overview.svg +++ b/docs/component-overview.svg @@ -1,3 +1,3 @@ -
AWS Cloud
AWS Cloud
Download binary
Download binaryRunners
POST event
POST event
API Gateway
API Gateway
Webhook
Webhook
Github App
Github App
Request run event
Request run event
Webhook
Webhook
WebhookQueue SQS
(DelayedMessage)
WebhookQueue...
Register runner
Register runner
Scale runners up
Scale runners...
Terminates
Terminates
Deregister runner
Deregister runner
Scale Runners Down
Scale Runners...
Actions Runners Binaries
Actions Runne...
Upload
Upload
Github Organization
Github Organi...
UpdateBinary
UpdateBinary
Creates
CreatesViewer does not support full SVG 1.1 \ No newline at end of file +
AWS Cloud
AWS Cloud
Download binary
Download binary
Register runner and retrieve build
Register runner and retrieve buildRunners on AWS Spot Instances
POST event
POST event
API Gateway
API Gateway
Webhook
Webhook
Github App
Github App
Webhook
Webhook
Quedued builds
Quedued builds
Create runner token
Create runner token
Scale runners up
Scale runners...
Terminates
Terminates
Remove runner
Remove runner
Scale Runners Down
Scale Runners...
Actions Runners Binaries
Actions Runne...
Upload
Upload
Github Organization
Github Organi...
UpdateBinary
UpdateBinary
Creates
CreatesViewer does not support full SVG 1.1 \ No newline at end of file From ba59ef62a8add1cf615b50347c0ce1fde1a66012 Mon Sep 17 00:00:00 2001 From: Niek Palm Date: Thu, 7 May 2020 18:24:24 +0200 Subject: [PATCH 3/5] Update docs --- README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 9715b1b697..3a5203fe3f 100644 --- a/README.md +++ b/README.md @@ -6,19 +6,19 @@ This modules create the required infra structure needed to host GitHub Action se ## Motivation -GitHub action runners `self hosted` runners provide you with a flexible option to run you build load on compute of your choice. Currently there is no option provided to automate the creation, and scaling of the runners. Besides required AWS infra structure definition the module provides the logic to handle GitHub `check_events` for queued builds and scale up and down runners on Org and Repo level. +GitHub action runners `self hosted` runners provides you with a flexible option to run you CI workloads on compute of your choice. Currently there is no option provided to automate the creation, and scaling of action runners. This module takes care of creating the AWS infra structure and provides lambda modules ot orchestrate the the life cycle of the action runners. -Lambda is chooses as runtime for two major reasons. First is allows us to create small components with minimal access to AWS cloud and GitHub. Secondly is provides a scalable setup for minimal costs that could work on repo level and could scale to Org level. +Lambda is chooses as runtime for two major reasons. First is allows to create small components with minimal access to AWS and GitHub. Secondly is provides a scalable setup for minimal costs that works on repo level and scales to Org level setup. The lambda's will create linux based EC2 instances with Docker to serve CI workloads that can run on linux and or Docker. The main goal is here to support Docker based wark loads. -A logical question would why not Kubernetes? In the current approach we stay very close to the the way the GitHub action runners are available today. A agent will run on its own dedicated instances during a workflow run. Another logical choice could be AWS Auto Scaling groups. This choice would require typically much more permission on instance level to GitHub. And besides that scaling up and down is not that trivial which requires custom logic as well. +A logical question would why not Kubernetes? In the current approach we stay close to the way the GitHub action runners are available today. The approach is to install the runner on a host where the required software is available. With this setup we stay quite close to th GitHub approach. Another logical choice could be AWS Auto Scaling groups. This choice would require typically much more permission on instance level to GitHub. And besides that, scaling up and down is not trivial. ## Overview -The process of scaling runners on demand starts by registering a GitHub App which delivers via a webhook protected a check run event at the API Gateway. The Gateway will trigger a Lambda which will verify the messages signature and filter for queued build events. Accepted events are posted on a queue. Messages on this queue will be delayed for x seconds to give available runners the time to take up the build event. +The process of scaling runners on demand starts by registering a GitHub App which delivers via a webhook a check run event to the API Gateway. The Gateway triggers a Lambda which will verify the signature and filter for queued build events. Accepted events are posted on a queue. Messages on this queue will be delayed for x seconds to give available runners the time to start a build. -In case the build is not started and yet, and no limits are reach the lambda will create a new spot instances is via a launch template. The lambda will request a registration token on GitHub and store this in SSM. The EC2 instances will install required software via a `user_data` script, fetch and delete the registration token from SSM. +In case the build is not started yet, and no limits are reach. The lambda requests a registration token for a new runner at GitHub, stores the token in SSM and starts an EC2 instance via a launch template. The EC2 instances installs the required software via a `user_data` script, fetch and delete the registration token from SSM. -Stopping the instances is at the moment brute forced, every x minutes a Lambda is checking if a runner (instance) is not busy. In case the runner is not busy it will be removed from GitHub and AWS. +Scaling down the runners is at the moment brute forced, every x minutes a Lambda is checking if a runner (instance) is not busy. In case the runner is not busy it will be removed from GitHub and AWS. At the moment there seems no other option to scale down more smooth. Downloading the GitHub action distribution can be occasionally slow, up to 10 more minutes. Therefore a lambda is introduced that synchronize based on a schedule the binary from GitHub to a cache in S3. The EC2 will fetch the distribution form the cache instead of internet. @@ -41,7 +41,7 @@ Examples are provided in [the example directory](examples/). Please ensure you h - TODO: building lambda ? - AWS cli -Before you start the setup you have to choose if you are planning to run the runners on repo level on or Org level. Running on repo level a runner will be dedicated to one repo only, no other repo can use the runner. On Org level you can use the runner(s) for all the org repo's. +The module support twe main scenario's for running tha action runners. Running on repo level a runner will be dedicated to one repo only, no other repo can use the runner. On Org level you can use the runner(s) for all the org repo's. Before starting the deployment you have to choose one option. The setup consists of running Terraform to create all AWS resources and configure the GitHub app. The problem here is that our Terraform module requires configuration parameters from the GitHub app and our GitHub app requires output parameters of Terraform. So first create the app, configure the basics, run terraform finalize the app configuration. From affc4266919852d3db5f4b96a79703594f0a1a83 Mon Sep 17 00:00:00 2001 From: Niek Palm Date: Mon, 11 May 2020 10:59:02 +0200 Subject: [PATCH 4/5] Update docs --- README.md | 48 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 40 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 3a5203fe3f..48f846db2f 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,11 @@ > WIP: Module is in development -This modules create the required infra structure needed to host GitHub Action self hosted runners on AWS spot instances. All logic required to handle the life cycle for an action runners are implemented in AWS Lambda functions. +This [Terraform](https://www.terraform.io/) modules create the required infra structure needed to host [GitHub Action](https://github.com/features/actions) self hosted runners on [AWS spot instances](https://aws.amazon.com/ec2/spot/). All logic required to handle the life cycle for an action runners are implemented in AWS Lambda functions. ## Motivation -GitHub action runners `self hosted` runners provides you with a flexible option to run you CI workloads on compute of your choice. Currently there is no option provided to automate the creation, and scaling of action runners. This module takes care of creating the AWS infra structure and provides lambda modules ot orchestrate the the life cycle of the action runners. +GitHub action runners `self hosted` runners provides you with a flexible option to run you CI workloads on compute of your choice. Currently there is no option provided to automate the creation, and scaling of action runners. This module takes care of creating the AWS infra structure to host action runners on spot instances. And provides lambda modules to orchestrate the life cycle of the action runners. Lambda is chooses as runtime for two major reasons. First is allows to create small components with minimal access to AWS and GitHub. Secondly is provides a scalable setup for minimal costs that works on repo level and scales to Org level setup. The lambda's will create linux based EC2 instances with Docker to serve CI workloads that can run on linux and or Docker. The main goal is here to support Docker based wark loads. @@ -14,9 +14,9 @@ A logical question would why not Kubernetes? In the current approach we stay clo ## Overview -The process of scaling runners on demand starts by registering a GitHub App which delivers via a webhook a check run event to the API Gateway. The Gateway triggers a Lambda which will verify the signature and filter for queued build events. Accepted events are posted on a queue. Messages on this queue will be delayed for x seconds to give available runners the time to start a build. +The process of scaling runners on demand starts by registering a GitHub App which delivers via a webhook a [check run event](https://developer.github.com/v3/activity/events/types/#checkrunevent) to the API Gateway. The Gateway triggers a Lambda which will verify the signature and filter for queued build events. Accepted events are posted on a queue. Messages on this queue will be delayed for x seconds to give available runners the time to start a build. -In case the build is not started yet, and no limits are reach. The lambda requests a registration token for a new runner at GitHub, stores the token in SSM and starts an EC2 instance via a launch template. The EC2 instances installs the required software via a `user_data` script, fetch and delete the registration token from SSM. +In case the build is not started yet, and no limits are reach. The lambda requests a registration token for a new runner at GitHub, stores the token in SSM parameter store and starts an EC2 instance via a launch template. The EC2 instances installs the required software via a [`user_data`](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html) script, fetch and delete the registration token from SSM. Scaling down the runners is at the moment brute forced, every x minutes a Lambda is checking if a runner (instance) is not busy. In case the runner is not busy it will be removed from GitHub and AWS. At the moment there seems no other option to scale down more smooth. @@ -58,19 +58,51 @@ The module requires a GitHub app. Go to GitHub and create a new app. Be-aware yo 7. _Only for Org level runner!_ - Organization permissions, `Administration` - Read and Write (to register runner) 8. Save the new app. 9. Next generate a private key on the General page. -10. Make a note of the following app parameters: App Id and Client ID +10. Make a note of the following app parameters: app id , client ID, and client secret ### Setup terraform module -TODO +Create a terraform workspace and initiate the module, see the examples for more details. + +```terraform +module "runners" { + source = "git::https://github.com/philips-labs/terraform-aws-github-runner/" + + aws_region = "eu-west-1" + vpc_id = "vpc-123" + subnet_ids = ["subnet-123", "subnet-456"] + + environment = "gh-ci" + + github_app = { + key_base64 = "base64string" + id = "1" + client_id = "c-123" + client_secret = "secret" + webhook_secret = "secret" + } + + enable_organization_runners = true +} +``` + +Next run + +```bash +terraform init +terrafrom apply +``` + +Check the terraform output for the API gateway url, which you need in the next step. ### Setup GitHub App (part 12 Go back to the GitHub App and update the following settings. 1. Enable the webhook. -2. Provide the webhook secret. -3. Enable the `Check` event for the webhook. +2. Provide the webhook url, see output previous step. +3. Provide the webhook secret. +4. Enable the `Check` event for the webhook. ## Examples From 799ed744323d339d0fd95f1332e53e9baec3e443 Mon Sep 17 00:00:00 2001 From: Gertjan Maas Date: Tue, 12 May 2020 14:21:43 +0200 Subject: [PATCH 5/5] Update doc --- README.md | 46 ++++++++++++++++++------------------- docs/architecture.drawio | 2 +- docs/component-overview.svg | 2 +- 3 files changed, 25 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 48f846db2f..eefdbcbc45 100644 --- a/README.md +++ b/README.md @@ -2,35 +2,35 @@ > WIP: Module is in development -This [Terraform](https://www.terraform.io/) modules create the required infra structure needed to host [GitHub Action](https://github.com/features/actions) self hosted runners on [AWS spot instances](https://aws.amazon.com/ec2/spot/). All logic required to handle the life cycle for an action runners are implemented in AWS Lambda functions. +This [Terraform](https://www.terraform.io/) modules create the required infra structure needed to host [GitHub Action](https://github.com/features/actions) self hosted runners on [AWS spot instances](https://aws.amazon.com/ec2/spot/). All logic required to handle the lifecycle for an action runners is implemented in AWS Lambda functions. ## Motivation -GitHub action runners `self hosted` runners provides you with a flexible option to run you CI workloads on compute of your choice. Currently there is no option provided to automate the creation, and scaling of action runners. This module takes care of creating the AWS infra structure to host action runners on spot instances. And provides lambda modules to orchestrate the life cycle of the action runners. +GitHub Actions `self hosted` runners provides you with a flexible option to run your CI workloads on compute of your choice. Currently there is no option provided to automate the creation and scaling of action runners. This module takes care of creating the AWS infra structure to host action runners on spot instances. And provides lambda modules to orchestrate the lifecycle of the action runners. -Lambda is chooses as runtime for two major reasons. First is allows to create small components with minimal access to AWS and GitHub. Secondly is provides a scalable setup for minimal costs that works on repo level and scales to Org level setup. The lambda's will create linux based EC2 instances with Docker to serve CI workloads that can run on linux and or Docker. The main goal is here to support Docker based wark loads. +Lambda is chosen as runtime for two major reasons. First it allows to create small components with minimal access to AWS and GitHub. Secondly it provides a scalable setup for minimal costs that works on repo level and scales to organization level. The lambdas will create Linux based EC2 instances with Docker to serve CI workloads that can run on Linux and/or Docker. The main goal is here to support Docker based workloads. -A logical question would why not Kubernetes? In the current approach we stay close to the way the GitHub action runners are available today. The approach is to install the runner on a host where the required software is available. With this setup we stay quite close to th GitHub approach. Another logical choice could be AWS Auto Scaling groups. This choice would require typically much more permission on instance level to GitHub. And besides that, scaling up and down is not trivial. +A logical question would be why not Kubernetes? In the current approach we stay close to the way the GitHub action runners are available today. The approach is to install the runner on a host where the required software is available. With this setup we stay quite close to the current GitHub approach. Another logical choice would be AWS Auto Scaling groups. This choice would typically require much more permissions on instance level to GitHub. And besides that, scaling up and down is not trivial. ## Overview -The process of scaling runners on demand starts by registering a GitHub App which delivers via a webhook a [check run event](https://developer.github.com/v3/activity/events/types/#checkrunevent) to the API Gateway. The Gateway triggers a Lambda which will verify the signature and filter for queued build events. Accepted events are posted on a queue. Messages on this queue will be delayed for x seconds to give available runners the time to start a build. +The process of scaling runners on demand starts by registering a GitHub App which sends a [check run event](https://developer.github.com/v3/activity/events/types/#checkrunevent) via a webhook to the API Gateway. The Gateway triggers a lambda which will verify the signature and filter for queued build events. Accepted events are posted on a SQS queue. Messages on this queue will be delayed for a configurable amount of seconds to give the available runners time to pick up this build. -In case the build is not started yet, and no limits are reach. The lambda requests a registration token for a new runner at GitHub, stores the token in SSM parameter store and starts an EC2 instance via a launch template. The EC2 instances installs the required software via a [`user_data`](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html) script, fetch and delete the registration token from SSM. +In case the build is not picked up yet and no limits are reached the lambda requests a registration token for a new runner at GitHub, stores the token in the SSM parameter store and starts an EC2 instance via a launch template. The EC2 instance installs the required software via a [`user_data`](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html) script, fetches and deletes the registration token from SSM and configures the action runner. -Scaling down the runners is at the moment brute forced, every x minutes a Lambda is checking if a runner (instance) is not busy. In case the runner is not busy it will be removed from GitHub and AWS. At the moment there seems no other option to scale down more smooth. +Scaling down the runners is at the moment brute-forced, every configurable amount of minutes a lambda will check every runner (instance) if it is busy. In case the runner is not busy it will be removed from GitHub and the instance terminated in AWS. At the moment there seems no other option to scale down more smoothly. -Downloading the GitHub action distribution can be occasionally slow, up to 10 more minutes. Therefore a lambda is introduced that synchronize based on a schedule the binary from GitHub to a cache in S3. The EC2 will fetch the distribution form the cache instead of internet. +Downloading the GitHub Action Runner distribution can be occasionally slow (more than 10 minutes). Therefore a lambda is introduced that synchronizes the action runner binary from GitHub to an S3 bucket. The EC2 instance will fetch the distribution from the S3 bucket instead of the internet. ![Architecture](docs/component-overview.svg) Permission are managed on several places. Below the most important ones. For details check the Terraform sources. -- The GitHub App requires access to actions and publish check events to AWS. +- The GitHub App requires access to actions and publish `check_run` events to AWS. - The scale up lambda should have access to EC2 for creating and tagging instances. - The scale down lambda should have access to EC2 to terminate instances. -Besides the permissions are required to S3, CloudWatch, SSM, and S3. +Besides these permissions, the lambdas also need permission to CloudWatch (for logging and scheduling), SSM and S3. ## Usages @@ -41,28 +41,28 @@ Examples are provided in [the example directory](examples/). Please ensure you h - TODO: building lambda ? - AWS cli -The module support twe main scenario's for running tha action runners. Running on repo level a runner will be dedicated to one repo only, no other repo can use the runner. On Org level you can use the runner(s) for all the org repo's. Before starting the deployment you have to choose one option. +The module support two main scenarios for creating runners. On repository level a runner will be dedicated to only one repository, no other repository can use the runner. On organization level you can use the runner(s) for all the repositories within the organization. See https://help.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners for more information. Before starting the deployment you have to choose one option. -The setup consists of running Terraform to create all AWS resources and configure the GitHub app. The problem here is that our Terraform module requires configuration parameters from the GitHub app and our GitHub app requires output parameters of Terraform. So first create the app, configure the basics, run terraform finalize the app configuration. +The setup consists of running Terraform to create all AWS resources and configure the GitHub App. The Terraform module requires configuration from the GitHub App and the GitHub app requires output from Terraform. Therefore you should first create the GitHub App, configure the basics. Then run Terraform and finalize the configuration of the GitHub App afterwards. ### Setup GitHub App (part 1) -The module requires a GitHub app. Go to GitHub and create a new app. Be-aware you can create apps your Org or for a user. For now we handle only the Org level app. +Go to GitHub and create a new app. Beware you can create apps your organization or for a user. For now we handle only the organization level app. 1. Create app in Github 2. Choose a name 3. Choose a website (mandatory, not required for the module). -4. Disable the webhook (for now). -5. Repository permissions, enable `Checks` to receive vents for now builds. -6. _Only for repo level runner!_ - Repository permissions, `Administration` - Read and Write (to register runner) -7. _Only for Org level runner!_ - Organization permissions, `Administration` - Read and Write (to register runner) +4. Disable the webhook for now (we will configure this later). +5. Repository permissions, enable `Checks` to receive events for new builds. +6. _Only for repo level runners!_ - Repository permissions, `Administration` - Read and Write (to register runner) +7. _Only for organization level runners!_ - Organization permissions, `Administration` - Read and Write (to register runner) 8. Save the new app. 9. Next generate a private key on the General page. 10. Make a note of the following app parameters: app id , client ID, and client secret ### Setup terraform module -Create a terraform workspace and initiate the module, see the examples for more details. +1. Create a terraform workspace and initiate the module, see the examples for more details. ```terraform module "runners" { @@ -86,23 +86,23 @@ module "runners" { } ``` -Next run +2. Run terraform by using the following commands ```bash terraform init terrafrom apply ``` -Check the terraform output for the API gateway url, which you need in the next step. +3. Check the terraform output for the API gateway url, which you need in the next step. -### Setup GitHub App (part 12 +### Setup GitHub App (part 2) Go back to the GitHub App and update the following settings. 1. Enable the webhook. -2. Provide the webhook url, see output previous step. +2. Provide the webhook url, should be part of the output of terraform. 3. Provide the webhook secret. -4. Enable the `Check` event for the webhook. +4. Enable the `Check run` event for the webhook. ## Examples diff --git a/docs/architecture.drawio b/docs/architecture.drawio index 16c9bbde1c..6e9f233475 100644 --- a/docs/architecture.drawio +++ b/docs/architecture.drawio @@ -1 +1 @@ -7VtbU9s4FP41mdl9gLEt3/KYC7Dd0i2UdpnuS0aJFcfFsVJZaYBfv7ItO9bFIcRJaAvADNaxLMk63/nOOZLcAYP5/QWBi9kHHKC4YxnBfQcMO5Zlds0u+5dJHgqJb7uFICRRwCutBTfRI+JCg0uXUYBSoSLFOKbRQhROcJKgCRVkkBC8EqtNcSz2uoAhUgQ3Exir0tsooDP+Fpa3lv+FonBW9my6/IXnsKzM3ySdwQCvaiJw1gEDgjEtrub3AxRnk1fOS/HcecPdamAEJXSbB4wYf4nv3ev/Pj9OPllf/a/w+ttJ9kDWzA8YL/kb925vmGAQ42XAB04fytlY4Cih+Yw6ffbHOhwYHYfdGWSlU8uRBHLZEwWmWsraEAVy2RMFpty8KfVvygOsCZSS0Lwh9W/UBsj+QB8vaRwlaFBhz2DCkMAgYjoZ4BgTJktwwmavP6PzmJVMdrmaRRTdLOAkm9UVsxsmm+KEcvSbVlnmE5+1ytCzyK7n92FmaKdwldqnIcHLRd7lO4Z/7d0RuxxNcmWyRijBd6gcWMcC7Pc8Q0t/GsWxNOAfiNCIGUIvjsKsbYqzriAvxWhKsxbZW0RJeJmXhsDgI9d1EcB0hgL+Oip4OZ6zXtF9TcTBfIHwHFHywKqUd11uWJxZbF5crc3UtZ1CNquZqF9SC+TUEFZNr62HXXAD0hvTx74/hDMyHSXQHL2/6t2l7/496Sq2NMSrJMaQGZIxjhLI+6hbFAoY3fAiJnSGQ5zA+Gwt7TMtJkE1ces6lzhTSA6ob4jSB44euKRYhFvRZ9ZRI3NwUYqXZII20QXgFAxJiOimil297giKIY1+iCPRKYI/epURzlrnruUIOne7rthEMTD+1FqdPULgQ60a57HmfrpdsR8gcevz6rOLYgRrbFVzshXcNiqjhrdPS8ZEJFVQpjFvhQucnjvw3bqhmo0sILOTZPNVU3swc8cUzRxo7Nzi0y3YueVJ2NjFzrUT7yrzfvXx5nNmYz8Qx92RTTyGYxRf4TSiUe4HJmwciNTUdylVEMlcqV5qe4wpxfNNinySMsqg7SnGsLU01YJBWqnYVFTcu3rHBBeQohVUaVwbCTR5QjlCYPe6tjM8t2r3hhFhDRXKSjJESE46tzJg9B2dKU/zH9nnNmGhUvJmj1+BpIa7p6IWmC6K6ZhG9yhoCmMIKlBUBDF9VtSFM3ARjUI+/XthFmAAgVlMS2UWz1eJpZTtHXRqLH6LxjOM73aglJqSUBL0skwoU3WMJ3eZKB7n5VLjOYggoWU97iPYk+dRXLajegzb6bu2wl+7c8W20YW5JVfUFGkaXLktIxDLE91Rle+VTRQvqUQgakOu3JB/6mwVzOwrflDDh4uIzpZjJustFh3LjbMIf0zYVUgrvdZQKLCLlgFrKCxphBSGpLgsnlA0kdQ8CoJYl5ZUN2RqWSGG9hCnpyF/q9aE4cp8AVS+0CQcntMMslb5huWrASD6vkRpBjOyTH6miOQJ9YqupsTIE+HIXlIae0vOcVqGIzvlIXLwa/ldCTUHyCvsrd1Q26jn3PPPDPt5Uc/QcAam92qinhjOxwHcUyplSC7niAGPnsDUFZOX4Ko9somzbQRj6DV1WDqxLSmXNjYvayj1u0dY1nCa6Od6iZjEMm6uswVqNT75Y8gm7wEFH1CawhD9uX+6ArbtPzNJ6w9M4Livhq7S7+lhkrNqieeluMrUBVthlGbTnUdbSXbx8oHWgZZ+9sKOZTRTp8d/Ht1Ll/ord3T3D/377vb9xF6ctF370dOZaUs7BnITe1o9llM72wQbafaJ+iLNrp8uh4On0xQdJEMs9VXDfLE1WgKeTYKxXLzFhb92XAikuBD41tG4Vr8QpnLtZ0TmUQIpUjc3fh+S3YlTu1qCblT2iXFqAJ9PcEtCdQxPAI7jeVsR6oE4TO+31WWuISJvnns3lO3qudfgczzTage+I/g9NS8t/V61w2pke/tvnu/38nwO8F/Y86nI6+XqTwXs9bPDJJHGF7bFn2v0APCehz/L80zzFaW4YD/Yc6Wo3+2+cIZbZti/7OIb2Hr/sO3ppHbzbL6WeQZtN01acSlQI7/98iXfJd9uxY9X/umYMF/MQOQs2yhMeSNadsy3Ekc0mqPRGKb5swfwwMdkQT1qvJezzp2McdstB2vbRPE4LGiqR6y+LIqzsprpzy2koz1Q0JgHVScDmCOPHuG4sgHxFPuwmn4FtdVnBfzhTnWYXzh23wyjTYkQcIErQL9lWlQZlNDoiZSR7yVpas789MdJPpIQJkwFOb3J6n3lp0hAeTzkCKdINhKIYIgBpKivP7H+luL+Uimu7fxkaYa1Y1R2aAVr9AVO0wWmoyhJKUwmmljv3PEdYGtwzqO9Rr+iqLT50IbrC/qzNefffc3x91K2d/2px4MGBP3mC/MH2/3c8GFHIyJY6NAF4vcmVe7eNnjwJLDJTWx7utUzxAH6tgTHPW212p5+S7fxRMvm+k1brffSNMvTfvhtDNVJv7HmpiBLSittV42yjsqa6jdDb/rbGCRL+tNELXvSHyuuP8IuzHX9KTs4+x8= \ No newline at end of file +7Vttc5s4EP41nrn7kAwg3vzRL0mu1/aaNO1lel88spExDUaukGMnv/4kEBgk2SExzkuTtjNFi5Bg99lndyW5Awbz9RmBi9lnHKC4YxnBugOGHcsyu2aX/cclt7mk6wtBSKJAdNoILqM7JISGkC6jAKW1jhTjmEaLunCCkwRNaE0GCcGrercpjuuzLmCIFMHlBMaq9CoK6CyX+pa3kf+FonBWzGy64vvmsOgsviSdwQCvKiJw0gEDgjHNr+brAYq58gq95M+dbrlbvhhBCW3ygBHj7/Havfjv293kq/XD/wEvfh7xB/gwNzBeii/uXV0ywSDGy0C8OL0ttLHAUUIzjTp99o9NODA6Drsz4K1jy5EEcturC0y1xceoC+S2VxeY8vCmNL8pv2BFoLRqwxvS/EblBdk/0MdLGkcJGpTYM5gwJDCImE0GOMaEyRKcMO31Z3Qes5bJLleziKLLBZxwra6Y3zDZFCdUoN+0irZQPB+VoWfBr+frkDvaMVyl9nFI8HKRTfmB4V97d8QuR5PMmGwQSvA1Kl6sYwH295SjpT+N4lh64RtEaMQcoRdHIR+bYj4VFK0YTSkfkX1FlISfstYQGOLNdVMEMJ2hQHyOCl6BZz4rWldEAsxnCM8RJbesS3HXFY4lmMUWzdXGTV3byWWziov6BbVAQQ1hOfTGe9iFcCC9M33p+0M4I9NRAs3Rx/Pedfrh36Ou4ktDvEpiDJkjGeMogWKOqkehgNGNaGJCZzjECYxPNtI+s2ISlIrb9PmEuUEyQP1ElN4K9MAlxXW4bdV1ipdkgnaxAxCMC0mI6K6O4tP51+y0HUExpNFNnVx1hhCPnnPC2djctZyazd2uWx8if1Px1MacPULgbaWb4LHt83S79XmAxK0P688u8jfYYKvUSSO47bROBW9fl4yJSKqgTOPeChc4PXfgu1VHNbeygMxOks+XQ7Xg5o5Zd3Og8XNLqLvm55YnYeMxfq5VvKvo/fzL5TcO/xskcPfELh7DMYrPcRrRKIsDE/YeiFTM90nqUCdzpXth7TGmFM/34pAiabuPQuy2GWQvE5uKiXvnH5jgDFK0giqNazOBbZFQzhDYva7tDE+tyr1hRNhAubESjggpSGdeBoy+o3PlafZHjrnbsFAaeXfEL0FSwd19WQtMF7k6ptEaBdvSGIJyFOVJTJ81dekMXESjUKi/FWYBBqgxi2mpzOL5KrEUstZBp+biV2g8w/j6EZRSMRJKgh6vhLipYzy55qJ4nLULi2cggoQW/USMYE+eRnExjhoxbKfv2gp/PZ4rmqYbZkOuqBjSNIRx98xALK8ejsp6rxgi/0glA1EHcuWB/GOnUTLTVv6gpg9nEZ0tx0zWWyw6lhvzDH9M2FVIS7tWUFhjFy0DVlBY0AjJHUkJWaKg2EZS8ygIYl1ZUt6QqWWFGNpDnB6H4qv2JgxX5gug8oWm4PCc7SDbq96wfDUBRL+WKOUwI8vkJWUk95i3HmoKjBwgHbEbUozTdjryqDrEAfW6whJsc9C6wm4chvbNek49/8SwH5b1DA1nYHpvJuuJ4XwcwJZKKUMKOU+Y8OgJTF0xeQ6uejybOE0TFuNF0IltSbW0sXtZQ+5vG0+wrOFso5+LJWISy7i84AvUan7yx5Ap7xYFn1GawhD92T5dAdv2H1ik9QcmcNw3Q1fpr/QwxVm5xPNcXGXqkq0wSrm6s2wr4RfPn2i9pKUfQ6XHf+7cTy71V+7o+h/69/XVx4m9aFzP7Vm7mbZEZ/IQLa0ey6WdbYKdNHtP/zrNbp4uXgdPpyk6SIVYGLCC+Xxr1DLKlWZjuXjPC193XgikvBD41pNxrX4hTOXab4jMowRSpG5uvHGSbbr1Jox9ZBwbwBcK3pNQHcOrAcfxvEaEeiAO08dtzS4ZmuMb9B61dwOqtai9AZ7jmdZ+wHuCmKfWpGrM4/v671Hv94p6DvCfOeqpyOtl5k9r2OvzgySRJg7uiz/X6AHgPQx/lueZ5hsqb0E72HOljN/tPnN1W1TXr2XhDTTeKmz9ZNJ+ejZ/Uz2D1jdM9uJSoGZ97fKl2CFvttonOr84JswWMhA54ZuEqRhEy47ZNuKIRnM0GsM0e/YAEfgpWVCPGu+VeWfT/Yf2z2fut5aqHq/6vsjPyWrUn3lIR3uYYGsdVJ4KYIE8uoPj0gfqJ9iHWvXvxIYC7/K3B2KWTvV4/5ZCCLjArUF/z7KodKjaoEdSNd5K0bSrFNQdJflCQpgwE2T0Jpv3jZ8gAcXRkCc4QbKTQGqOGECK+vrT6u8l7qsqcW3nhZUZ1iOzskMbWGMvcJwuMB1FSUphMtHkeqeO7wBbg3OR7bVgP8f1a/azNWfffc3R90LWuv3Uo0EDgt4X5RvtfO78DUfTRVSWOnSlM2Fl7b5v8uBJYJOHaHqy1TPqL+jbEhxb2ma1Pf127tbTLLv7b9tmXUtqltV++C0MNUi/s+auJEsqK21XzbKelDXV3wu9229nkizZT5O1tGQ/1tz8ADt3183P2MHJ/w== \ No newline at end of file diff --git a/docs/component-overview.svg b/docs/component-overview.svg index 433ac07387..49fe873255 100644 --- a/docs/component-overview.svg +++ b/docs/component-overview.svg @@ -1,3 +1,3 @@ -
AWS Cloud
AWS Cloud
Download binary
Download binary
Register runner and retrieve build
Register runner and retrieve buildRunners on AWS Spot Instances
POST event
POST event
API Gateway
API Gateway
Webhook
Webhook
Github App
Github App
Webhook
Webhook
Quedued builds
Quedued builds
Create runner token
Create runner token
Scale runners up
Scale runners...
Terminates
Terminates
Remove runner
Remove runner
Scale Runners Down
Scale Runners...
Actions Runners Binaries
Actions Runne...
Upload
Upload
Github Organization
Github Organi...
UpdateBinary
UpdateBinary
Creates
CreatesViewer does not support full SVG 1.1 \ No newline at end of file +
AWS Cloud
AWS Cloud
Download binary
Download binaryRunners
POST event
POST event
API Gateway
API Gateway
Webhook
Webhook
Github App
Github App
Request run event
Request run event
Webhook
Webhook
WebhookQueue SQS
(DelayedMessage)
WebhookQueue...
Register runner
Register runner
Scale Runners up
Scale Runners...
Terminates
Terminates
Remove runner
Remove runner
Scale Runners Down
Scale Runners...
Actions Runners Binaries
Actions Runne...
Upload
Upload
Github Organization
Github Organi...
UpdateBinary
UpdateBinary
Creates
CreatesViewer does not support full SVG 1.1 \ No newline at end of file