Skip to content

Latest commit

 

History

History
131 lines (101 loc) · 10 KB

README.md

File metadata and controls

131 lines (101 loc) · 10 KB
Raw

AWS CodeBuild Cloud Native Buildpack pack CLI Docker image

This repository holds a Dockerfile based on AWS CodeBuild Docker Images with focus on building Cloud Native Buildpacks using pack CLI.

Build Status

Advantages

  • Does not require to install pack and other dependencies at runtime
  • Will provide docker login implicitly as CodeBuild service role allows
  • Build step is automatically skipped if commit is the same
  • Will register deployments within NewRelic API

Current process steps:

  1. Test (TBD)
  2. Build
  3. Release/Render
  4. Provision
  5. Deploy

How to use this image on AWS Codebuild

Create or edit your CodeBuild project's evironment sections and override as described below:

CodeBuild Environment

Create a buildspec.yml like this:

version: 0.2
phases:
  build:
    on-failure: ABORT
    commands:
      - dockerd-entrypoint.sh main.sh

Extra Features

  • Creation of scheduled tasks
  • Execution of code before release (database migrations, assets upload, etc)

Procfile

Below you can see a sample Procfile that demonstrates some of Maestro's capabilities:

web: bundle exec unicorn -p $PORT -c config/unicorn.rb # binds unicorn via $PORT to an ALB for incoming traffic
worker: bundle exec sidekiq -C config/sidekiq.yml      # runs sidekiq for processing background jobs
clock: bundle exec rails runner bin/clock              # loads a cron based service that will trigger commands
scheduledtasks: tasks.run                              # creates tasks for one-off executions scheduled at tasks.run
release: bundle exec db:migrate                        # runs database migration right before the release is deployed

Environment Variables

Variable Description Examples/Values Default
ALB_SCHEME Scheme of the ALB

Choose only one of the example values		 internet-facing
internal		 internet-facing
ALB_SECURITY_GROUPS Security Groups linked to ALB

Multiple values can be assigned using comma as separator		 sg-qwerty
sg-asdfgh,sg-nth
ALB_SUBNETS Subnets linked to ALB

Multiple values can be assigned using comma as separator		 subnet-qwer1234567890
subnet-asdf0987654321,subnet-nth
ECS_EFS_VOLUMES Add EFS volumes for ECS tasks

Multiple values can be assigned using comma as separator		 <volume-name>:<filesystem-id>{<efs-root>@<path-to-task-build>;<to-encrypt-in-transit>}
storage-efs:fs-013a693f90df46413{/@public/storage;encrypted},images-efs:fs-0bd8f82bba0a89448{/@public/images;encrypted}
ECS_EXECUTION_ROLE_ARN IAM Role ARN linked to ECS Execution arn:aws:iam::0123456789:role/<role-name>
ECS_SERVICE_SECURITY_GROUPS Security Groups linked to the ECS Service

Multiple values can be assigned using comma as separator		 sg-qwerty
sg-asdfgh,sg-nth
ECS_SERVICE_SUBNETS Subnets linked to the ECS Service

Multiple values can be assigned using comma as separator		 subnet-qwer1234567890
subnet-asdf0987654321,subnet-nth
ECS_SERVICE_TASK_PROCESSES Processes intended to have a service to be created according to Procfile

Multiple services can be assigned using comma as separator

For CPU and RAM assignment use semicolon as separator and curly brackets as container
For number of tasks assignment use colon as separator and a dash between numbers to define max task auto scaling value.
For CPU and RAM usage percentage and ALB request count auto scaling triggers assignment use semicolon as separator and square brackets as container. By adding the optional set "&nosclin", after percentage usage value for CPU,RAM or/and ALB count, the scale in will be turned off.		 web
web,worker
web{1024;2048}:2-5[mem=55&nosclin;cpu=60;alb=1000]
web{512;1024}:2,worker{1024;2048}:1-3

Add scheduledtasks{256;512} process to be able to create Scheduled Tasks. Does not create an ECS service, only enables the feature

<process>{<v-cpus>;<mem>}:<min_tasks>-<max_tasks>[<mem=percent>;<cpu=percent>&nosclin]

<process> = Process name
<v-cpus> = Desired number of v-cpus per task x1024
<mem> = Desired allocated MB of RAM per task
<min_tasks> = Minimum amount of tasks per process, can be used without a max value so as to not use auto scaling
<max_tasks> = Maximum amount of tasks per process
<mem=percent>= RAM usage percentage auto scaling trigger
<cpu=percent> = CPU usage percentage auto scaling trigger
<alb=requestcount> = Load Balancer request count auto scaling trigger

Example using all default values except for Tasks per Process:
test:2-5,scheduledtasks
Fully disclosed example would be:
test{512;512}:2-5[cpu=55],scheduledtasks{256;512}		 <v-cpus> = 512
<mem> = 512
<min_tasks> = 1
<cpu=percent> = 55 (if auto scaling is active)

scheduledtasks <v-cpus> = 256
scheduledtasks <mem> = 512
ECS_TASK_ROLE_ARN IAM Role ARN linked to ECS Task arn:aws:iam::0123456789:role/<role-name>
MAESTRO_BRANCH_OVERRIDE Temporary overriding of the working branch staging
production
MAESTRO_REPO_OVERRIDE Temporary overriding of the repo name
MAESTRO_DEBUG Amplify verbosity of the build

Choose only one of the example values		 true
false
MAESTRO_NO_CACHE If the cache layer shouldn't be used in the pack build

Choose only one of the example values		 true
false
MAESTRO_ONLY_BUILD Stops after build if true, leave empty otherwise true
MAESTRO_SKIP_BUILD Skips build and process following steps if true, leave empty otherwise true
MAESTRO_RUN_IMAGE Provides the base image for application images. 123456789.dkr.ecr.us-east-1.amazonaws.com/base_image:latest null
NEW_RELIC_API_KEY A user API key from NewRelic for registering deploys NRAK-XXXXXXXXXXXXXXXXXXXXXXXXX
NEW_RELIC_APP_ID NewRelic APM's application identifier 123456789
NEW_RELIC_DESCRIPTION Git source's deploys description to NewRelic. (Reference: https://git-scm.com/docs/git-log) The author of %h was %an, %ar%nThe title was >>%s<<%n
DEPLOY_WEBHOOK_URL A custom webhook URL for registering deploys https://example.com/my-webhook?apikey=123456&cluster_id={{CLUSTER}}&service={{SERVICE}}&repository={{REPOSITORY}}
REPO_SUB_FOLDER Use a sub folder with the application code
WORKLOAD_RESOURCE_TAGS Tags related to the workload that will be used to all resources provisioned

Examples include tag name (case-insensitive) and value		 workload=myapp
environment=staging
owner=me
WORKLOAD_VPC_ID VPC ID of the workload vpc-ad1234df
vpc-qw56er78
vpc-zxcvghjk
DEPLOYMENT_CIRCUIT_BREAKER_RULE Enable or disable circuit breaker enable=true,rollback=true
ECS_CONTAINER_STOP_TIMEOUT Set stopTimeout on taskdefinition min: 0, max: 120, default: 30
TZ Set this variable to the desired task timezone America/Sao_Paulo
ALB_NAME_OVERRIDE Set this variable to temporary overriding the ALB name test-alb

How to enable scheduled tasks

  • Create a file tasks/run_tasks.conf with the schedules on your code: Content example: update-feeed rate(3 hours) bundle exec rake feed:update send-mail rate(12 hours) bundle exec rake mail:send update-invoices rate(6 hours) bundle exec rake client:update_invoices

  • Update your project code, adding the following to Procfile: scheduledtasks: scheduledtasks: tasks/run_task.conf

  • On codebuild environment variables, increment the ECS_SERVICE_TASK_PROCESSES with "scheduledtasks{256;512}" as described in the table above.

How to build Docker image

Steps to build image:

$ git clone https://github.com/veezor/maestro.git
$ docker build -t maestro:latest .

Release process

To trigger the release workflow and deploy the image on AWS public ECR repository use one of the two approaches below:

  1. Create a pull request for your changes or bug fixes. This will deploy a pre-release image that can be tested with the proposed changes.
  2. Create a new tag and a new release, following semantic versioning parameters.

How to register a deploy with a webhook

Maestro allows you to register a deploy with a custom webhook using a standard GET request which expects a 200 response. To trigger this feature, you need to define DEPLOY_WEBHOOK_URL environment variable and its value may contain the following placeholders within the URL:

  • {{CLUSTER}} for the cluster's name
  • {{CLUSTER_LINK}} for the cluster's link
  • {{REPOSITORY}} for the repository's slug
  • {{REPO_LINK}} for the repository's link
  • {{BUILD_LINK}} for the build's log link

Troubleshooting

  1. DockerHub hate limit If you are esperiencing DockerHub rate limits, clone the heroku/heroku:xx-cnb image to your ECR repository. Edit the buildspec section on your Codebuild, and add the following instruction inside the commands: session, at the first line.

`- pack config run-image-mirrors add heroku/heroku:20-cnb --mirror public.ecr.aws/x1o4r4n3/maestro-builder-20:runimage`

Contributing

Feel free to suggest improvements by opening pull requests.