Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add Aspect Workflows CI (on AWS + Buildkite) #154

Merged
merged 1 commit into from
Aug 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions .aspect/workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Aspect Workflows demonstration deployment

This deployment of [Aspect Workflows](https://www.aspect.build/workflows) is configured to run on AWS + Buildkite.

You can see this Aspect Workflows demonstration deployment live at
https://buildkite.com/aspect/rules-jest.

The three components of the configuration are,

1. Aspect Workflows terraform module
1. Aspect Workflows configuration yaml
1. Buildkite pipeline configuration (in the Buildkite UI)

## Aspect Workflows terraform module

This is found under the [.aspect/workflows/terraform](./terraform) directory.

## Aspect Workflows configuration yaml

This is the [config.yaml](./config.yaml) file in this directory.

## Buildkite pipeline configuration (in the Buildkite UI)

There are two pipelines configured on Buildkite.

1. Main build & test pipeline: https://buildkite.com/aspect/rules-jest
2. Scheduled warming pipeline: https://buildkite.com/aspect/rules-jest-warming
gregmagolan marked this conversation as resolved.
Show resolved Hide resolved

### Main build & test pipeline configuration

The main build & test pipeline found at https://buildkite.com/aspect/rules-jest is configured
with the following yaml steps:

```
steps:
- key: aspect-workflows-setup
label: ":aspect: Setup Aspect Workflows"
commands:
- "rosetta steps | buildkite-agent pipeline upload"
agents:
queue: default
```

### Scheduled warming pipeline configuration

The scheduled warming pipeline found at https://buildkite.com/aspect/rules-jest-warming is
configured with the following yaml steps:

```
steps:
- label: ":fire: Create warming archives"
commands:
- 'echo "--- :aspect: Configure environment"'
- 'configure_workflows_env'
- 'echo "--- :stethoscope: Agent health checks"'
- 'agent_health_check'
- 'echo "--- :bazel: Create warming archive for ."'
- 'rosetta run warming'
- 'warming_archive'
agents:
queue: warming
```

A scheduled is configured for this pipeline with the cron interval `0 08-22/4 * * * America/Toronto`
so that it runs periodically to create up-to-date warming archives that caches repository rules so
that the "default" build & test runners don't have to re-fetch them on their first build.
11 changes: 11 additions & 0 deletions .aspect/workflows/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# See https://docs.aspect.build/v/workflows/config
---
queue: default
workspaces:
.:
bazel:
flags:
- --enable_bzlmod
tasks:
buildifier:
test:
204 changes: 204 additions & 0 deletions .aspect/workflows/terraform/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 7 additions & 0 deletions .aspect/workflows/terraform/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Aspect Workflows demonstration deployment terraform

The terraform configuration found here is for a clean AWS sub-account with only Aspect Workflows deployed.

- `main.tf` : terraform backend configuration
- `vpc.tf` : VPC configuration
- `workflows.tf` : Aspect Workflows terraform module & AMI configuration
56 changes: 56 additions & 0 deletions .aspect/workflows/terraform/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
terraform {
required_version = "~> 1.4.0"

backend "s3" {
bucket = "aw-deployment-terraform-state-rules-jest"
key = "global/s3/terraform.tfstate"
region = "us-west-2"
}

required_providers {
aws = {
source = "hashicorp/aws",
version = "~> 4.58.0"
}
}
}

provider "aws" {
region = "us-west-2"
}

resource "aws_s3_bucket" "terraform_state" {
bucket = "aw-deployment-terraform-state-rules-jest"

lifecycle {
prevent_destroy = true
}
}

resource "aws_s3_bucket_versioning" "terraform_state_versioning" {
bucket = aws_s3_bucket.terraform_state.id
versioning_configuration {
status = "Enabled"
}
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state_encryption" {
bucket = aws_s3_bucket.terraform_state.id

rule {
apply_server_side_encryption_by_default {
sse_algorithm = "AES256"
}
}
}

resource "aws_s3_bucket_public_access_block" "terraform_state_pab" {
bucket = aws_s3_bucket.terraform_state.id

block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}

data "aws_region" "default" {}
49 changes: 49 additions & 0 deletions .aspect/workflows/terraform/vpc.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
locals {
cidr = "10.0.0.0/16"
azs = ["us-west-2a", "us-west-2b", "us-west-2c"]

num_azs = length(local.azs)
num_bits_needed_for_azs = ceil(log(local.num_azs, 2))

private_cidr = cidrsubnet(local.cidr, 1, 0)
private_subnets = [
for i in range(local.num_azs) : cidrsubnet(local.private_cidr, local.num_bits_needed_for_azs, i)
]

public_cidr = cidrsubnet(local.cidr, 1, 1)
public_subnets = [
for i in range(local.num_azs) : cidrsubnet(local.public_cidr, local.num_bits_needed_for_azs, i)
]
}

module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "4.0.2"

name = "aw_dev_vpc"
cidr = local.cidr

azs = local.azs
private_subnets = local.private_subnets
public_subnets = local.public_subnets

enable_nat_gateway = true
single_nat_gateway = true
enable_vpn_gateway = false
map_public_ip_on_launch = true
}

module "vpc_endpoints" {
source = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"
version = "4.0.2"

vpc_id = module.vpc.vpc_id
endpoints = {
s3 = {
service = "s3"
service_type = "Gateway"
tags = { Name = "s3-vpc-endpoint" }
route_table_ids = module.vpc.private_route_table_ids
},
}
}
Loading