Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: CDK for infrastructure management and deployment #2576

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions .github/workflows/build_and_release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
name: Build and Deploy

on:
workflow_dispatch:

jobs:
release:
if: github.repository == 'dekkerglen/CubeCobraCDK' && github.ref == 'refs/heads/master'
permissions:
contents: write
runs-on: ubuntu-latest
outputs:
version: ${{ steps.get_version.outputs.version }}
steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm

- run: npm ci

- name: Run Semantic Release
id: semantic_release
run: |
npx semantic-release --ci
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Comment on lines +24 to +29
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should work because it's not a PR coming in from a fork. semantic-release needs write access to make a commit (update version in package.json) and create a tag.


- name: Extract Version from Tag
id: get_version
run: |
VERSION=$(git describe --tags --abbrev=0)
echo "version=${VERSION}" >> $GITHUB_OUTPUT

build:
if: github.repository == 'dekkerglen/CubeCobraCDK' && github.ref == 'refs/heads/master'
needs: release
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm

- run: npm ci

- run: npm run build

- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.GITHUB_IAM_ROLE }}
aws-region: ${{ secrets.AWS_REGION }}

- name: Upload archive to S3
run: npm run publish
env:
VERSION: ${{ needs.release.outputs.version }}

deploy:
uses: ./.github/workflows/cdk_deploy.yml
needs:
- build
- release
strategy:
matrix:
target:
- development
- production
Comment on lines +68 to +72
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will create 2 parallel jobs but both require manual approval before starting so development can be deployed first. We could have a deploy_dev and deploy_prod job that needs the dev one instead.

with:
version: ${{ needs.release.outputs.version }}
environment: ${{ matrix.target }}
secrets: inherit
109 changes: 109 additions & 0 deletions .github/workflows/cdk_deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
name: CDK Deployment

on:
workflow_dispatch: # Manual deployment via the GitHub UI
inputs:
version:
description: The version of the CubeCobra application to deploy (e.g. v1.2.3)
required: true
type: string
environment:
description: The deployment environment
required: true
default: development
type: choice
options:
- development
- production

workflow_call: # Allows triggering from other deploy workflows
inputs:
version:
description: The version of the CubeCobra application to deploy (e.g. v1.2.3)
required: true
type: string
environment:
description: The deployment environment
required: true
type: string

env:
EMAIL_USER: ${{ secrets.EMAIL_USER }}
EMAIL_PASS: ${{ secrets.EMAIL_PASS }}
JOBS_TOKEN: ${{ secrets.JOBS_TOKEN }}
PATREON_CLIENT_ID: ${{ secrets.PATREON_CLIENT_ID }}
PATREON_CLIENT_SECRET: ${{ secrets.PATREON_CLIENT_SECRET }}
PATREON_HOOK_SECRET: ${{ secrets.PATREON_HOOK_SECRET }}
SESSION_TOKEN: ${{ secrets.SESSION_TOKEN }}
SESSION_SECRET: ${{ secrets.SESSION_SECRET }}
TCG_PLAYER_PUBLIC_KEY: ${{ secrets.TCG_PLAYER_PUBLIC_KEY }}
TCG_PLAYER_PRIVATE_KEY: ${{ secrets.TCG_PLAYER_PRIVATE_KEY }}
CAPTCHA_SITE_KEY: ${{ secrets.CAPTCHA_SITE_KEY }}
CAPTCHA_SECRET_KEY: ${{ secrets.CAPTCHA_SECRET_KEY }}
DRAFTMANCER_API_KEY: ${{ secrets.DRAFTMANCER_API_KEY }}
STRIPE_SECRET_KEY: ${{ secrets.STRIPE_SECRET_KEY }}
STRIPE_PUBLIC_KEY: ${{ secrets.STRIPE_PUBLIC_KEY }}

defaults:
run:
working-directory: infra

permissions:
id-token: write

jobs:
plan:
if: github.repository == 'dekkerglen/CubeCobra'
environment: ${{ inputs.environment }}
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.CDK_IAM_ROLE }}
aws-region: ${{ secrets.AWS_REGION }}

- run: npm ci

- run: npm test

- run: npx cdk diff \
--context environment=${{ inputs.environment }} \
--context version=${{ inputs.version }}

deploy:
needs: plan
runs-on: ubuntu-latest
environment: ${{ inputs.environment }}
if: github.repository == 'dekkerglen/CubeCobra'

steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: infra/package-lock.json

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.CDK_IAM_ROLE }}
aws-region: ${{ secrets.AWS_REGION }}

- run: npm ci

- name: CDK Deploy
run: |
npx cdk deploy --all --require-approval never \
--context environment=${{ inputs.environment }} \
--context version=${{ inputs.version }}
30 changes: 30 additions & 0 deletions .github/workflows/cdk_tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: CDK Tests

on:
push:
paths:
- 'infra/**'
pull_request:
paths:
- 'infra/**'

defaults:
run:
working-directory: infra

jobs:
test:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
cache-dependency-path: infra/package-lock.json

- run: npm ci

- run: npm test
8 changes: 8 additions & 0 deletions infra/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
*.js
!jest.config.js
*.d.ts
node_modules

# CDK asset staging directory
.cdk.staging
cdk.out
6 changes: 6 additions & 0 deletions infra/.npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
*.ts
!*.d.ts

# CDK asset staging directory
.cdk.staging
cdk.out
83 changes: 83 additions & 0 deletions infra/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# CubeCobra CDK

This directory contains the CubeCobra CDK code that manages our infrastructure in AWS.

## Useful commands

* `npm run build` compile typescript to js
* `npm run watch` watch for changes and compile
* `npm run test` perform the jest unit tests
* `npx cdk deploy` deploy this stack to your default AWS account/region
* `npx cdk diff` compare deployed stack with current state
* `npx cdk synth` emits the synthesized CloudFormation template

## Bootstrapping

There are resources needed before CDK can be automated. By deploying once per environment with `--context bootstrap=true`
these resources will be created. For each new environment, run:

```bash
npx cdk diff --context environment=<environment> --context bootstrap=true

# then

npx cdk deploy --context environment=<environment> --context bootstrap=true
```

This should create all the required resources to then deploy the main stack.

## Manual Deployment

The following environment variables must be set:

| Name |
|--------------------------|
| `EMAIL_USER` |
| `EMAIL_PASS` |
| `JOBS_TOKEN` |
| `PATREON_CLIENT_ID` |
| `PATREON_CLIENT_SECRET` |
| `PATREON_HOOK_SECRET` |
| `SESSION_TOKEN` |
| `SESSION_SECRET` |
| `TCG_PLAYER_PUBLIC_KEY` |
| `TCG_PLAYER_PRIVATE_KEY` |
| `CAPTCHA_SITE_KEY` |
| `CAPTCHA_SECRET_KEY` |
| `DRAFTMANCER_API_KEY` |
| `STRIPE_SECRET_KEY` |
| `STRIPE_PUBLIC_KEY` |

You'll also need the app version which must match an app bundle on S3 (_v1.2.3_).

```bash
# First check the changes that will be applied by doing a diff with the live environment
npx cdk diff --context environment=<environment> --context version=<version>

# then deploy
npx cdk deploy --context environment=<environment> --context version=<version>
```

## Deploying your own environment

It's possible to use CDK to deploy your own environment with some effort. We use a combination of GitHub actions and
CDK to do so, but it's possible to do everything locally.

Do not commit any changes you make to the CDK code that is specific to your environment.

1. Add a new environment in [`config.ts`](./config.ts).
2. Build the CubeCobra application with `npm run build`
3. Upload the build to S3 using the [`publish.js`](./../scripts/publish.js) script. You'll need AWS credentials and
an S3 bucket. Make sure to set the `CUBECOBRA_APP_BUCKET` environment variable to your S3 bucket.
4. Set the required environment variables. An up-to-date list can be found in [
`cdk_deploy.yml`](./../.github/workflows/cdk_deploy.yml)
5. Deploy the environment with CDK:
`npx cdk deploy --context environment=<your-environment> --context version=<version>`

* `<your-environment`> is the key you added in [`config.ts`](./config.ts).
* `<version>` is taken from [`package.json`](./../package.json)

If everything worked you should now have your own CubeCobra environment.

You can run `npx cdk deploy --context environment=<your-environment>` to destroy all cloud resources CDK created when
you no longer need your environment.
72 changes: 72 additions & 0 deletions infra/app/infra.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
#!/usr/bin/env node
import * as cdk from 'aws-cdk-lib';

import 'source-map-support/register';

import { environments } from '../config';
import { BootstrapStack } from '../lib/bootstrap-stack';
import { CubeCobraStack } from '../lib/cubecobra-stack';

const app = new cdk.App();

const environment = app.node.tryGetContext('environment');
if (!environment || !environments[environment]) {
throw new Error(`Invalid or missing environment. Available environments: ${Object.keys(environments).join(', ')}`);
}

const config = environments[environment];

const bootstrap = app.node.tryGetContext('bootstrap');
if (bootstrap && bootstrap === 'true') {
console.log(`Bootstrapping environment '${environment}'`);
new BootstrapStack(app, `BootstrapStack${environment}`, {
env: { account: config.account, region: config.region },
});
} else {
const version = app.node.tryGetContext('version');
if (!version || version === '') {
throw new Error('Invalid or missing version. Version should be "v1.2.3"');
}

console.log(`Deploying CubeCobra stack to '${environment}'`);

new CubeCobraStack(
app,
config.stackName,
{
accessKey: process.env.AWS_ACCESS_KEY_ID || '',
secretKey: process.env.AWS_SECRET_ACCESS_KEY || '',
emailUser: process.env.EMAIL_USER || '',
emailPass: process.env.EMAIL_PASS || '',
domain: config.domain,
environmentName: environment,
version: version,
awsLogGroup: config.awsLogGroup,
awsLogStream: config.awsLogStream,
dataBucket: config.dataBucket,
appBucket: config.appBucket,
downTimeActive: config.downTimeActive,
dynamoPrefix: config.dynamoPrefix,
env: environment,
jobsToken: process.env.JOBS_TOKEN || '',
nitroPayEnabled: config.nitroPayEnabled,
patreonClientId: process.env.PATREON_CLIENT_ID || '',
patreonClientSecret: process.env.PATREON_CLIENT_SECRET || '',
patreonHookSecret: process.env.PATREON_HOOK_SECRET || '',
patreonRedirect: config.patreonRedirectUri,
sessionToken: process.env.SESSION_TOKEN || '',
sessionSecret: process.env.SESSION_SECRET || '',
tcgPlayerPublicKey: process.env.TCG_PLAYER_PUBLIC_KEY || '',
tcgPlayerPrivateKey: process.env.TCG_PLAYER_PRIVATE_KEY || '',
fleetSize: config.fleetSize,
captchaSiteKey: process.env.CAPTCHA_SITE_KEY || '',
captchaSecretKey: process.env.CAPTCHA_SECRET_KEY || '',
draftmancerApiKey: process.env.DRAFTMANCER_API_KEY || '',
stripeSecretKey: process.env.STRIPE_SECRET_KEY || '',
stripePublicKey: process.env.STRIPE_PUBLIC_KEY || '',
},
{
env: { account: config.account, region: config.region },
},
);
}
Loading
Loading