-
Notifications
You must be signed in to change notification settings - Fork 127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build: CDK for infrastructure management and deployment #2576
Open
almarcotte
wants to merge
17
commits into
dekkerglen:master
Choose a base branch
from
almarcotte:feat/move-cdk-to-main-repo
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 14 commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
82c2de2
build: moving CDK code to main repo
almarcotte e847f99
fix: jest config eslint
almarcotte 149616f
build: add app deploy from previous PR
almarcotte e3b040e
build: reduce duplicate code
almarcotte 6e9e633
feat: bringing in latest CDK changes and doing some clean up
almarcotte 0fd3b3e
build: adding environment to plan step
almarcotte 5f63eab
chore: make sure jest works with typescript
almarcotte 49ddb2a
build: run cdk tests on code changes
almarcotte 7608f78
test: adding CDK tests
almarcotte d1cb35f
feat: setting `NODE_ENV`
almarcotte b560ec2
build: fix working directory still using old name
almarcotte 035cca2
docs: manual cdk deploy doc
almarcotte 5cf0774
build: remove ref to old repository
almarcotte 3af9871
build: bootstrap stack
almarcotte d4bc7bc
Merge branch 'refs/heads/master' into feat/move-cdk-to-main-repo
almarcotte d007fd7
build: text output for coverage
almarcotte d7925d9
build: fixing typo
almarcotte File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
name: Build and Deploy | ||
|
||
on: | ||
workflow_dispatch: | ||
|
||
jobs: | ||
release: | ||
if: github.repository == 'dekkerglen/CubeCobraCDK' && github.ref == 'refs/heads/master' | ||
permissions: | ||
contents: write | ||
runs-on: ubuntu-latest | ||
outputs: | ||
version: ${{ steps.get_version.outputs.version }} | ||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- uses: actions/setup-node@v4 | ||
with: | ||
node-version: 20 | ||
cache: npm | ||
|
||
- run: npm ci | ||
|
||
- name: Run Semantic Release | ||
id: semantic_release | ||
run: | | ||
npx semantic-release --ci | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
||
- name: Extract Version from Tag | ||
id: get_version | ||
run: | | ||
VERSION=$(git describe --tags --abbrev=0) | ||
echo "version=${VERSION}" >> $GITHUB_OUTPUT | ||
|
||
build: | ||
if: github.repository == 'dekkerglen/CubeCobraCDK' && github.ref == 'refs/heads/master' | ||
needs: release | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- uses: actions/setup-node@v4 | ||
with: | ||
node-version: 20 | ||
cache: npm | ||
|
||
- run: npm ci | ||
|
||
- run: npm run build | ||
|
||
- uses: aws-actions/configure-aws-credentials@v4 | ||
with: | ||
role-to-assume: ${{ secrets.GITHUB_IAM_ROLE }} | ||
aws-region: ${{ secrets.AWS_REGION }} | ||
|
||
- name: Upload archive to S3 | ||
run: npm run publish | ||
env: | ||
VERSION: ${{ needs.release.outputs.version }} | ||
|
||
deploy: | ||
uses: ./.github/workflows/cdk_deploy.yml | ||
needs: | ||
- build | ||
- release | ||
strategy: | ||
matrix: | ||
target: | ||
- development | ||
- production | ||
Comment on lines
+68
to
+72
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This will create 2 parallel jobs but both require manual approval before starting so development can be deployed first. We could have a |
||
with: | ||
version: ${{ needs.release.outputs.version }} | ||
environment: ${{ matrix.target }} | ||
secrets: inherit |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
name: CDK Deployment | ||
|
||
on: | ||
workflow_dispatch: # Manual deployment via the GitHub UI | ||
inputs: | ||
version: | ||
description: The version of the CubeCobra application to deploy (e.g. v1.2.3) | ||
required: true | ||
type: string | ||
environment: | ||
description: The deployment environment | ||
required: true | ||
default: development | ||
type: choice | ||
options: | ||
- development | ||
- production | ||
|
||
workflow_call: # Allows triggering from other deploy workflows | ||
inputs: | ||
version: | ||
description: The version of the CubeCobra application to deploy (e.g. v1.2.3) | ||
required: true | ||
type: string | ||
environment: | ||
description: The deployment environment | ||
required: true | ||
type: string | ||
|
||
env: | ||
EMAIL_USER: ${{ secrets.EMAIL_USER }} | ||
EMAIL_PASS: ${{ secrets.EMAIL_PASS }} | ||
JOBS_TOKEN: ${{ secrets.JOBS_TOKEN }} | ||
PATREON_CLIENT_ID: ${{ secrets.PATREON_CLIENT_ID }} | ||
PATREON_CLIENT_SECRET: ${{ secrets.PATREON_CLIENT_SECRET }} | ||
PATREON_HOOK_SECRET: ${{ secrets.PATREON_HOOK_SECRET }} | ||
SESSION_TOKEN: ${{ secrets.SESSION_TOKEN }} | ||
SESSION_SECRET: ${{ secrets.SESSION_SECRET }} | ||
TCG_PLAYER_PUBLIC_KEY: ${{ secrets.TCG_PLAYER_PUBLIC_KEY }} | ||
TCG_PLAYER_PRIVATE_KEY: ${{ secrets.TCG_PLAYER_PRIVATE_KEY }} | ||
CAPTCHA_SITE_KEY: ${{ secrets.CAPTCHA_SITE_KEY }} | ||
CAPTCHA_SECRET_KEY: ${{ secrets.CAPTCHA_SECRET_KEY }} | ||
DRAFTMANCER_API_KEY: ${{ secrets.DRAFTMANCER_API_KEY }} | ||
STRIPE_SECRET_KEY: ${{ secrets.STRIPE_SECRET_KEY }} | ||
STRIPE_PUBLIC_KEY: ${{ secrets.STRIPE_PUBLIC_KEY }} | ||
|
||
defaults: | ||
run: | ||
working-directory: infra | ||
|
||
permissions: | ||
id-token: write | ||
|
||
jobs: | ||
plan: | ||
if: github.repository == 'dekkerglen/CubeCobra' | ||
environment: ${{ inputs.environment }} | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- uses: actions/setup-node@v4 | ||
with: | ||
node-version: 20 | ||
cache: npm | ||
|
||
- name: Configure AWS credentials | ||
uses: aws-actions/configure-aws-credentials@v4 | ||
with: | ||
role-to-assume: ${{ secrets.CDK_IAM_ROLE }} | ||
aws-region: ${{ secrets.AWS_REGION }} | ||
|
||
- run: npm ci | ||
|
||
- run: npm test | ||
|
||
- run: npx cdk diff \ | ||
--context environment=${{ inputs.environment }} \ | ||
--context version=${{ inputs.version }} | ||
|
||
deploy: | ||
needs: plan | ||
runs-on: ubuntu-latest | ||
environment: ${{ inputs.environment }} | ||
if: github.repository == 'dekkerglen/CubeCobra' | ||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- uses: actions/setup-node@v4 | ||
with: | ||
node-version: 20 | ||
cache: npm | ||
cache-dependency-path: infra/package-lock.json | ||
|
||
- name: Configure AWS credentials | ||
uses: aws-actions/configure-aws-credentials@v4 | ||
with: | ||
role-to-assume: ${{ secrets.CDK_IAM_ROLE }} | ||
aws-region: ${{ secrets.AWS_REGION }} | ||
|
||
- run: npm ci | ||
|
||
- name: CDK Deploy | ||
run: | | ||
npx cdk deploy --all --require-approval never \ | ||
--context environment=${{ inputs.environment }} \ | ||
--context version=${{ inputs.version }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
name: CDK Tests | ||
|
||
on: | ||
push: | ||
paths: | ||
- 'infra/**' | ||
pull_request: | ||
paths: | ||
- 'infra/**' | ||
|
||
defaults: | ||
run: | ||
working-directory: infra | ||
|
||
jobs: | ||
test: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- uses: actions/setup-node@v4 | ||
with: | ||
node-version: 20 | ||
cache: 'npm' | ||
cache-dependency-path: infra/package-lock.json | ||
|
||
- run: npm ci | ||
|
||
- run: npm test |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
*.js | ||
!jest.config.js | ||
*.d.ts | ||
node_modules | ||
|
||
# CDK asset staging directory | ||
.cdk.staging | ||
cdk.out |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
*.ts | ||
!*.d.ts | ||
|
||
# CDK asset staging directory | ||
.cdk.staging | ||
cdk.out |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,83 @@ | ||
# CubeCobra CDK | ||
|
||
This directory contains the CubeCobra CDK code that manages our infrastructure in AWS. | ||
|
||
## Useful commands | ||
|
||
* `npm run build` compile typescript to js | ||
* `npm run watch` watch for changes and compile | ||
* `npm run test` perform the jest unit tests | ||
* `npx cdk deploy` deploy this stack to your default AWS account/region | ||
* `npx cdk diff` compare deployed stack with current state | ||
* `npx cdk synth` emits the synthesized CloudFormation template | ||
|
||
## Bootstrapping | ||
|
||
There are resources needed before CDK can be automated. By deploying once per environment with `--context bootstrap=true` | ||
these resources will be created. For each new environment, run: | ||
|
||
```bash | ||
npx cdk diff --context environment=<environment> --context bootstrap=true | ||
|
||
# then | ||
|
||
npx cdk deploy --context environment=<environment> --context bootstrap=true | ||
``` | ||
|
||
This should create all the required resources to then deploy the main stack. | ||
|
||
## Manual Deployment | ||
|
||
The following environment variables must be set: | ||
|
||
| Name | | ||
|--------------------------| | ||
| `EMAIL_USER` | | ||
| `EMAIL_PASS` | | ||
| `JOBS_TOKEN` | | ||
| `PATREON_CLIENT_ID` | | ||
| `PATREON_CLIENT_SECRET` | | ||
| `PATREON_HOOK_SECRET` | | ||
| `SESSION_TOKEN` | | ||
| `SESSION_SECRET` | | ||
| `TCG_PLAYER_PUBLIC_KEY` | | ||
| `TCG_PLAYER_PRIVATE_KEY` | | ||
| `CAPTCHA_SITE_KEY` | | ||
| `CAPTCHA_SECRET_KEY` | | ||
| `DRAFTMANCER_API_KEY` | | ||
| `STRIPE_SECRET_KEY` | | ||
| `STRIPE_PUBLIC_KEY` | | ||
|
||
You'll also need the app version which must match an app bundle on S3 (_v1.2.3_). | ||
|
||
```bash | ||
# First check the changes that will be applied by doing a diff with the live environment | ||
npx cdk diff --context environment=<environment> --context version=<version> | ||
|
||
# then deploy | ||
npx cdk deploy --context environment=<environment> --context version=<version> | ||
``` | ||
|
||
## Deploying your own environment | ||
|
||
It's possible to use CDK to deploy your own environment with some effort. We use a combination of GitHub actions and | ||
CDK to do so, but it's possible to do everything locally. | ||
|
||
Do not commit any changes you make to the CDK code that is specific to your environment. | ||
|
||
1. Add a new environment in [`config.ts`](./config.ts). | ||
2. Build the CubeCobra application with `npm run build` | ||
3. Upload the build to S3 using the [`publish.js`](./../scripts/publish.js) script. You'll need AWS credentials and | ||
an S3 bucket. Make sure to set the `CUBECOBRA_APP_BUCKET` environment variable to your S3 bucket. | ||
4. Set the required environment variables. An up-to-date list can be found in [ | ||
`cdk_deploy.yml`](./../.github/workflows/cdk_deploy.yml) | ||
5. Deploy the environment with CDK: | ||
`npx cdk deploy --context environment=<your-environment> --context version=<version>` | ||
|
||
* `<your-environment`> is the key you added in [`config.ts`](./config.ts). | ||
* `<version>` is taken from [`package.json`](./../package.json) | ||
|
||
If everything worked you should now have your own CubeCobra environment. | ||
|
||
You can run `npx cdk deploy --context environment=<your-environment>` to destroy all cloud resources CDK created when | ||
you no longer need your environment. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
#!/usr/bin/env node | ||
import * as cdk from 'aws-cdk-lib'; | ||
|
||
import 'source-map-support/register'; | ||
|
||
import { environments } from '../config'; | ||
import { BootstrapStack } from '../lib/bootstrap-stack'; | ||
import { CubeCobraStack } from '../lib/cubecobra-stack'; | ||
|
||
const app = new cdk.App(); | ||
|
||
const environment = app.node.tryGetContext('environment'); | ||
if (!environment || !environments[environment]) { | ||
throw new Error(`Invalid or missing environment. Available environments: ${Object.keys(environments).join(', ')}`); | ||
} | ||
|
||
const config = environments[environment]; | ||
|
||
const bootstrap = app.node.tryGetContext('bootstrap'); | ||
if (bootstrap && bootstrap === 'true') { | ||
console.log(`Bootstrapping environment '${environment}'`); | ||
new BootstrapStack(app, `BootstrapStack${environment}`, { | ||
env: { account: config.account, region: config.region }, | ||
}); | ||
} else { | ||
const version = app.node.tryGetContext('version'); | ||
if (!version || version === '') { | ||
throw new Error('Invalid or missing version. Version should be "v1.2.3"'); | ||
} | ||
|
||
console.log(`Deploying CubeCobra stack to '${environment}'`); | ||
|
||
new CubeCobraStack( | ||
app, | ||
config.stackName, | ||
{ | ||
accessKey: process.env.AWS_ACCESS_KEY_ID || '', | ||
secretKey: process.env.AWS_SECRET_ACCESS_KEY || '', | ||
emailUser: process.env.EMAIL_USER || '', | ||
emailPass: process.env.EMAIL_PASS || '', | ||
domain: config.domain, | ||
environmentName: environment, | ||
version: version, | ||
awsLogGroup: config.awsLogGroup, | ||
awsLogStream: config.awsLogStream, | ||
dataBucket: config.dataBucket, | ||
appBucket: config.appBucket, | ||
downTimeActive: config.downTimeActive, | ||
dynamoPrefix: config.dynamoPrefix, | ||
env: environment, | ||
jobsToken: process.env.JOBS_TOKEN || '', | ||
nitroPayEnabled: config.nitroPayEnabled, | ||
patreonClientId: process.env.PATREON_CLIENT_ID || '', | ||
patreonClientSecret: process.env.PATREON_CLIENT_SECRET || '', | ||
patreonHookSecret: process.env.PATREON_HOOK_SECRET || '', | ||
patreonRedirect: config.patreonRedirectUri, | ||
sessionToken: process.env.SESSION_TOKEN || '', | ||
sessionSecret: process.env.SESSION_SECRET || '', | ||
tcgPlayerPublicKey: process.env.TCG_PLAYER_PUBLIC_KEY || '', | ||
tcgPlayerPrivateKey: process.env.TCG_PLAYER_PRIVATE_KEY || '', | ||
fleetSize: config.fleetSize, | ||
captchaSiteKey: process.env.CAPTCHA_SITE_KEY || '', | ||
captchaSecretKey: process.env.CAPTCHA_SECRET_KEY || '', | ||
draftmancerApiKey: process.env.DRAFTMANCER_API_KEY || '', | ||
stripeSecretKey: process.env.STRIPE_SECRET_KEY || '', | ||
stripePublicKey: process.env.STRIPE_PUBLIC_KEY || '', | ||
}, | ||
{ | ||
env: { account: config.account, region: config.region }, | ||
}, | ||
); | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should work because it's not a PR coming in from a fork.
semantic-release
needs write access to make a commit (updateversion
inpackage.json
) and create a tag.