Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for CD and environment variables #26

Merged
merged 15 commits into from
Apr 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
MONGO_AUTH_CREDENTIALS=""
37 changes: 37 additions & 0 deletions .github/workflows/deploy_to_prod.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# This workflow will update the code for the production environment

on:
push:
branches: ["release"]

name: Deploy to production

jobs:
build_and_deploy:
name: Deploy
environment:
name: Production

runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v2

- name: Set up SAM
uses: aws-actions/setup-sam@v1

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1

- name: Build
run: sam build --use-container -t templates/prod.yaml

- name: Deploy
run: sam deploy --stack-name QuizBackendProd --s3-bucket quiz-prod-backend --no-confirm-changeset --no-fail-on-empty-changeset --region us-east-1 --capabilities CAPABILITY_IAM --parameter-overrides MongoAuthCredentials=${{ secrets.MONGO_AUTH_CREDENTIALS }}
38 changes: 38 additions & 0 deletions .github/workflows/deploy_to_staging.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# This workflow will update the code for the staging environment

on:
pull_request:
push:
branches: ["main"]

name: Deploy to staging

jobs:
build_and_deploy:
name: Deploy
environment:
name: Staging

runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v2

- name: Set up SAM
uses: aws-actions/setup-sam@v1

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1

- name: Build
run: sam build --use-container -t templates/staging.yaml

- name: Deploy
run: sam deploy --stack-name QuizBackendStaging --s3-bucket quiz-staging-backend --no-confirm-changeset --no-fail-on-empty-changeset --region us-east-1 --capabilities CAPABILITY_IAM --parameter-overrides MongoAuthCredentials=${{ secrets.MONGO_AUTH_CREDENTIALS }}
53 changes: 4 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ pip install pre-commit
pre-commit install
```

- Copy `.env.example` to `.env` and set all the environment variables as mentioned in `docs/ENV.md`.

## Running locally

Simply run:
Expand All @@ -58,53 +60,6 @@ Use `http://127.0.0.1:8000` as the base URL of the endpoints and navigate to `ht

## Deployment

- Install [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)

- Install Docker and start Docker.

### Staging

- Create the deployment stack (only required for the first time):

```bash
sam deploy --stack-name QuizBackendStaging --s3-bucket quiz-staging-backend --capabilities CAPABILITY_IAM -t templates/staging.yaml
```

If the deployment was successful, you should see a message as shown in the image below:
![deployment successful](images/deployment-succeeded.png)

The app will be deployed on the URL corresponding to `Value` in the image above!

- Once the stack has been deployed, subsequent changes to the code can be uploaded to the lambda function by running:

```bash
sam sync --stack-name QuizBackendStaging -t templates/staging.yaml
```

- If you want your files to automatically be synced to your deployment, simply add `--watch` at the end of the previous command.

```
sam sync --stack-name QuizBackendStaging -t templates/staging.yaml --watch
```

### Production

The steps are similar to that in Staging.

- Create the deployment stack (only required for the first time):

```
sam deploy --stack-name QuizBackendProd --s3-bucket quiz-prod-backend --capabilities CAPABILITY_IAM -t templates/prod.yaml
```
We are deploying our FastAPI instance on AWS Lambda which is triggered via an API Gateway. In order to automate the process, we are using [AWS SAM](https://www.youtube.com/watch?v=tA9IIGR6XFo&ab_channel=JavaHomeCloud), which creates the stack required for the deployment and updates it as needed with just a couple of commands and without having to do anything manually on the AWS GUI. Refer to [this](https://www.eliasbrange.dev/posts/deploy-fastapi-on-aws-part-1-lambda-api-gateway/) blog post for more details.

- Once the stack has been deployed, subsequent changes to the code can be uploaded to the lambda function by running:

```
sam sync --stack-name QuizBackendProd -t templates/prod.yaml
```

- If you want your files to automatically be synced to your deployment, simply add `--watch` at the end of the previous command.

```
sam sync --stack-name QuizBackendProd -t templates/prod.yaml --watch
```
The actual deployment happens through Github Actions. Look at [`.github/workflows/deploy_to_staging.yml`](.github/workflows/deploy_to_staging.yml) for understanding the deployment to `Staging` and [`.github/workflows/deploy_to_prod.yml`](.github/workflows/deploy_to_prod.yml) for `Production`. Make sure to set all the environment variables mentioned in [`docs/ENV.md`](docs/ENV.md) in the `Production` and `Staging` environments in your Github repository.
13 changes: 10 additions & 3 deletions app/database.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
import os
from pymongo import MongoClient

client = MongoClient(
"mongodb+srv://quiz:p$#p4h7y_Z44R-n@cluster0.uocfg.mongodb.net/quiz?retryWrites=true&w=majority"
)
# this is required for loading environment variables when
# running the app locally as the environment variable should
# be set when the app is running on staging/production by Github Actions
if "MONGO_AUTH_CREDENTIALS" not in os.environ:
from dotenv import load_dotenv

load_dotenv("../.env")

client = MongoClient(os.getenv("MONGO_AUTH_CREDENTIALS"))
1 change: 1 addition & 0 deletions app/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ pymongo==4.0.2
uvicorn==0.17.6
dnspython==2.2.1
mangum==0.14.1
python-dotenv==0.20.0
2 changes: 1 addition & 1 deletion app/schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,4 @@ class QuizLanguage(Enum):

class QuizType(Enum):
assessment = "assessment"
jee = "JEE"
homework = "homework"
7 changes: 7 additions & 0 deletions docs/ENV.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## Environment Variables

### `MONGO_AUTH_CREDENTIALS`

Credentials to connect to your MongoDB instance.

**NOTE**: While setting this value in the `Production` and `Staging` environments on Github, make sure to include the value within double quotes and escape any characters using `\` as necessary. If you don't set this correctly, your deployment will fail.
8 changes: 8 additions & 0 deletions templates/prod.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@ AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Quiz Backend - Prod - FastAPI on Lambda

Parameters:
MongoAuthCredentials:
Type: String
Description: Credentials to connect to the MongoDB instance

Resources:
Function:
Type: AWS::Serverless::Function
Expand All @@ -10,6 +15,9 @@ Resources:
CodeUri: ../app
Handler: main.handler
Runtime: python3.9
Environment:
Variables:
MONGO_AUTH_CREDENTIALS: !Ref MongoAuthCredentials
Events:
Api:
Type: HttpApi
Expand Down
8 changes: 8 additions & 0 deletions templates/staging.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@ AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Quiz Backend - Staging - FastAPI on Lambda

Parameters:
MongoAuthCredentials:
Type: String
Description: Credentials to connect to the MongoDB instance

Resources:
Function:
Type: AWS::Serverless::Function
Expand All @@ -10,6 +15,9 @@ Resources:
CodeUri: ../app
Handler: main.handler
Runtime: python3.9
Environment:
Variables:
MONGO_AUTH_CREDENTIALS: !Ref MongoAuthCredentials
Events:
Api:
Type: HttpApi
Expand Down