-
Notifications
You must be signed in to change notification settings - Fork 6
The Infrastructure
Infrastructrue and CI/CD pipeline as of Sprint 1
ℹ️ "Progphil Bot" or "Bot" will be referred to ProgPhil Discord Bot itself.
ℹ️ "Server" will refer to "remote server" or "hosted server", not "discord server".
We follow Trunk Based Development as our git branching model. It eliminates long-lived branches and the possibility of having merging conflicts by committing only to one single branch or 'trunk'. Therefore all the development, testing, and bug fixing only happen to the main branch of the repo. Developers doesn't develop in the release branch, each release branch only contains the specific version of the repo and only deleted when it is out-of-support. In the case of bug-fixes, the development and the fixing of the bugs only happen in the main, then it is cherry-picked to release branch as a sub-version of initial release in that branch i.e. 1.2.0 -> 1.2.1.
---
title: ProgPhil Discord Bot Diagram
---
gitGraph
commit id: "init"
commit id: "a"
commit id: "b"
branch release/2023-1
commit id: "release" tag: "v1.2.0"
checkout main
commit id: "bug fix"
commit id: "d"
checkout release/2023-1
cherry-pick id: "bug fix" tag: "v1.2.1"
checkout main
commit id: "e"
ℹ️ release commit is just a representation of release action that generates a release upon create of release/xxxx-* branches, not an actual commit in the branch.
ℹ️ Only cherry-pick commits in the release/xxxx-* branches are allowed, and these commits will be subversions of original commits.
Docker is a popular software used in creating and building containerized applications. Why do we need to containerize our application? Docker allows us to package our app with its required libs and dependencies inside the docker container to make our deployment into the server more efficient, taking up less space and time. It will be easier for the ops team to deliver changes in the codebase and dependencies to the serving by rebuilding the image of the new app and restarting the running container deployed in the server.
To run Docker in your local machine you must have installed it. You can use other frontend for Docker Management like Portainer if you want to.
Please refer to these link for download and installation:
The Operations team will be utilizing the docker container in maintaining, stabilizing, and scaling the bot.
The Dockerfile is like a blueprint, where Docker can read instructions on how to build a Docker image. It includes instructions which operating system will be used, what programming language, dependencies, or libraries are need to be installed to run the ProgPhil Bot, and other specific operations on how we will manage our images, containers, and the bot inside it.
Dockerignore on other hand functions like gitignore, which excludes files and directories to ignore in the code when building the image.
This is the current Dockerfile: Dockerfile
First, we will use python:3.10-slim, an existing image in the dockerhub where we will build our bot.
FROM python:3.10-slim
ARG are build-time variables, this is useful if you want to pass discord-token as ARG in which will be converted into ENV during build-time. ENV are variables available both during build-time and run-time. Remember, passing discord-token as ARG is completely optional.
ARG token
ENV token=$token
This are the essential ENV configurations for python and poetry to work.
PYTHONUNBUFFERED=1 \
PYTHONFAULTHANDLER=1 \
PYTHONHASHSEED=random \
PIP_NO_CACHE_DIR=off \
PIP_DISABLE_PIP_VERSION_CHECK=on \
PIP_DEFAULT_TIMEOUT=100 \
POETRY_VERSION=1.3.2
This part will install poetry using pip and adds it to PATH.
RUN pip install -U pip \
&& apt-get update \
&& pip install "poetry==1.3.2"
ENV PATH="${PATH}:/root/.poetry/bin"
Now, it will install the bot's required dependencies and libraries.
# Install Poetry dependencies
WORKDIR /progphil-bot
COPY poetry.lock pyproject.toml ./
RUN poetry config virtualenvs.create false \
&& poetry install --no-dev
The remaining commands include putting the source code on /progphil-bot directory, and the CMD whose function is to run the bot.
Why not Docker Compose?
- Due to the current nature of the bot as a standalone container app, we don't need to configure it using a docker compose. Docker compose will be utilized when we evolved our bot that will use multiple containers and needs to be orchestrate.
docker build -t <image-name> .
-t optional, used to tag image in this form: :.
--build-args optional, here we pass our build-time variable e.g. token.
We can run docker build to build image from our dockerfile. The same command is also used when we want to update our image based on our current repository.
docker run --env token=discord_token_value -it -d --name <container-name> <image-name>
--env required for discord bot to run. We used to pass the discord token to our code through .env file during runtime.
-d required. Run the container in detached mode.
--name optional. Name the container.
For the bot to actually run, we have to build and start our container using docker run. This command alone will build and run the container from our images. Inside the container is where our code for the bot runs and it is isolated.
docker stop <previously-running-container>
docker rm <previously-running-container>
docker build -t <new-image-name> .
docker run --env token=discord_token_value -it -d --name <container-name> <image-name>
docker image prune -f
To update our image and the container, we must stop first the currently running docker container before rebuilding using docker stop. Then, we remove it using docker rm in order for us to replace it we the new container built with new image. We will use docker run to build and run the new container from the updated image we just built using docker build. Lastly, we remove any unused image or container.
The current CI/CD Pipeline
flowchart TB
codechanges(Code changes) --> | latest tested commit | releasebr(create release branch)
releasebr --> | triggers auto-release | release(releases new version)
release --> ssh(connects to server)
ssh --> | if prod | dockerprod(create and updates prod container)
dockerprod --> dcserv(updates discord bot in the ProgPhil discord server)
codechanges --> | on pr on main | pylint(triggers pylint checks)
codechanges --> | on push on main | pylint
pylint --> | triggers Docker CICD | ssh
ssh --> | if dev | dockerdev(create and updates dev container)
dockerdev --> dctest(updates discord bot in the discord testing server)
dctest --> testing(Testing and Quality Assurance)
testing --> | commits on main/trunk | codechanges
As of sprint 1, we, the operations is using the Github Actions in delivering the CI/CD Pipeline. We have deployed 2 initial automations that will run in parallel.
First, is the Pylint Checks. The purpose of the Pylint Checks is to scan and check all pushed commits if there's any syntax errors or PEP8 python coding convention violation.
The second one Docker CICD handles both the continuous integration and continuous deployment of the repository. In the deployment of the bot, we used git to pull latest changes from our repository and we did the updating images, and building and running the dev container.
Lastly, we have the release automation which only triggered on push on release branches with the pattern similar to release/2023-1 or release/2024-12. Upon triggering this action it will create a release conforming into semver versioning strategy, and deploys it to the remote server as production inside the production container. The release automation utilizes google-github-actions/release-please-actionv3 in which it automatically create Changelog.md and in order for it track changes in the repo, Every commit must follow conventional-commits commit convention.