Online Shopping Application: This online shopping application was architected and built using cloud-first related principles and methodologies that promotes the adoption of application management strategies such as Microservices. The Online Shopping Application consists of about 11 microservices.
- Create a GitHub Repository with the name
multi-microservices-application-project
and push the code in this branch (main) to your remote repository (your newly created repository).- Go to GitHub: https://github.com
- Login to
Your GitHub Account
- Create a Repository called
multi-microservices-application-project
- Clone the Repository in the
Repository
directory/folder on yourlocal machine
- Download the code in in this repository
"multi-microservices-application-project main branch"
: https://github.com/awanmbandi/realworld-microservice-project.git Unzip
thecode/zipped file
Copy
andPaste
everythingfrom the zipped file
into therepository you cloned
in your local- Open your
Terminal
- Add the code to git, commit and push it to your upstream branch "main or master"
- Add the changes:
git add -A
- Commit changes:
git commit -m "adding project source code"
- Push to GitHub:
git push
- Confirm that the code is now available on GitHub
- Download the code from ALL the other BRANCHES as well
- Create the following branches in the Repository, you just created above
app-ad-serverice
app-cart-service
app-checkout-service
app-currency-service
app-database
app-email-service
app-frontend-service
app-loadgenerator-service
app-payment-service
app-product-catalog-service
app-recommendation-service
app-shipping-service
- Download the Source Code of each microservice, from their respective branches from this Repository https://github.com/awanmbandi/realworld-microservice-project.git
- And Push the Code based on the Microservice to the specific Branch you Created for that Service.
- Create An IAM Profile/Role For The
Jenkins-CI
Server
- Create an EC2 Service Role in IAM with AdministratorAccess Privilege
- Navigate to IAM
- Click on
Roles
- Click on
Create Role
- Select
Service Role
- Use Case: Select
EC2
- Click on
Next
- Attach Policy:
AdministratorAccess
- Click
Next
- Role Name:
AWS-EC2-Administrator-Role
- Click
Create
- Click on
- Jenkins CI
- Region:
Ohio - us-east-2
- Create a Jenkins VM instance
- Name:
Jenkins-CI
- AMI:
Ubuntu 22.04
- Instance type:
t2.large
- Key pair:
Select
orcreate a new keypair
- Security Group (Edit/Open):
All Traffic
to0.0.0.0/0
- Name & Description:
Multi-Microservices-Jenkins-CI-SG
- What we actually need:
8080
,9000
and22
to0.0.0.0/0
- Name & Description:
- Storage: Increase to
50 GB
- IAM instance profile: Select the
AWS-EC2-Administrator-Role
- User data (Copy the following user data): https://github.com/awanmbandi/realworld-microservice-project/blob/main/installtions/installations.sh
- Launch Instance
- Region:
- ONLY VISIT THIS SECTION IF YOU STOPPED AND RESTARTED YOUR JENKINS SERVER
- The above
Jenkins Userdata
includes aSonarQube
container deployment task- As a result, we know containers are
Ephemeral
by natuure, so if youStop
yourJenkins CI Server
at any point in time... You'll have toDeploy the Container
again when youStart
it back or bring the instance up again. - If you don't do this, you will not be able able to proceed with the project.
- I have also Included a
Docker Volume
setup task as well forSonarQube
, where the Container Data will be persisted to avoid Data lost.
- As a result, we know containers are
# Volume inspection, confirm the docker volume exist
docker volume inspect volume sonarqube-volume
# Create a new conainter, provide your container name and deploy in the `Jenkins-CI` server
docker run -d --name PROVIDE_NAME_HERE -v sonarqube-volume:/opt/sonarqube/data -p 9000:9000 sonarqube:lts-community
- Slack
- Go to the bellow Workspace and create a Private Slack Channel and name it "yourfirstname-jenkins-cicd-pipeline-alerts"
- Link: https://join.slack.com/t/jjtechtowerba-zuj7343/shared_invite/zt-24mgawshy-EhixQsRyVuCo8UD~AbhQYQ
- You can either join through the browser or your local Slack App
- Create a
Private Channel
using the naming conventionYOUR_INITIAL--multi-microservices-alerts
- NOTE:
(The Channel Name Must Be Unique, meaning it must be available for use)
- NOTE:
- Visibility: Select
Private
- Click on the
Channel Drop Down
and selectIntegrations
and Click onAdd an App
- Search for
Jenkins
and Click onView
- Click on
Configuration/Install
and ClickAdd to Slack
- On Post to Channel: Click the Drop Down and select your channel above
YOUR_INITIAL-multi-microservices-alerts
- Click
Add Jenkins CI Integration
- Scrol Down and Click
SAVE SETTINGS/CONFIGURATIONS
- Leave this page open
- SSH into the
Jenkins-CI
server- Run the following commands and confirm that the
services
are allRunning
- Run the following commands and confirm that the
# Confirm Java version
sudo java --version
# Confirm that Jenkins is running
sudo systemctl status jenkins
# Confirm that docker is running
sudo systemctl status docker
# Confirm that Terraform is running
terraform version
# Confirm that the Kubectl utility is running
kubectl version --client
# Confirm that AWS CLI is running
aws --version
# Confirm that the SonarQube container is running
docker ps | grep sonarqube:lts-community
# Lastly confirm that the `sonarqube-volume docker volume` was created
docker volume inspect volume sonarqube-volume
- Navigate to the
IAM Servce
- Click on
Roles
- Use the
Search Bar
to file roles that starts witheks
- Delete any Role with the name
eks
- Click on
UPDATE
Your Terraform Provider Region toYour Choice REGION
*⚠️ NOTE:ALERT!
⚠️ : Do Not Use North Virginia, that's US-EAST-1⚠️ NOTE:ALERT!
⚠️ : Also Confirm that The Selected Region Has ADefault VPC
You're Confident Has Internet Connection⚠️ NOTE:ALERT!
⚠️ : The Default Terraform Provider Region Defined In The Config IsOhio(US-EAST-2)
- Confirm you're still logged into the
Jenkins-CI
Server viaSSH
- Run the following commands to deploy the
EKS Cluster
in theJenkins-CI
- NOTE: You Can As Well Deploy The Cluster Using Terraform From Your Local System
# Clone your project reporisoty
git clone https://github.com/awanmbandi/realworld-microservice-project.git
# cd and checkout into the DevSecOps project branch
cd realworld-microservice-project
cd terraform/AWS/eks-cluster
# Deploy EKS Environment
terraform init
terraform plan
terraform apply --auto-approve
- Give it about
10 MINUTES
for the cluster creation to complete - Then
Duplicate or Open
a New ConsoleTab
andSwitch
to theOhio(us-east-2) Region
- Navigate to
EKS
and confirm that your Cluster was created successfully with the nameEKS_Cluster
- Also confirm there's no issue regarding your Terraform execution
-
If the Error Message says anything about
EKS IAM Roles
then... -
Destroy everything by running:
terraform destroy --auto-approve
-
Wait for everything to get destroy/terminated successfully.
-
Then Navigate to
IAM
- In the
Search section
- Search for the word
EKS
and select ALL the EKS Role that shows up - Delete every one of them
- In the
-
Go back to where you're executing Terraform(that's the Jenkins Instance)
- Re-run:
terraform apply --auto-approve
- Wait for another
10 Minute
- Re-run:
- Run this command from the
Jenkins-CI
instance
eksctl utils associate-iam-oidc-provider \
--region us-east-2 \
--cluster EKS_Cluster \
--approve
- Run this command from the
Jenkins-CI
instance
aws eks update-kubeconfig --name <clustername> --region <region>
- Run this command from the
Jenkins-CI
instance
kubectl create ns test-env
kubectl create ns prod-env
kubectl get ns
- Navigate to
EC2
-
Copy your Jenkins Public IP Address and paste on the browser = ExternalIP:8080
-
- Click on
Manage Jenkins
- Click on
Plugins
- Click
Available
- Search and Install the following Plugings and
"Install"
- SonarQube Scanner
- Snyk
- Multibranch Scan Webhook Trigger
- Eclipse Temurin installer
- Pipeline: Stage View
- Docker
- Docker Commons
- Docker Pipeline
- docker-build-step
- Docker API
- Kubernetes
- Kubernetes CLI
- Kubernetes Credentials
- Kubernetes Client API
- Kubernetes Credentials Provider
- Kubernetes :: Pipeline :: DevOps Steps
- Slack Notification
- ssh-agent
- BlueOcean
- Build Timestamp
- Click on
Install
- Once all plugins are installed
- Select/Check the Box
Restart Jenkins when installation is complete and no jobs are running
- Refresh your Browser and Log back into Jenkins
- Once you log back into Jenkins
- Click on
-
-
JDK
- Click on
Add JDK
-->> Make sure Install automatically is enabled
Note: By default the Install Oracle Java SE Development Kit from the website make sure to close that option by clicking on the image as shown below.
- Name:
JDK17
- Click on
Add installer
- Select
Install from adoptium.net
- Version:
jdk-17.0.8.1+1
- Click on
-
Gradle Installation
-
SonarQube Scanner
-
Snyk Installations
-
Docker installations
-
-
Click on
Manage Jenkins
- Click on
Credentials
- Click on
Jenkins - System
- Click on
Global Credentials (Unrestricted)
- Click on
Add Credentials
-
-
-
Login to your SonarQube Application (http://SonarServer-Sublic-IP:9000)
- Default username:
admin
- Default password:
admin
- Default username:
-
Click on
Login
-
Old Password:
admin
-
New Password:
adminadmin
-
Confirm Password:
adminadmin
-
Click on
Manually
(Create theapp-shipping-service
microservice test project)- Project display name:
app-shipping-service
- Display key:
app-shipping-service
- Main branch name:
app-shipping-service
- Project display name:
-
Click on
Projects
(Create theapp-recommendation-service
microservice test project)- Project display name:
app-recommendation-service
- Display key:
app-recommendation-service
- Main branch name:
app-recommendation-service
- Project display name:
-
Click on
Projects
(Create theapp-product-catalog-service
microservice test project)- Project display name:
app-product-catalog-service
- Display key:
app-product-catalog-service
- Main branch name:
app-product-catalog-service
- Project display name:
-
Click on
Projects
(Create theapp-payment-service
microservice test project)- Project display name:
app-payment-service
- Display key:
app-payment-service
- Main branch name:
app-payment-service
- Project display name:
-
Click on
Projects
(Create theapp-loadgenerator-service
microservice test project)- Project display name:
app-loadgenerator-service
- Display key:
app-loadgenerator-service
- Main branch name:
app-loadgenerator-service
- Project display name:
-
Click on
Projects
(Create theapp-frontend-service
microservice test project)- Project display name:
app-frontend-service
- Display key:
app-frontend-service
- Main branch name:
app-frontend-service
- Project display name:
-
Click on
Projects
(Create theapp-email-service
microservice test project)- Project display name:
app-email-service
- Display key:
app-email-service
- Main branch name:
app-email-service
- Project display name:
-
Click on
Projects
(Create theapp-database
microservice test project)- Project display name:
app-database
- Display key:
app-database
- Main branch name:
app-database
- Project display name:
-
Click on
Projects
(Create theapp-currency-service
microservice test project)- Project display name:
app-currency-service
- Display key:
app-currency-service
- Main branch name:
app-currency-service
- Project display name:
-
Click on
Projects
(Create theapp-checkout-service
microservice test project)- Project display name:
app-checkout-service
- Display key:
app-checkout-service
- Main branch name:
app-checkout-service
- Project display name:
-
Click on
Projects
(Create theapp-cart-service
microservice test project)- Project display name:
app-cart-service
- Display key:
app-cart-service
- Main branch name:
app-cart-service
- Project display name:
-
Click on
Projects
(Create theapp-ad-serverice
microservice test project)- Project display name:
app-ad-serverice
- Display key:
app-ad-serverice
- Main branch name:
app-ad-serverice
- Project display name:
-
Click on
Set Up
-
-
Generate a
Global Analysis Token
This is the Token you need for Authorization- Click on the
User Profile
icon at top right of SonarQube - Click on
My Account
- Click
Security
Generate Token:
Generate this TOKEN and Use in the Next Step to Create The SonarQube Credential- Click on
GENERATE
- NOTE:
Save The Token Somewhere...
- Click on the
-
-
- Navigate back to Jenkins http://JENKINS_PUBLIC_IP:8080
- Click on
Manage Jenkins
- Click on
Jenkins System
- Click
Global credentials (unrestricted)
- Click on
- Click on
Add Credentials
- Kind:
Secret text
- Secret:
Paste the SonarQube TOKEN
value that we have created on the SonarQube server - ID:
SonarQube-Credential
- Description:
SonarQube-Credential
- Click on
Create
-
-
-
- Slack: https://join.slack.com/t/jjtechtowerba-zuj7343/shared_invite/zt-24mgawshy-EhixQsRyVuCo8UD~AbhQYQ
- Navigate to the Slack "Channel you created":
YOUR_INITIAL-devsecops-cicd-alerts
- Click on your
Channel Drop Down
- Click on
Integrations
and Click onAdd an App
- Click on
Jenkins CI VIEW
and Click onConfiguration
- Click on
Add to Slack
, Click on the Drop Down andSelect your Channel
- Click on
Add Jenkins CI Integration
NOTE:
The TOKEN is on Step 3
-
- Click on
Add Credentials
- Click on
Jenkins System
- Click
Global credentials (unrestricted)
- Click on
- Kind: Secret text
- Secret: Place the Integration Token Credential ID (Note: Generate for slack setup)
- ID:
Slack-Credential
- Description:
Slack-Credential
- Click on
Create
- Click on
-
-
-
- Access DockerHub at: https://hub.docker.com/
- Provide Username:
YOUR USERNAME
- Provide Username:
YOUR PASSWORD
- Click on
Sign In
orSign Up
- NOTE: If you have an account
Sign in
If notSign up
- NOTE: If you have an account
-
- Click on
Add Credentials
- Click on
Jenkins System
- Click
Global credentials (unrestricted)
- Click on
- Kind: Username with password
- Username:
YOUR USERNAME
- Password:
YOUR PASSWORD
- ID:
DockerHub-Credential
- Description:
DockerHub-Credential
- Click on
Create
- Click on
-
-
SSH
back into yourJenkins-CI
server- RUN the command:
aws eks update-kubeconfig --name <clustername> --region <region>
- COPY the Cluster KubeConfig:
cat ~/.kube/config
COPY
the KubeConfig file content- You can use your
Notepad
or any otherText Editor
as well - Open your Local
GitBash
orTerminal
- Create a File Locally
- RUN:
rm ~/Downloads/kubeconfig-secret.txt
- RUN:
touch ~/Downloads/kubeconfig-secret.txt
- RUN:
vi ~/Downloads/kubeconfig-secret.txt
PASTE
andSAVE
the KubeConfig content in the file
- You can use your
-
- Navigate back to Jenkins
- Click on
Add Credentials
- Click on
Jenkins System
- Click
Global credentials (unrestricted)
- Click on
- Kind:
Secret File
- File: Click
Choose File
- NOTE: Seletct the KubeConfig file you saved locally
- ID:
Kubernetes-Credential
- Description:
Kubernetes-Credential
- Click on
Create
-
-
- Open your
GitBash Terminal
orMacOS Terminal
- Navigate to the Location where your
Jenkins-CI
Server SSH Key is Stored (Usually in Downloads) - Run the Command
cat /Your_Key_PATH/YOUR_SSH_KEY_FILE_NAME.pem
Note:
Your.pem
private key will most like be inDownloads
- COPY the KEY content and Navigate back to Jenkins to store it...
- Open your
-
- Navigate to the
Jenkins Global Credential Dash
- Click on
Create Credentials
- Scope: Select
Global......
- Type: Select
SSH Username with Private Key
- ID and Description:
OWASP-Zap-Credential
- Username:
ubuntu
- Private key: Select
- Key: Click on
Add
- Key:
Paste The Private Key Content You Copied
- Key: Click on
- Click on
Create
- Navigate to the
-
-
-
Navigate to: https://snyk.com/
- Click on
Sign Up
- Select
GitHub
- Once you're login to your Snyk account
- Click on
Your Name
belowHelp
on the Botton left hand side of your Snyk Account - Click on
Account Settings
- Auth Token (KEY): Click on
Click To Show
- COPY the TOKEN and SAVE somewhere
- Click on
-
- Click on
-
-
-
- Still on
Manage Jenkins
andConfigure System
- Scroll down to the
Slack
Section (at the very bottom) - Go to section
Slack
NOTE:
Make sure you still have the Slack Page that has theteam subdomain
&integration token
open- Workspace: Provide the
Team Subdomain
value (created above) - Credentials: select the
Slack-Credential
credentials (created above) - Default channel / member id:
#PROVIDE_YOUR_CHANNEL_NAME_HERE
- Click on
Test Connection
- Click on
Apply
andSave
- Still on
-
UPDATE YOUR
Jenkinsfiles...
-
Update the
Repository URL
in theCheckout To Mcroservice Branch STAGE
on all Microservices JenkinsfilesLine 19
-
Update your
Frontend Service
-OWASP Zap Server IP
andEKS Worker Node IP
in theJenkinsfile
onLine 86
NOTE
to update theFrontend Service
, you mustSwitch
to theFrontend Branch
-
Update the
EKS Worker Node IP
with yours in theJenkinsfile
onLine 86
-
Update your
Slack Channel Name
in theJenkinsfiles...
-All Microservices
-
Update
SonarQube projectName
of your Microservices in theJenkinsfiles...
-All Microservices
-
Update the
SonarQube projectKey
of your Microservices in theJenkinsfiles...
-All Microservices
-
Update the
DockerHub username
of your Microservices in theJenkinsfiles...
-All Microservices
, provide Yours -
Update the
DockerHub username/Image name
in all thedeployment.yaml
files for the different environmentstest-env
andprod-env
folders acrossEvery Single Microservice Branch
-
Log into Jenkins: http://Jenkins-Public-IP:8080/
-
Click on
New Item
-
Enter an item name:
Online-Shop-Microservices-CICD-Automation
-
Select the category as
Multibranch Pipeline
-
Click
OK
-
BRANCH SOURCES:
- Git:
- Project Repository
- Repository URL:
Provide Your Project Repo Git URL
(the one you created at the beginning)
- Repository URL:
- Project Repository
- Git:
-
BEHAVIORS
- Set it to:
Discover Branches
and - Click
Add
- Select:
Filter by name (with wildcards)
- Include:
app-*
- Select:
- Set it to:
-
Property strategy:
All branches get the same properties
-
BUILD CONFIGURATION
- Mode: Select
by Jenkinsfile
- Script Path:
Jenkinsfile
- Mode: Select
-
SCAN MULTIBRANCH PIPELINE TRIGGER
- Select
Scan by webhook
- Trigger token:
automation
- Select
-
Click on
Apply
andSave
-
CONFIGURE MULTIBRANCH PIPELINE WEBHOOK
- Copy this URL and Update the Jenkins IP (to yours):
http://PROVIDE_YOUR_JENKINS_IP:8080/multibranch-webhook-trigger/invoke?token=automation
- Navigate to your
Project Repository
- Click on
Settings
in the Repository - Click on
Webhooks
- Click on
Add Webhook
- Payload URL:
http://PROVIDE_YOUR_JENKINS_IP:8080/multibranch-webhook-trigger/invoke?token=automation
- Content type:
application/json
- Which events would you like to trigger this webhook: Select
Just the push event
- Enable
Active
- Click
ADD WEBHOOK
- Click on
- Copy this URL and Update the Jenkins IP (to yours):
-
Navigate Back To Jenkins and Confirm That All 12 Pipeline Jobs Are Running (11 Microservices Jobs and 1 DB Job)
- To perform the DEPLOYMENT in the staging Envrionment
- You Just Have To
UNCOMMENT
theDEPLOY STAGE
in theJenkinsfiles.....
andPUSH
to GitHub - DEPLOY the Microservices in the STAGING Environment in the following ORDER (To Resolve DEPENDENCIES around the SERVICES)
Redis DB
Product Catalog Service
Email Service
Currency Service
Payment Service
Shipping Service
Cart Service
Ad Service
Recommendation Service
Checkout Service
Frontend
Load Generator
-
SSH Back into your
Jenkins-CI
Server -
Access The Application Running in the
Test Environment
within the Cluster -
Update
the EKS Cluster Security Group (If you've not already)- To do this, navigate to
EC2
- Select one of the
Worker Nodes
--> Click onSecurity
--> Click onThe Security Group ID
- Click on
Edit Inbound Rules
: Port =30000
and Source0.0.0.0/0
- To do this, navigate to
-
Open your Browser
- To perform the DEPLOYMENT to the Prod Envrionment
- You Just Have To
UNCOMMENT
theDEPLOY STAGE
in theJenkinsfiles.....
andPUSH
to GitHub - DEPLOY the Microservices to the Prod Environment in the following ORDER (To Resolve DEPENDENCIES around the SERVICES)
Redis DB
Product Catalog Service
Email Service
Currency Service
Payment Service
Shipping Service
Cart Service
Ad Service
Recommendation Service
Checkout Service
Frontend
Load Generator
Home Page | Checkout Page |
---|---|