Skip to content

Latest commit

 

History

History
198 lines (143 loc) · 8.83 KB

README.md

File metadata and controls

198 lines (143 loc) · 8.83 KB

gitops-poc

Table of Contents

Summary

Repository deploying an AKS cluster on demand, installing ArgoCD or Flux on those clusters enabling GitOps with Helm charts or Kustomization's

Features

  • AKS deployment trough terraform cli tool and HCL files.
  • ArgoCD or Flux installations on deployed k8s cluster
  • CD wofklow for on demand deployments of an Azure Storage Account Container (For storing terraform state files)
  • CD wofklow for on demand deployments of k8s clusters (Options: AKS) and installation of GitOps tools (Options: ArgoCD or Flux) or destruction of k8s clusters trough Github workflow_dispatch trigger (Requires an Azure Storage Account Container)
  • Sample C# ASP.NET Core HelloWorld service along with a CI workflow for building and pushing the container image, including the build artifacts of the service, to an Azure Container Registry (ACR)
  • Nginx and sample-service helm charts and kustomization's required for GitOps

Getting started

Github workflows will be utilized in this. Required secrets need to be set therefore for this Github repository. Once the workflows described in the Preconditions and Deploy an AKS cluster and install the ArgoCD or FluxCD helm charts sections have been successfully executed, all resource groups listed should be visible in the Azure Portal UI:

deployed-rgs.PNG deployed-cloud-infra.PNG

Preconditions

  1. Optional: Create an ACR trough the deploy-container-registry workflow
  2. Optional: Build and push a sample service with version tag to the ACR trough the build-and-push-docker.yml workflow.
  3. Deploy an Azure Storage Account Service including containers persisting terraform state files trough the deploy-tf-backend workflow

Deploy an AKS cluster, install ArgoCD or FluxCD helm charts or setup kubernetes resources for applications

  1. Deploy an AKS trough the manage-k8s-cluster workflow.
  2. Optional: Install external helm charts into the deployed kubernetes cluster trough manage-helm-charts workflow
  3. Optional: Deploy kubernetes resources for application (secrets or reverse-proxy ingress) trough manage-internal-k8s-resources workflow

NOTE: Set all the required Github secrets for aboves workflows

Port forwarding

In order to port forward and check helm chart pods, run following commands:

kubectl get pods -A
kubectl port-forward -n <namespace>  <pod-name> <local-port>:<server-port>

When checking for example the ArgoCD Web UI, you would run:

kubectl port-forward -n external-services <argocd-server-pod> 8080:8080

and visit in a browser of choice localhost:8080. You would need to authenticate with admin credentials.

argocd-web-ui.PNG

The default username is admin. The default password can be obtained trough: kubectl -n argocd get secret argocd-initial-admin-secret -n external-services -o jsonpath="{.data.password}" | base64 -d

Showcase GitOps

ArgoCD applications

In order to register and sync ArogCD applications run:

# Port forward in terminal process A
kubectl port-forward -n external-services <argocd-server-pod> 8080:8080

# In terminal process B - Login
argocd login localhost:8080
# Prompted to provide username and password

# In terminal process B - Register Application
argocd app create nginx \
  --repo <GIT_REPO_URL> \
  --path <PATH_IN_REPO> \
  --dest-server <K8S_CLUSTER_URL> \
  --dest-namespace <NAMESPACE> \
  --revision <TARGET_REVISION> \
  --server <ARGOCD_SERVER>

# e.g. for nginx chart
argocd app create nginx \
  --repo https://github.com/MGTheTrain/gitops-poc.git \
  --path gitops/argocd/nginx \ 
  --dest-server https://kubernetes.default.svc \
  --dest-namespace internal-apps \
  --revision main \
  --server localhost:8080

# In terminal process B - Sync Application
argocd app sync nginx
# In terminal process B - Monitor Application Status
argocd app get nginx

# e.g. sample service
argocd app create sample-service \
  --repo https://github.com/MGTheTrain/gitops-poc.git \
  --path gitops/argocd/sample-service \ 
  --dest-server https://kubernetes.default.svc \
  --dest-namespace internal-apps \
  --revision main \
  --server localhost:8080

# In terminal process B - Sync Application
argocd app sync sample-service
# In terminal process B - Monitor Application Status
argocd app get sample-service

# If an error appears in the ArgoCD Web UI while pulling Docker images for the sample service try manually deleting and then recreating the Docker secret
kubectl delete secret acr-secret -n internal-apps
kubectl create secret docker-registry acr-secret --docker-server=<> --docker-username=<> --docker-password=<> -n internal-apps

# Some relatable links to the issue:
# - https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret#example-usage-docker-config
# - https://stackoverflow.com/questions/41203970/pull-image-azure-container-registry-kubernetes

# Check if the internal services could be created properly 
kubectl get svc -n internal-apps
# Additionally, verify the public IP address of the nginx-controller and access the default nginx view using a preferred web browser by navigating to http://<public IP>.

The ArgoCD applications that have been registered and synchronized should resemble the following:

argocd-applications.jpg

sample-service-argocd-app.jpg

To obtain the external IP address of the Nginx controller's load balancer, run:

load-balancer-ip.jpg

Checkout nginx-app route in a preferred browser

nginx-app-route.jpg

and sample-service-app route in a preferred browser

sample-service-app-route.jpg

FluxCD Kustomizations

In order to deploy FluxCD Kustomizations run:

cd gitops/fluxcd/nginx/overlays/dev
kubectl apply -f kustomization.yaml

# See the source status
kubectl get gitrepositories -n internal-apps
# IMPORTANT - See the reconciliation status
kubectl get kustomizations -n internal-apps
# Describe customization
kubectl describe kustomization nginx -n internal-apps

# Check if the nginx service could be created properly 
kubectl get svc -n internal-apps
# Additionally, verify the public IP address of the nginx-controller and access the default nginx view using a preferred web browser by navigating to http://<public IP>.

or utilize flux cli tool (NOTE: Need to be further checked):

flux create kustomization nginx \
--source=<GIT_REPO_URL> \
--path="<PATH_IN_REPO>" \
--prune=true \
--interval=5m \
--wait=true \
--health-check-timeout=3m \
--namespace=<NAMESPACE>

flux create kustomization nginx \
--source=https://github.com/MGTheTrain/gitops-poc.git/nginx \
--path="./gitops/fluxcd/nginx/overlays/dev" \
--prune=true \
--interval=5m \
--wait=true \
--health-check-timeout=3m \
--namespace=gitops

# Check if the nginx service could be created properly 
kubectl get svc -n internal-apps
# Additionally, verify the public IP address of the nginx-controller and access the default nginx view using a preferred web browser by navigating to http://<public IP>.

Registered ArgoCD applications or FluxCD Kustomizations will be treated as the sole source of truth within the Kubernetes cluster for the latest commit on this Github repositories main branch, with the reconciliation loop ensuring that the desired state described within Kubernetes manifests matches the cluster's actual state through Kubernetes controllers.

Destroy the AKS cluster, uninstall helm charts or remove kubernetes resources for applications

  1. Optional: Uninstall external helm charts of the deployed kubernetes cluster trough manage-helm-charts workflow
  2. Optional: Destroy kubernetes resources for application (secrets or reverse-proxy ingress) trough manage-internal-k8s-resources workflow
  3. Destroy an AKS trough the manage-k8s-cluster workflow