aks-installation.md

Deploying Nephe in Azure AKS

Table of Contents

• Prerequisites
• Create an AKS cluster via terraform
  • Setup Terraform Environment
  • Create an AKS cluster
  • Deploy Nephe Controller
  • Interact with AKS cluster
  • Display AKS attributes
  • Destroy AKS cluster
• Create Azure VMs
  • Setup Terraform Environment
  • Create VMs
  • Display VNET attributes
  • Destroy VMs

Prerequisites

1. Install Kubectl v1.24+.

2. Install Terraform. Recommend v1.2.2.

3. Install jq and pv.

4. Create or obtain an azure service principal and set the below environment variables. Please refer to Azure documentation for more information.

   export TF_VAR_azure_client_id=<YOUR_SERVICE_PRINCIPAL_ID>
   export TF_VAR_azure_client_secret=<YOUR_SERVICE_PRINCIPAL_SECRET>
   export TF_VAR_azure_client_subscription_id=<YOUR_SUBCRIPTION_ID>
   export TF_VAR_azure_client_tenant_id=<YOUR_TENANT_ID>
   export TF_VAR_owner=<YOUR_NAME>

   Note: TF_VAR_owner may be set so that you can identify your own cloud resources. It should be one word, with no spaces and in lower case.

Create an AKS cluster via terraform

Setup Terraform Environment

./hack/install-cloud-tools.sh

The install cloud tools script copies the required bash and terraform scripts to the user home directory, under ~/terraform/.

Create an AKS cluster

Create an AKS cluster using the provided terraform scripts. Once the AKS cluster is created, worker nodes are accessible via their external IP using ssh. Terraform state files and other runtime info will be stored under ~/tmp/terraform-aks/. You can also create an AKS cluster in other ways and deploy prerequisites manually.

This also deploys cert-manager v1.8.2 and antrea v1.10.

~/terraform/aks create

Deploy Nephe Controller

To deploy a released version of Nephe, pick a deployment manifest from the list of releases. For any given release (e.g. v0.1.0), you can deploy Nephe as follows:

kubectl apply -f https://github.com/antrea-io/nephe/releases/download/<TAG>/nephe.yml

To deploy the latest version of Nephe (built from the main branch), use the checked-in deployment yaml:

~/terraform/aks kubectl apply -f https://raw.githubusercontent.com/antrea-io/nephe/main/config/nephe.yml

Interact with AKS cluster

Issue kubectl commands to AKS cluster using the helper scripts. To run kubectl commands directly, set KUBECONFIG environment variable.

~/terraform/aks kubectl ...
export KUBECONFIG=~/tmp/terraform-aks/kubeconfig

Loading locally built antrea/nephe image to AKS cluster.

docker tag antrea/nephe:latest projects.registry.vmware.com/antrea/nephe:latest
~/terraform/aks load projects.registry.vmware.com/antrea/nephe

Display AKS attributes

~/terraform/aks output

Destroy AKS cluster

~/terraform/aks destroy

Create Azure VMs

Additionally, you can also create compute VNET with 3 VMs using terraform scripts for testing purpose. Each VM will have a public IP and an Apache Tomcat server deployed on port 80. Use curl <PUBLIC_IP>:80 to access a sample web page. Create or obtain Azure Service Principal credential and configure the below environment variables, see Prerequisites section for more details.

export TF_VAR_azure_client_id=<YOUR_SERVICE_PRINCIPAL_ID>
export TF_VAR_azure_client_secret=<YOUR_SERVICE_PRINCIPAL_SECRET>
export TF_VAR_azure_client_subscription_id=<YOUR_SUBCRIPTION_ID>
export TF_VAR_azure_client_tenant_id=<YOUR_TENANT_ID>
export TF_VAR_owner=<YOUR_NAME>

To create VMs with antrea agent deployed, additionally set the following variables. For more information on how to generate the kubeconfig files, please refer to antrea ExternalNode documentation.

Note: Terraform working directory is different from current working directory, please use absolute file path to avoid confusion. The install wrapper scripts is located under hack/.

export TF_VAR_agent=true
export TF_VAR_antrea_agent_k8s_config=<PATH_TO_K8S_APISERVER_KUBECONFIG>
export TF_VAR_antrea_agent_antrea_config=<PATH_TO_ANTREA_APISERVER_KUBECONFIG>
export TF_VAR_install_vm_agent_wrapper=<PATH_TO_INSTALL_VM_AGENT_WRAPPER_SCRIPT>

Setup Terraform Environment

./hack/install-cloud-tools.sh

Create VMs

~/terraform/azure-tf create

Terraform state files and other runtime info will be stored under ~/tmp/terraform-azure/.

Display VNET attributes

~/terraform/azure-tf output

Destroy VMs

~/terraform/azure-tf destroy