Skip to content

Commit

Permalink
Improve replacing EKS-A Cilium guide
Browse files Browse the repository at this point in the history
  • Loading branch information
@chrisdoherty4
chrisdoherty4 committed Apr 28, 2023
1 parent b09536d commit 7f9137b
Show file tree
Hide file tree
Showing 2 changed files with 89 additions and 52 deletions.
1 change: 1 addition & 0 deletions docs/content/en/docs/reference/clusterspec/optional/cni.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,7 @@ spec:

The [Cilium CLI](https://github.com/cilium/cilium-cli) can be used to uninstall EKS Anywhere Cilium
via `cilium uninstall`.
See the [replacing Cilium task]({{< ref "/docs/tasks/cluster/cluster-replace-cilium" >}}) for a walkthrough on how to successfully replace EKS Anywhere Cilium.

{{% alert title="Warning" color="warning" %}}
When uninstalling EKS Anywhere Cilium, nodes will become unhealthy. If nodes are left without a CNI
Expand Down
140 changes: 88 additions & 52 deletions docs/content/en/docs/tasks/cluster/cluster-replace-cilium.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,70 +6,96 @@ description: >
Replace EKS Anywhere Cilium with a custom CNI
---

This page provides walkthroughs on replacing the EKS Anywhere Cilium with a custom CNI.
For more information on CNI customization see [Use a custom CNI]({{< ref "/docs/reference/clusterspec/optional/cni#use-a-custom-cni" >}}).

## Prerequisites

* EKS Anywhere v0.15+.
* A recent version of [Cilium CLI](https://github.com/cilium/cilium-cli).

## Add a custom CNI to a new cluster

If an operator intends to uninstall EKS Anywhere Cilium from a new cluster they can enable the `skipUpgrade`
option when creating the cluster.
If an operator intends to uninstall EKS Anywhere Cilium from a new cluster they can enable the `skipUpgrade` option when creating the cluster.
Any future upgrades to the newly created cluster will not have EKS Anywhere Cilium upgraded.

1. Generate a cluster configuration according to the [Getting Started]({{< ref "/docs/getting-started" >}}) section.

2. Modify the `Cluster` object's `spec.clusterNetwork.cniConfig.cilium.skipUpgrade` field to equal `true`.

```yaml
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: eks-anywhere
spec:
clusterNetwork:
cniConfig:
cilium:
skipUpgrade: true
...
```
```yaml
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: eks-anywhere
spec:
clusterNetwork:
cniConfig:
cilium:
skipUpgrade: true
...
```

3. Create the cluster according to the [Getting Started]({{< ref "/docs/getting-started" >}}) guide.

3. Create the cluster according to the [Getting Started]({{< ref "/docs/getting-started" >}}) section.
4. Pause reconciliation of the cluster. This ensures EKS Anywhere components do not attempt to remediate issues arising from a missing CNI.

4. Uninstall EKS Anywhere Cilium:
```bash
kubectl --kubeconfig=MANAGEMENT_KUBECONFIG -n eksa-system annotate clusters.cluster.x-k8s.io WORKLOAD_CLUSTER_NAME cluster.x-k8s.io/paused=true
```

5. Uninstall EKS Anywhere Cilium.

```bash
cilium uninstall
```

5. Install the custom CNI.
6. Install a custom CNI.

7. Resume reconciliation of the cluster object.

```bash
kubectl --kubeconfig=MANAGEMENT_KUBECONFIG -n eksa-system annotate clusters.cluster.x-k8s.io WORKLOAD_CLUSTER_NAME cluster.x-k8s.io/paused-
```

## Add a custom CNI to an existing cluster with eksctl

1. Modify the existing `Cluster` object's `spec.clusterNetwork.cniConfig.cilium.skipUpgrade` field to equal `true`.
```yaml
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: eks-anywhere
spec:
clusterNetwork:
cniConfig:
cilium:
skipUpgrade: true
...
```
```yaml
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: eks-anywhere
spec:
clusterNetwork:
cniConfig:
cilium:
skipUpgrade: true
...
```
2. [Upgrade the EKS Anywhere cluster]({{< ref "/docs/tasks/cluster/cluster-upgrades" >}}).
3. [Upgrade the EKS Anywhere cluster]({{< ref "/docs/tasks/cluster/cluster-upgrades" >}}).
4. Pause reconciliation of the cluster. This ensures EKS Anywhere components do not attempt to remediate issues arising from a missing CNI.
4. Uninstall EKS Anywhere Cilium:
```bash
kubectl --kubeconfig=MANAGEMENT_KUBECONFIG -n eksa-system annotate clusters.cluster.x-k8s.io WORKLOAD_CLUSTER_NAME cluster.x-k8s.io/paused=true
```
4. Uninstall EKS Anywhere Cilium.
```bash
cilium uninstall
```
5. Install the custom CNI.
5. Install a custom CNI.
6. Resume reconciliation of the cluster object.
```bash
kubectl --kubeconfig=MANAGEMENT_KUBECONFIG -n eksa-system annotate clusters.cluster.x-k8s.io WORKLOAD_CLUSTER_NAME cluster.x-k8s.io/paused-
```
## Add a custom CNI to an existing cluster with Lifecycle Controller
Expand All @@ -83,31 +109,41 @@ anywhere.eks.amazonaws.com/eksa-cilium: ""
1. Modify the existing `Cluster` object's `spec.clusterNetwork.cniConfig.cilium.skipUpgrade` field to equal `true`.

```yaml
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: eks-anywhere
spec:
clusterNetwork:
cniConfig:
cilium:
skipUpgrade: true
...
```
3. Apply the cluster configuration to the cluster.
```yaml
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: eks-anywhere
spec:
clusterNetwork:
cniConfig:
cilium:
skipUpgrade: true
...
```

2. Apply the cluster configuration to the cluster and _await successful object reconciliation_.

```bash
kubectl apply -f <cluster config path>
```

5. Await the cluster to reconcile.
4. Pause reconciliation of the cluster. This ensures EKS Anywhere components do not attempt to remediate issues arising from a missing CNI.

4. Uninstall EKS Anywhere Cilium:
```bash
kubectl --kubeconfig=MANAGEMENT_KUBECONFIG -n eksa-system annotate clusters.cluster.x-k8s.io WORKLOAD_CLUSTER_NAME cluster.x-k8s.io/paused=true
```

```bash
cilium uninstall
```
4. Uninstall EKS Anywhere Cilium.

```bash
cilium uninstall
```

5. Install a custom CNI.

6. Resume reconciliation of the cluster object.

5. Install the custom CNI.
```bash
kubectl --kubeconfig=MANAGEMENT_KUBECONFIG -n eksa-system annotate clusters.cluster.x-k8s.io WORKLOAD_CLUSTER_NAME cluster.x-k8s.io/paused-
```

0 comments on commit 7f9137b

Please sign in to comment.