Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document cluster topologies #4070

Merged
merged 7 commits into from
Nov 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions docs/content/en/docs/concepts/cluster-topologies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
title: "Cluster topologies"
linkTitle: "Cluster topologies"
weight: 20
description: >
Explanation of standalone vs. management/workload cluster topologies
---

For trying out EKS Anywhere or for times when a single cluster is needed, it is fine to create a _standalone cluster_ and run your workloads on it.
However, if you plan to create multiple clusters for running Kubernetes workloads, we recommend you create a _management cluster_.
Then use that management cluster to manage a set of workload clusters.

This document describes those two different EKS Anywhere cluster topologies.

## What is an EKS Anywhere management cluster?
An EKS Anywhere management cluster is a long-lived, on-premises Kubernetes cluster that can create and manage a fleet of EKS Anywhere workload clusters.
The workload clusters are where you run your applications.
The management cluster can only be created and managed by the Amazon CLI `eksctl`.

The management cluster runs on you hardware on-premises and it does not require any connectivity back to AWS to function.
Customers are responsible for operating the management cluster including (but not limited to) patching, upgrading, scaling, and monitoring the cluster control plane and data plane.

## What’s the difference between a management cluster and a standalone cluster?
From a technical point of view, they are the same.
Regardless of which deployment topology you choose, you always start by creating a singleton, standalone cluster that’s capable of managing itself.
This shows examples of separate, standalone clusters:

![Standalone clusters self-manage and can run applications](/images/eks-a_cluster_standalone.png)

Once a standalone cluster is created, you have an option to use it to use it as a management cluster to create separate workload cluster(s) under it, hence making this cluster a long-lived management cluster.
You can only use `eksctl` to create or delete the management cluster or a standalone cluster.
This shows examples of a management cluster that deploys and manages multiple workload clusters:

![Management clusters can create and manage multiple workload clusters](/images/eks-a_cluster_management.png)

## What’s the difference between a management cluster and a bootstrap cluster for EKS Anywhere?

A management cluster is a long-lived entity you have to actively operate.
The _bootstrap_ cluster is a temporary, short-lived kind cluster that is created on a separate [Administrative machine]({{< relref "clusterworkflow/#administrative-machine" >}}) to facilitate the creation of an initial standalone or management cluster.
You do not need to interact or operate the `kind` cluster.
The `kind` cluster will be automatically deleted by the end of the initial cluster creation.

## When should I deploy a management cluster?
If you want to run three or more EKS Anywhere clusters, we recommend that you choose a management/workload cluster deployment topology because of the advantages listed in the table below.
The EKS Anywhere Curated Packages feature recommends deploying certain packages such as the container registry package or monitoring packages on the management cluster to avoid circular dependency.


| | Standalone cluster topology | Management/workload cluster topology |
|--------|-----------------------------|---------------------------------------|
| **Pros** | Save hardware resources | Isolation of secrets |
| | Reduced operational overhead of maintaining a separate management cluster | Resource isolation between different teams. Reduced noisy-neighbor effect. |
| | | Isolation between development and production workloads. |
| | | Isolation between applications and fleet management services, such as monitoring server or container registry. |
| | | Provides a central control plane and API to automate cluster lifecycles |
| **Cons** | Shared secrets such, as SSH credentials or VMware credentials, across all teams who share the cluster. | Consumes extra resources. |
| | Without a central control plane (such as a parent management cluster), it is not possible to automate cluster creation/deletion with advanced methods like GitOps or IaC. |The creation/deletion of the management cluster itself can’t be automated through GitOps or IaC. |
| | Circular dependencies arise if the cluster has to host a monitoring server or a local container registry. |
||||


## Which EKS Anywhere features support the management/workload cluster deployment topology today?

| Features | Supported |
|------------|-----------|
| Create/update/delete a workload cluster on... ||
| <ul><li>VMware via CLI</li> | Y |
| <ul><li>CloudStack via CLI</li> | Y |
| <ul><li>Bare Metal via CLI</li> | N |
| Update a workload cluster on... ||
| <ul><li>VMware via GitOps/Terraform</li> | Y |
| <ul><li>CloudStack via GitOps/Terraform</li> | Y |
| <ul><li>Bare Metal via GitOps/Terraform</li> | N |
| Create/delete a workload cluster on...
| <ul><li>VMware via GitOps/Terraform</li> | N |
| <ul><li>CloudStack via GitOps/Terraform</li> | N |
| <ul><li>Bare Metal via GitOps/Terraform</li> | N |
| Install a curated package on the management cluster | Y ||
| Install a curated package on the workload cluster from the management cluster | Y |
| Install a curated package on the management cluster during a workload cluster creation | N |
16 changes: 15 additions & 1 deletion docs/content/en/docs/concepts/clusterworkflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,21 @@ description: >

Each EKS Anywhere cluster is built from a cluster specification file, with the structure of the configuration file based on the target provider for the cluster.
Currently, Bare Metal, CloudStack, and VMware vSphere are the recommended providers for supported EKS Anywhere clusters.
We step through the cluster creation workflow for those two providers here.
We step through the cluster creation workflow for those providers here.


## Management and workload clusters

EKS Anywhere offers two cluster deployment topology options:

* **Standalone cluster**: If want only a single EKS Anywhere cluster, you can deploy a self-managed, standalone cluster.
This type of cluster contains all Cluster API (CAPI) management components needed to manage itself, including managing its own upgrades.
It can also run workloads.

* **Management cluster with workload clusters**: If you plan to deploy multiple clusters, the project recommends you first deploy a _management cluster_.
The management cluster can then be used to deploy, upgrade, delete, and otherwise manage a fleet of _workload clusters_.

For further details about the different cluster topologies, see [Cluster topologies]({{< relref "cluster-topologies.md" >}})

## Before cluster creation

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ EKS Anywhere allows you to provision and manage Kubernetes clusters based on Ama

This document walks you through setting up EKS Anywhere as a self-managed cluster.
It does not yet support the concept of a separate management cluster for managing one or more workload clusters.
See [Cluster topologies]({{< relref "../../concepts/cluster-topologies" >}}) for details.

## Prerequisite checklist

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ If your initial cluster is a management cluster, it is intended to stay in place
Using a management cluster makes it faster to provision and delete workload clusters.
Also it lets you keep CloudStack credentials for a set of clusters in one place: on the management cluster.
The alternative is to simply use your initial cluster to run workloads.
See [Cluster topologies]({{< relref "../../concepts/cluster-topologies" >}}) for details.

{{% alert title="Important" color="warning" %}}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ If your initial cluster is a management cluster, it is intended to stay in place
Using a management cluster makes it faster to provision and delete workload clusters.
Also it lets you keep vSphere credentials for a set of clusters in one place: on the management cluster.
The alternative is to simply use your initial cluster to run workloads.
See [Cluster topologies]({{< relref "../../concepts/cluster-topologies" >}}) for details.

{{% alert title="Important" color="warning" %}}

Expand Down
Binary file added docs/static/images/eks-a_cluster_management.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading