Skip to content

Commit

Permalink
Add spec doc for cluster infra provider
Browse files Browse the repository at this point in the history
Add a specification document containing the required data model and
behaviors for cluster infrastructure providers.

Signed-off-by: Andy Goldstein <goldsteina@vmware.com>
  • Loading branch information
ncdc committed Dec 2, 2019
1 parent 14c81fe commit 62092ef
Show file tree
Hide file tree
Showing 4 changed files with 126 additions and 0 deletions.
1 change: 1 addition & 0 deletions docs/book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
- [MachineDeployment](./architecture/controllers/machine-deployment.md)
- [Provider Implementers](./providers/implementers.md)
- [v1alpha1 to v1alpha2](./providers/v1alpha1-to-v1alpha2.md)
- [Cluster Infrastructure](./providers/cluster-infrastructure.md)

- [Reference](./reference/reference.md)
- [Glossary](./reference/glossary.md)
Expand Down
34 changes: 34 additions & 0 deletions docs/book/src/images/cluster-infra-provider.plantuml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
@startuml

start

:New/Updated/Deleted resource;

if (Deleted?) then (yes)
if (Has cluster owner?) then (yes)
:Reconcile deletion;
if (Error?) then (yes)
stop
else (no)
endif
else (no)
endif
:Delete provider-specific finalizer;
else (no)
if (Has cluster owner?) then (yes)
:Add provider-specific finalizer if needed;
:Reconcile provider cluster infrastructure;
if (Error?) then (yes)
stop
elseif (Created load balancer?) then (yes)
:Set status.controlPlaneEndpoint;
else (no)
endif
:Set status.ready to true;
else (no)
endif
endif
:Patch resource to persist changes;
stop

@enduml
Binary file added docs/book/src/images/cluster-infra-provider.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
91 changes: 91 additions & 0 deletions docs/book/src/providers/cluster-infrastructure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Cluster Infrastructure Provider Specification

## Overview

A cluster infrastructure provider supplies whatever prerequisites are necessary for running machines.
Examples might include networking, load balancers, firewall rules, and so on.

## Data Types

A cluster infrastructure provider must define an API type for "infrastructure cluster" resources. The type:

1. Must belong to an API group served by the Kubernetes apiserver
2. May be implemented as a CustomResourceDefinition, or as part of an aggregated apiserver
3. Must be namespace-scoped
4. Must have the standard Kubernetes "type metadata" and "object metadata"
5. Must have a `spec` field with the following:
1. Required fields:
1. `controlPlaneEndpoint` (`apiEndpoint`): the endpoint for the cluster's control plane. `apiEndpoint` is defined
as:
- `host` (string): DNS name or IP address
- `port` (int32): TCP port
6. Must have a `status` field with the following:
1. Required fields:
1. `ready` (boolean): indicates the provider-specific infrastructure has been provisioned and is ready
2. Optional fields:
1. `failureReason` (string): indicates there is a fatal problem reconciling the provider's infrastructure;
meant to be suitable for programmatic interpretation
2. `failureMessage` (string): indicates there is a fatal problem reconciling the provider's infrastructure;
meant to be a more descriptive value than `failureReason`

## Behavior

A cluster infrastructure provider must respond to changes to its "infrastructure cluster" resources. This process is
typically called reconciliation. The provider must watch for new, updated, and deleted resources and respond
accordingly.

The following diagram shows the typical logic for a cluster infrastructure provider:

![Cluster infrastructure provider activity diagram](../images/cluster-infra-provider.png)

### Normal resource

1. If the resource does not have a `Cluster` owner, exit the reconciliation
1. The Cluster API `Cluster` reconciler populates this based on the value in the `Cluster`'s `spec.infrastructureRef`
field.
1. Add the provider-specific finalizer, if needed
1. Reconcile provider-specific cluster infrastructure
1. If any errors are encountered, exit the reconciliation
1. If the provider created a load balancer for the control plane, record its hostname or IP in `spec.controlPlaneEndpoint`
1. Set `status.ready` to `true`
1. Patch the resource to persist changes

### Deleted resource

1. If the resource has a `Cluster` owner
1. Perform deletion of provider-specific cluster infrastructure
1. If any errors are encountered, exit the reconciliation
1. Remove the provider-specific finalizer from the resource
1. Patch the resource to persist changes

## RBAC

### Provider controller

A cluster infrastructure provider must have RBAC permissions for the types it defines. If you are using `kubebuilder` to
generate new API types, these permissions should be configured for you automatically. For example, the AWS provider has
the following configuration for its `AWSCluster` type:

```
// +kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=awsclusters,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=awsclusters/status,verbs=get;update;patch
```

A cluster infrastructure provider may also need RBAC permissions for other types, such as `Cluster`. If you need
read-only access, you can limit the permissions to `get`, `list`, and `watch`. The AWS provider has the following
configuration for retrieving `Cluster` resources:

```
// +kubebuilder:rbac:groups=cluster.x-k8s.io,resources=clusters;clusters/status,verbs=get;list;watch
```

### Cluster API controllers

The Cluster API controller for `Cluster` resources is configured with full read/write RBAC permissions for all resources
in the `infrastructure.cluster.x-k8s.io` API group. This group represents all cluster infrastructure providers for SIG
Cluster Lifecycle-sponsored provider subprojects. If you are writing a provider not sponsored by the SIG, you must add
new RBAC permissions for the Cluster API `manager-role` role, granting it full read/write access to the "infrastructure
cluster" resource in your API group.

Note, the write permissions allow the `Cluster` controller to set owner references and labels on the "infrastructure
cluster" resources; they are not used for general mutations of these resources.

0 comments on commit 62092ef

Please sign in to comment.