Examples on how to deploy MinIO tenants with Let's Encrypt certificat…

…es (#1083)

Signed-off-by: Lenin Alevski <alevsk.8772@gmail.com>

docs/lets-encrypt.md

@@ -0,0 +1,121 @@
+# MinIO tenant with Let's Encrypt [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
+
+This document explains how to deploy a MinIO tenant using certificates generated by [Let's Encrypt](https://letsencrypt.org/).
+
+## Getting Started
+
+### Prerequisites
+
+- Kubernetes version `+v1.19`. While cert-manager supports [earlier K8s versions](https://cert-manager.io/docs/installation/supported-releases/), the MinIO Operator requires 1.19 or later.
+- MinIO Operator installed
+- `kubectl` access to your `k8s` cluster
+- [cert-manager](https://cert-manager.io/docs/installation/) 1.7.X or later installed
+```bash
+kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.7.2/cert-manager.yaml
+```
+- Support for assigning public IPs for `LoadBalancer` type services, if you are deploying `MinIO` on `GKE`, `EKS`, `AKS`
+or any other major public cloud provider this functionality is included out of the box, if you are deploying this on a 
+bare metal `kubernetes` cluster you can use [metallb](https://metallb.universe.tf/), ie:
+```bash
+kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/namespace.yaml
+kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml
+kubectl apply -f https://kind.sigs.k8s.io/examples/loadbalancer/metallb-configmap.yaml
+```
+- [Nginx](https://docs.nginx.com/nginx-ingress-controller/) ingress controller installed
+```bash
+helm repo add nginx-stable https://helm.nginx.com/stable
+helm repo update
+helm install nginx-ingress  nginx-stable/nginx-ingress \
+    --set rbac.create=true \
+    --set controller.service.type=LoadBalancer \
+    --set controller.service.externalTrafficPolicy=Local \
+    --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-proxy-protocol"="*" \
+    --set controller.config.use-proxy-protocol="true"
+```
+- [kustomize](https://kustomize.io/) installed
+- Configure your DNS to route traffic from the MinIO Tenant S3 API hostname (e.g. minio.example.com) and the Tenant Console  hostname(e.g. console.example.com) to the IP address of the worker node running ingress.
+
+
+### Deploy tenant
+
+In this example you are going to request a certificate valid for two domains, `minio.example.com` and `console.example.com`, replace `example.com`
+for the actual domain you want to use.
+
+Create a new `ClusterIssuer` that will request a certificate from `Let's Encrypt`:
+
+```bash
+cat <<EOF | kubectl apply -f -
+apiVersion: cert-manager.io/v1
+kind: ClusterIssuer
+metadata:
+  name: letsencrypt-prod
+  namespace: cert-manager
+spec:
+  acme:
+    server: https://acme-v02.api.letsencrypt.org/directory
+    email: contact@example.com
+    privateKeySecretRef:
+      name: letsencrypt-prod
+    solvers:
+      - http01:
+          ingress:
+            class:  nginx
+EOF
+```
+
+Use the example to deploy a `MinIO` tenant, go into base folder of the operator project and run the following command.
+
+```bash
+kustomize build examples/kustomization/tenant-letsencrypt | kubectl apply -f -
+```
+
+This tenant was deployed without TLS on purpose (`requestAutoCert: false`), however if you look at ingress rule on `examples/kustomization/tenant-letsencrypt/ingress.yaml`:
+
+```yaml
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: tenant-ingress
+  annotations:
+    kubernetes.io/ingress.class: nginx
+    cert-manager.io/cluster-issuer: "letsencrypt-prod"
+    nginx.ingress.kubernetes.io/proxy-body-size: 5t
+spec:
+  tls:
+    - hosts:
+        - minio.example.com
+        - console.example.com
+      secretName: tenant-tls
+  rules:
+    - host: minio.example.com
+      http:
+        paths:
+          - pathType: Prefix
+            path: "/"
+            backend:
+              service:
+                name: minio
+                port:
+                  number: 80
+    - host: console.example.com
+      http:
+        paths:
+          - pathType: Prefix
+            path: "/"
+            backend:
+              service:
+                name: storage-letsencrypt-console
+                port:
+                  number: 9090
+```
+
+`cert-manager` will request a certificate for this tenant using `Let's Encrypt` and store the actual public and private key on the `tenant-tls` secret.
+
+Once all MinIO pods are up and running you can query your endpoints with curl to make sure the communication happens over `TLS`.
+
+```bash
+curl -v https://minio.example.com
+curl -v https://console.example.com
+```
+
+In this example the `nginx ingress controller` will do the TLS termination.

examples/kustomization/tenant-letsencrypt/ingress.yaml

@@ -0,0 +1,35 @@
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: tenant-ingress
+  annotations:
+    kubernetes.io/ingress.class: nginx
+    cert-manager.io/cluster-issuer: "letsencrypt-prod"
+    nginx.ingress.kubernetes.io/proxy-body-size: 5t
+spec:
+  tls:
+    - hosts:
+        - minio.example.com
+        - console.example.com
+      secretName: tenant-tls
+  rules:
+    - host: minio.example.com
+      http:
+        paths:
+          - pathType: Prefix
+            path: "/"
+            backend:
+              service:
+                name: minio
+                port:
+                  number: 80
+    - host: console.example.com
+      http:
+        paths:
+          - pathType: Prefix
+            path: "/"
+            backend:
+              service:
+                name: storage-letsencrypt-console
+                port:
+                  number: 9090

examples/kustomization/tenant-letsencrypt/kustomization.yaml

@@ -0,0 +1,15 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - ingress.yaml
+  - ../base
+namespace: tenant-letsencrypt
+patchesStrategicMerge:
+  - tenant.yaml
+patchesJson6902:
+  - target:
+      group: minio.min.io
+      version: v2
+      kind: Tenant
+      name: storage
+    path: tenantNamePatch.yaml

examples/kustomization/tenant-letsencrypt/tenant.yaml

@@ -0,0 +1,16 @@
+apiVersion: minio.min.io/v2
+kind: Tenant
+metadata:
+  name: storage
+  namespace: minio-tenant
+spec:
+  ## Disable default tls certificates.
+  requestAutoCert: false
+  ## Set env variables to reference HTTPS endpoints.
+  env:
+    - name: MINIO_DOMAIN
+      value: "minio.example.com"
+    - name: MINIO_BROWSER_REDIRECT_URL
+      value: "https://console.example.com"
+    - name: MINIO_SERVER_URL
+      value: "https://minio.example.com"

examples/kustomization/tenant-letsencrypt/tenantNamePatch.yaml

@@ -0,0 +1,3 @@
+- op: replace
+  path: /metadata/name
+  value: storage-letsencrypt

pkg/controller/cluster/minio-services.go

@@ -62,7 +62,7 @@ func (c *Controller) checkMinIOSvc(ctx context.Context, tenant *miniov2.Tenant,
 	// check the specification of the MinIO ClusterIP service
 	if !minioSvcMatchesSpec {
 		if err != nil {
-			klog.Infof("Services don't match: %s", err)
+			klog.Infof("MinIO Services don't match: %s", err)
 		}
 
 		svc.ObjectMeta.Annotations = expectedSvc.ObjectMeta.Annotations