Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add kustomize for minioinstance #168

Merged
merged 2 commits into from
Jul 20, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ MinIO-Operator brings native MinIO, [MCS](https://github.com/minio/mcs), and [KE
| Feature | Reference Document |
|-------------------------|--------------------|
| Create and delete highly available distributed MinIO clusters | [Create a MinIO Instance](https://github.com/minio/minio-operator#create-a-minio-instance). |
| Automatic TLS for MinIO | [Automatic TLS for MinIO Instance](https://github.com/minio/minio-operator/blob/master/docs/tls.md#automatic-csr-generation). |
| Expand an existing MinIO cluster | [Expand a MinIO Cluster](https://github.com/minio/minio-operator/blob/master/docs/adding-zones.md). |
| Use a custom template for hostname discovery | [Custom Hostname Discovery](https://github.com/minio/minio-operator/blob/master/docs/custom-name-templates.md). |
| TLS Configuration | [TLS for MinIO Instance](https://github.com/minio/minio-operator/blob/master/docs/tls.md). |
| Expand an existing MinIO cluster | [Expand a MinIO Cluster](https://github.com/minio/minio-operator/blob/master/docs/adding-zones.md). |
| Use a custom template for hostname discovery | [Custom Hostname Discovery](https://github.com/minio/minio-operator/blob/master/docs/custom-name-templates.md). |
| Use PodSecurityPolicy for MinIO Pods | [Apply PodSecurityPolicy](https://github.com/minio/minio-operator/blob/master/docs/pod-security-policy.md). |
| Deploy MCS with MinIO cluster | [Deploy MinIO Instance with MCS](https://github.com/minio/minio-operator/blob/master/docs/mcs.md). |
| Deploy KES with MinIO cluster | [Deploy MinIO Instance with KES](https://github.com/minio/minio-operator/blob/master/docs/kes.md). |
Expand All @@ -35,7 +35,6 @@ Advanced users can leverage [kustomize](https://github.com/kubernetes-sigs/kusto

```bash
git clone https://github.com/minio/minio-operator
cd operator-deployment
kustomize build | kubectl apply -f -
```

Expand All @@ -57,8 +56,8 @@ These variables may be passed to operator Deployment in order to modify some of

| Name | Default | Description |
| --- | --- | --- |
| `WATCHED_NAMESPACE` | | If set, the operator will watch only MinIO resources deployed in the specified namespace. All namespaces are watched if empty |
| `CLUSTER_DOMAIN` | cluster.local | Cluster Domain of the Kubernetes cluster |
| `CLUSTER_DOMAIN` | `cluster.local` | Cluster Domain of the Kubernetes cluster |
| `WATCHED_NAMESPACE` | `-` | If set, the operator will watch MinIOInstance resources in specified namespace only. If empty, operator will watch all namespaces. |

## Explore Further

Expand Down
4 changes: 2 additions & 2 deletions examples/minioinstance-kes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ spec:
prometheus.io/port: "9000"
prometheus.io/scrape: "true"
## Registry location and Tag to download MinIO Server image
image: minio/minio:RELEASE.2020-07-13T18-09-56Z
image: minio/minio:RELEASE.2020-07-14T19-14-30Z
## A ClusterIP Service will be created with the given name
serviceName: minio-internal-service
## Secret with credentials to be used by MinIO instance.
Expand Down Expand Up @@ -85,7 +85,7 @@ spec:
storage: 1Ti
## Define configuration for MCS (Graphical user interface for MinIO)
mcs:
image: minio/mcs:v0.1.0
image: minio/mcs:v0.2.0
replicas: 2
mcsSecret:
name: mcs-secret
Expand Down
4 changes: 2 additions & 2 deletions examples/minioinstance-mcs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ spec:
prometheus.io/port: "9000"
prometheus.io/scrape: "true"
## Registry location and Tag to download MinIO Server image
image: minio/minio:RELEASE.2020-07-13T18-09-56Z
image: minio/minio:RELEASE.2020-07-14T19-14-30Z
## A ClusterIP Service will be created with the given name
serviceName: minio-internal-service
## Secret with credentials to be used by MinIO instance.
Expand Down Expand Up @@ -85,7 +85,7 @@ spec:
storage: 1Ti
## Define configuration for MCS (Graphical user interface for MinIO)
mcs:
image: minio/mcs:v0.1.0
image: minio/mcs:v0.2.0
replicas: 2
mcsSecret:
name: mcs-secret
Expand Down
2 changes: 1 addition & 1 deletion examples/minioinstance-pod-security-policy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ spec:
prometheus.io/port: "9000"
prometheus.io/scrape: "true"
## Registry location and Tag to download MinIO Server image
image: minio/minio:RELEASE.2020-07-13T18-09-56Z
image: minio/minio:RELEASE.2020-07-14T19-14-30Z
## A ClusterIP Service will be created with the given name
serviceName: minio-internal-service
## Service account to be used for all the MinIO Pods
Expand Down
2 changes: 1 addition & 1 deletion examples/minioinstance.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ spec:
prometheus.io/port: "9000"
prometheus.io/scrape: "true"
## Registry location and Tag to download MinIO Server image
image: minio/minio:RELEASE.2020-07-13T18-09-56Z
image: minio/minio:RELEASE.2020-07-14T19-14-30Z
## A ClusterIP Service will be created with the given name
serviceName: minio-internal-service
zones:
Expand Down
15 changes: 8 additions & 7 deletions operator-deployment/kustomization.yaml → kustomization.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: minio-operator

# Configure number of MinIO Operator Deployment Replicas
replicas:
Expand All @@ -11,7 +10,7 @@ replicas:
images:
- name: minio/k8s-operator
newName: minio/k8s-operator
newTag: 2.0.6
newTag: 2.0.9

# Configure the Cluster Domain and NameSpace to Watch
configMapGenerator:
Expand Down Expand Up @@ -39,6 +38,8 @@ patchesJson6902:
path: /metadata/name
value: "minio-operator"

namespace: minio-operator

vars:
- name: CLUSTER_DOMAIN
objref:
Expand All @@ -56,8 +57,8 @@ vars:
fieldpath: data.WATCHED_NAMESPACE

resources:
- namespace.yaml
- service-account.yaml
- crd.yaml
- rbac.yaml
- deployment.yaml
- operator-kustomize/namespace.yaml
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should call this folder base as in the examples of kustomize

- operator-kustomize/service-account.yaml
- operator-kustomize/crd.yaml
- operator-kustomize/rbac.yaml
- operator-kustomize/deployment.yaml
60 changes: 60 additions & 0 deletions minioinstance-kustomize/kes-secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
apiVersion: v1
kind: Secret
metadata:
name: kes-config
type: Opaque
stringData:
server-config.yaml: |-
address: 0.0.0.0:7373
root: _ # Effectively disabled since no root identity necessary.
tls:
key: /home/server.key # Path to the TLS private key
cert: /home/server.crt # Path to the TLS certificate
proxy:
identities: []
header:
cert: X-Tls-Client-Cert
policy:
my-policy:
paths:
- /v1/key/create/*
- /v1/key/generate/*
- /v1/key/decrypt/*
identities:
- ${MINIO_ID}
cache:
expiry:
any: 5m0s
unused: 20s
log:
error: on
audit: off
keys:
fs:
path: "/kes" # Path to directory. Keys will be stored as files. Not Recommended for Production.
# vault:
# endpoint: "" # The Vault endpoint - e.g. https://127.0.0.1:8200
# namespace: "" # An optional Vault namespace. See: https://www.vaultproject.io/docs/enterprise/namespaces/index.html
# prefix: "" # An optional K/V prefix. The server will store keys under this prefix.
# approle: # AppRole credentials. See: https://www.vaultproject.io/docs/auth/approle.html
# id: "" # Your AppRole Role ID
# secret: "" # Your AppRole Secret ID
# retry: 15s # Duration until the server tries to re-authenticate after connection loss.
# tls: # The Vault client TLS configuration for mTLS authentication and certificate verification
# key: "" # Path to the TLS client private key for mTLS authentication to Vault
# cert: "" # Path to the TLS client certificate for mTLS authentication to Vailt
# ca: "" # Path to one or multiple PEM root CA certificates
# status: # Vault status configuration. The server will periodically reach out to Vault to check its status.
# ping: 10s # Duration until the server checks Vault's status again.
# aws:
# # The AWS SecretsManager key store. The server will store
# # secret keys at the AWS SecretsManager encrypted with
# # AWS-KMS. See: https://aws.amazon.com/secrets-manager
# secretsmanager:
# endpoint: "" # The AWS SecretsManager endpoint - e.g.: secretsmanager.us-east-2.amazonaws.com
# region: "" # The AWS region of the SecretsManager - e.g.: us-east-2
# kmskey: "" # The AWS-KMS key ID used to en/decrypt secrets at the SecretsManager. By default (if not set) the default AWS-KMS key will be used.
# credentials: # The AWS credentials for accessing secrets at the AWS SecretsManager.
# accesskey: "" # Your AWS Access Key
# secretkey: "" # Your AWS Secret Key
# token: "" # Your AWS session token (usually optional)
84 changes: 84 additions & 0 deletions minioinstance-kustomize/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: minio
configurations:
- kustomizeconfig/minioinstance.yaml

# Configure tenant name, volumes per server, capacity per volume and storage class
configMapGenerator:
- name: minio-storage
literals:
- tenant="minio-tenant1"
- volumesPerServer=4
- capacity="5Ti"
- storageClass=""
- zones="rack1:4"

# Configure repo and tag of MinIO, MCS & KES Image
images:
- name: minio/minio
newName: minio/minio
newTag: RELEASE.2020-07-14T19-14-30Z
- name: minio/mcs
newName: minio/mcs
newTag: v0.2.0
- name: minio/kes
newName: minio/kes
newTag: v0.10.1

# Configure the Namespace
patchesJson6902:
- target:
version: v1
kind: Namespace
name: minio
patch: |-
- op: replace
path: /metadata/name
value: "minio"

vars:
- name: NAME
objref:
kind: ConfigMap
name: minio-storage
apiVersion: v1
fieldref:
fieldpath: data.tenant
- name: ZONES
objref:
kind: ConfigMap
name: minio-storage
apiVersion: v1
fieldref:
fieldpath: data.zones
- name: VOLUMES_PER_SERVER
objref:
kind: ConfigMap
name: minio-storage
apiVersion: v1
fieldref:
fieldpath: data.volumesPerServer
- name: CAPACITY
objref:
kind: ConfigMap
name: minio-storage
apiVersion: v1
fieldref:
fieldpath: data.capacity
- name: STORAGE_CLASS
objref:
kind: ConfigMap
name: minio-storage
apiVersion: v1
fieldref:
fieldpath: data.storageClass


resources:
- namespace.yaml
- mcs-secret.yaml
- kes-secret.yaml
- minio-secret.yaml
- minio-service.yaml
- minio-tenant.yaml
19 changes: 19 additions & 0 deletions minioinstance-kustomize/kustomizeconfig/minioinstance.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
images:
- path: spec/mcs/image
kind: MinIOInstance
- path: spec/kes/image
kind: MinIOInstance
- path: spec/image
kind: MinIOInstance

varReference:
- path: metadata/name
kind: MinIOInstance
- path: spec/volumesPerServer
kind: MinIOInstance
- path: spec/volumeClaimTemplate/spec/resources/requests/storage
kind: MinIOInstance
- path: spec/volumeClaimTemplate/spec/storageClassName
kind: MinIOInstance
- path: spec/zones
kind: MinIOInstance
11 changes: 11 additions & 0 deletions minioinstance-kustomize/mcs-secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
apiVersion: v1
kind: Secret
metadata:
name: mcs-secret
type: Opaque
data:
MCS_HMAC_JWT_SECRET: WU9VUkpXVFNJR05JTkdTRUNSRVQ= # base 64 encoded "YOURJWTSIGNINGSECRET" (echo -n 'YOURJWTSIGNINGSECRET' | base64)
MCS_PBKDF_PASSPHRASE: U0VDUkVU # base 64 encoded "SECRET" (echo -n 'SECRET' | base64)
MCS_PBKDF_SALT: U0VDUkVU # base 64 encoded "SECRET" (echo -n 'SECRET' | base64)
MCS_ACCESS_KEY: WU9VUk1DU0FDQ0VTUw== # base 64 encoded "YOURMCSACCESS" (echo -n 'YOURMCSACCESS' | base64)
MCS_SECRET_KEY: WU9VUk1DU1NFQ1JFVA== # base 64 encoded "YOURMCSSECRET" (echo -n 'YOURMCSSECRET' | base64)
8 changes: 8 additions & 0 deletions minioinstance-kustomize/minio-secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: v1
kind: Secret
metadata:
name: minio-creds-secret
type: Opaque
data:
accesskey: bWluaW8= # base 64 encoded "minio" (echo -n 'minio' | base64)
secretkey: bWluaW8xMjM= # based 64 encoded "minio123" (echo -n 'minio123' | base64)
15 changes: 15 additions & 0 deletions minioinstance-kustomize/minio-service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
name: minio-service
spec:
type: ClusterIP
ports:
- port: 9000
targetPort: 9000
protocol: TCP
# Optional field
# By default and for convenience, the Kubernetes control plane will allocate a port from a range (default: 30000-32767)
# nodePort: 30007
selector:
app: minio
55 changes: 55 additions & 0 deletions minioinstance-kustomize/minio-tenant.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
apiVersion: operator.min.io/v1
kind: MinIOInstance
metadata:
name: $(NAME)
namespace: minio
spec:
metadata:
labels:
app: minio
annotations:
prometheus.io/path: /minio/prometheus/metrics
prometheus.io/port: "9000"
prometheus.io/scrape: "true"
image: minio/minio:RELEASE.2020-07-14T19-14-30Z
serviceName: minio-internal-service
credsSecret:
name: minio-creds-secret
zones: $(ZONES)
volumesPerServer: $(VOLUMES_PER_SERVER)
mountPath: /export
volumeClaimTemplate:
metadata:
name: data
spec:
accessModes:
- ReadWriteOnce
storageClassName: $(STORAGE_CLASS)
resources:
requests:
storage: $(CAPACITY)
mcs:
image: minio/mcs:v0.2.0
replicas: 2
mcsSecret:
name: mcs-secret
metadata:
labels:
app: mcs
kes:
image: minio/kes:v0.10.1
replicas: 2
kesSecret:
name: kes-config
metadata:
labels:
app: kes
requestAutoCert: true
podManagementPolicy: Parallel
certConfig:
commonName: ""
organizationName: []
dnsNames: []
liveness:
initialDelaySeconds: 120
periodSeconds: 60
4 changes: 4 additions & 0 deletions minioinstance-kustomize/namespace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: minio
Loading