Updating documentation and charts

README.md

@@ -44,11 +44,17 @@ The Pravega Operator manages Pravega clusters deployed to Kubernetes and automat
 
 ## Quickstart
 
+We recommend using our [helm charts](charts) for all installation and upgrades (but not for rollbacks at the moment since helm rollbacks are still experimental). The helm charts for pravega operator (version 0.4.5 onwards) and pravega cluster (version 0.5.0 onwards) are published in [https://charts.pravega.io](https://charts.pravega.io/). To add this repository to your Helm repos, use the following command
+```
+helm repo add pravega https://charts.pravega.io
+```
+There are manual deployment, upgrade and rollback options available as well.
+
 ### Install the Operator
 
 > Note: If you are running on Google Kubernetes Engine (GKE), please [check this first](doc/development.md#installation-on-google-kubernetes-engine).
 
-We recommend using helm to deploy a Pravega Operator. Check out the [helm installation](charts/pravega-operator/README.md) document for instructions.
+To understand how to deploy a Pravega Operator using helm, refer to [this](charts/pravega-operator#installing-the-chart).
 
 #### Deploying in Test Mode
  The Operator can be run in a "test" mode if we want to create pravega on minikube or on a cluster with very limited resources by  enabling `testmode: true` in `values.yaml` file. Operator running in test mode skips minimum replica requirement checks on Pravega components. "Test" mode ensures a bare minimum setup of pravega and is not recommended to be used in production environments.
@@ -79,22 +85,9 @@ $ kubectl create -f ./example/pvc-tier2.yaml
 
 #### Install a Pravega cluster
 
-Use Helm to install a sample Pravega cluster with release name `bar`.
-
-```
-$ helm install bar charts/pravega --set zookeeperUri=[ZOOKEEPER_HOST] --set bookkeeperUri=[BOOKKEEPER_SVC] --set storage.longtermStorage.filesystem.pvc=[TIER2_NAME]
-```
-
-where:
-
-- **[ZOOKEEPER_HOST]** is the host or IP address of your Zookeeper deployment (e.g. `zookeeper-client:2181`). Multiple Zookeeper URIs can be specified, use a comma-separated list and DO NOT leave any spaces in between (e.g. `zookeeper-0:2181,zookeeper-1:2181,zookeeper-2:2181`).
-- **[BOOKKEEPER_SVC]** is the is the name of the headless service of your Bookkeeper deployment (e.g. `bookkeeper-bookie-0.bookkeeper-bookie-headless.default.svc.cluster.local:3181,bookkeeper-bookie-1.bookkeeper-bookie-headless.default.svc.cluster.local:3181,bookkeeper-bookie-2.bookkeeper-bookie-headless.default.svc.cluster.local:3181`).
-- **[TIER2_NAME]** is the longtermStorage `PersistentVolumeClaim` name. `pravega-tier2` if you created the PVC above.
-
-
-Check out the [Pravega Helm Chart](charts/pravega) for the complete list of configurable parameters.
+To understand how to deploy a pravega cluster using helm, refer to [this](charts/pravega#installing-the-chart).
 
-Verify that the cluster instances and its components are being created.
+Once the pravega cluster with release name `bar` has been created, use the following command to verify that the cluster instances and its components are being created.
 
 ```
 $ kubectl get PravegaCluster
@@ -132,27 +125,27 @@ statefulset.apps/bar-pravega-segmentstore       3         3         2m
 By default, a `PravegaCluster` instance is only accessible within the cluster through the Controller `ClusterIP` service. From within the Kubernetes cluster, a client can connect to Pravega at:
 
 ```
-tcp://<pravega-name>-pravega-controller.<namespace>:9090
+tcp://[CLUSTER_NAME]-pravega-controller.[NAMESPACE]:9090
 ```
 
 And the `REST` management interface is available at:
 
 ```
-http://<pravega-name>-pravega-controller.<namespace>:10080/
+http://[CLUSTER_NAME]-pravega-controller.[NAMESPACE]:10080/
 ```
 
 Check out the [external access documentation](doc/external-access.md) if your clients need to connect to Pravega from outside Kubernetes.
 
-Check out the [exposing Segmentstore service on single IP address ](https://github.com/pravega/pravega-operator/blob/4aa88641c3d5a1d5afbb2b9e628846639fd13290/doc/external-access.md#exposing-segmentstore-service-on-single-ip-address-and-different-ports) if your clients need to connect to Pravega Segment store on the same IP address from outside Kubernetes.
+Check out the [exposing Segmentstore service on single IP address](https://github.com/pravega/pravega-operator/blob/4aa88641c3d5a1d5afbb2b9e628846639fd13290/doc/external-access.md#exposing-segmentstore-service-on-single-ip-address-and-different-ports) if your clients need to connect to Pravega Segment store on the same IP address from outside Kubernetes.
 
 ### Scale a Pravega cluster
 
-You can scale Pravega components independently by modifying their corresponding field in the Pravega resource spec. You can either `kubectl edit` the cluster or `kubectl patch` it. If you edit it, update the number of replicas for BookKeeper, Controller, and/or Segment Store and save the updated spec.
+You can scale Pravega components independently by modifying their corresponding field in the Pravega resource spec. You can either `kubectl edit` the cluster or `kubectl patch` it. If you edit it, update the number of replicas for Controller, and/or Segment Store and save the updated spec.
 
 Example of patching the Pravega resource to scale the Segment Store instances to 4.
 
 ```
-kubectl patch PravegaCluster <pravega-name> --type='json' -p='[{"op": "replace", "path": "/spec/pravega/segmentStoreReplicas", "value": 4}]'
+kubectl patch PravegaCluster [CLUSTER_NAME] --type='json' -p='[{"op": "replace", "path": "/spec/pravega/segmentStoreReplicas", "value": 4}]'
 ```
 
 ### Upgrade a Pravega cluster
@@ -162,7 +155,7 @@ Check out the [upgrade guide](doc/upgrade-cluster.md).
 ### Uninstall the Pravega cluster
 
 ```
-$ helm uninstall bar
+$ helm uninstall [PRAVEGA_RELEASE_NAME]
 $ kubectl delete -f ./example/pvc-tier2.yaml
 ```
 
@@ -171,7 +164,7 @@ $ kubectl delete -f ./example/pvc-tier2.yaml
 > Note that the Pravega clusters managed by the Pravega operator will NOT be deleted even if the operator is uninstalled.
 
 ```
-$ helm uninstall foo
+$ helm uninstall [PRAVEGA_OPERATOR_RELEASE_NAME]
 ```
 
 If you want to delete the Pravega clusters, make sure to do it before uninstalling the operator.

charts/pravega-operator/README.md

@@ -13,35 +13,41 @@ This chart bootstraps a [pravega-operator](https://github.com/pravega/pravega-op
   - An existing Apache Bookkeeper 4.9.2 cluster. This can be easily deployed using our [BookKeeper Operator](https://github.com/pravega/bookkeeper-operator)
   - Cert-Manager v0.15.0+ or some other certificate management solution in order to manage the webhook service certificates. This can be easily deployed by referring to [this](https://cert-manager.io/docs/installation/kubernetes/)
   - An Issuer and a Certificate (either self-signed or CA signed) in the same namespace that the Pravega Operator will be installed (refer to [this](https://github.com/pravega/pravega-operator/blob/master/deploy/certificate.yaml) manifest to create a self-signed certificate in the default namespace)
-  > The name of the certificate (*webhookCert.certName*), the name of the secret created by this certificate (*webhookCert.secretName*), the tls.crt (*webhookCert.crt*) and tls.key (*webhookCert.key*) need to be specified against the corresponding fields in the values.yaml file, or can be provided with the install command as shown [here](#installing-the-chart).
-  The values *tls.crt* and *tls.key* are contained in the secret which is created by the certificate and can be obtained using the following command
-  ```
-  kubectl get secret <secret-name> -o yaml | grep tls.
-  ```
 
 ## Installing the Chart
 
-To install the chart with the release name `my-release`:
+To install the pravega-operator chart, use the following commands:
 
 ```
-$ helm install my-release pravega-operator --set webhookCert.crt=<tls.crt> --set webhookCert.generate=false --set webhookCert.certName=<cert-name> --set webhookCert.secretName=<secret-name>
+$ helm repo add pravega https://charts.pravega.io
+$ helm repo update
+$ helm install [RELEASE_NAME] pravega/pravega-operator --version=[VERSION] --set webhookCert.crt=[TLS_CRT] --set webhookCert.generate=false --set webhookCert.certName=[CERT_NAME] --set webhookCert.secretName=[SECRET_NAME]
 ```
+where:
+- **[RELEASE_NAME]** is the release name for the pravega-operator chart.
+- **[DEPLOYMENT_NAME]** is the name of the pravega-operator deployment so created. (If [RELEASE_NAME] contains the string `pravega-operator`, `[DEPLOYMENT_NAME] = [RELEASE_NAME]`, else `[DEPLOYMENT_NAME] = [RELEASE_NAME]-pravega-operator`. The [DEPLOYMENT_NAME] can however be overridden by providing `--set fullnameOverride=[DEPLOYMENT_NAME]` along with the helm install command)
+- **[VERSION]** can be any stable release version for pravega-operator from 0.5.0 onwards.
+- **[CERT_NAME]** is the name of the certificate created in the previous step
+- **[SECRET_NAME]** is the name of the secret created by the above certificate
+- **[TLS_CRT]** is contained in the above secret and can be obtained using the command `kubectl get secret [SECRET_NAME] -o yaml | grep tls.crt`
 
-The command deploys pravega-operator on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.
+This command deploys a pravega-operator on the Kubernetes cluster in its default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.
+
+>Note: If the pravega-operator version is 0.4.5, webhookCert.generate, webhookCert.crt, webhookCert.certName and webhookCert.secretName should not be set. Also in this case, bookkeeper operator, cert-manager and the certificate/issuer do not need to be deployed as prerequisites.
 
 ## Uninstalling the Chart
 
-To uninstall/delete the `my-release` deployment:
+To uninstall/delete the pravega-operator chart, use the following command:
 
 ```
-$ helm uninstall my-release
+$ helm uninstall [RELEASE_NAME]
 ```
 
-The command removes all the Kubernetes components associated with the chart and deletes the release.
+This command removes all the Kubernetes components associated with the chart and deletes the release.
 
 ## Configuration
 
-The following table lists the configurable parameters of the Pravega operator chart and their default values.
+The following table lists the configurable parameters of the pravega-operator chart and their default values.
 
 | Parameter | Description | Default |
 | ----- | ----------- | ------ |

charts/pravega-operator/templates/webhook.yaml

@@ -23,7 +23,7 @@ metadata:
   labels:
 {{ include "pravega-operator.commonLabels" . | indent 4 }}
   annotations:
-    cert-manager.io/inject-ca-from: default/selfsigned-cert
+    cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ .Values.certName }}
 webhooks:
 - clientConfig:
     service:

charts/pravega/README.md

@@ -16,27 +16,39 @@ This chart creates a [Pravega](https://github.com/pravega/pravega) cluster in [K
 
 ## Installing the Chart
 
-To install the chart with the release name `my-release`:
+To install the pravega chart, use the following commands:
 
 ```
-$ helm install my-release pravega
+$ helm repo add pravega https://charts.pravega.io
+$ helm repo update
+$ helm install [RELEASE_NAME] pravega/pravega --version=[VERSION] --set zookeeperUri=[ZOOKEEPER_HOST] --set bookkeeperUri=[BOOKKEEPER_SVC] --set storage.longtermStorage.filesystem.pvc=[TIER2_NAME]
 ```
+where:
+- **[RELEASE_NAME]** is the release name for the pravega chart.
+- **[CLUSTER_NAME]** is the name of the pravega cluster so created. (If [RELEASE_NAME] contains the string `pravega`, `[CLUSTER_NAME] = [RELEASE_NAME]`, else `[CLUSTER_NAME] = [RELEASE_NAME]-pravega`. The [CLUSTER_NAME] can however be overridden by providing `--set fullnameOverride=[CLUSTER_NAME]` along with the helm install command)
+  **Note:** You need to ensure that the [CLUSTER_NAME] is the same value as that provided in the [bookkeeper chart configuration](https://github.com/pravega/bookkeeper-operator/tree/master/charts/bookkeeper#configuration), the default value for which is `pravega` and can be achieved by either providing the `[RELEASE_NAME] = pravega` or by providing `--set fullnameOverride=pravega` at the time of installing the pravega chart. On the contrary, the default value of [CLUSTER_NAME] in the bookkeeper charts can also be overridden by providing `--set pravegaClusterName=[CLUSTER_NAME]` at the time of installing the bookkeeper chart)
+- **[VERSION]** can be any stable release version for pravega from 0.5.0 onwards.
+- **[ZOOKEEPER_HOST]** is the host or IP address of your Zookeeper deployment (e.g. `zookeeper-client:2181`). Multiple Zookeeper URIs can be specified, use a comma-separated list and DO NOT leave any spaces in between (e.g. `zookeeper-0:2181,zookeeper-1:2181,zookeeper-2:2181`).
+- **[BOOKKEEPER_SVC]** is the is the name of the headless service of your Bookkeeper deployment (e.g. `bookkeeper-bookie-0.bookkeeper-bookie-headless.default.svc.cluster.local:3181,bookkeeper-bookie-1.bookkeeper-bookie-headless.default.svc.cluster.local:3181,bookkeeper-bookie-2.bookkeeper-bookie-headless.default.svc.cluster.local:3181`).
+- **[TIER2_NAME]** is the longtermStorage `PersistentVolumeClaim` name (`pravega-tier2` if you created the PVC using the manifest provided).
 
-The command deploys pravega on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.
+This command deploys pravega on the Kubernetes cluster in its default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.
+
+>Note: If the underlying pravega operator version is 0.4.5, bookkeeperUri should not be set, and the pravega-bk chart should be used instead of the pravega chart
 
 ## Uninstalling the Chart
 
-To uninstall/delete the `my-release` deployment:
+To uninstall/delete the pravega chart, use the following command:
 
 ```
-$ helm uninstall my-release
+$ helm uninstall [RELEASE_NAME]
 ```
 
-The command removes all the Kubernetes components associated with the chart and deletes the release.
+This command removes all the Kubernetes components associated with the chart and deletes the release.
 
 ## Configuration
 
-The following table lists the configurable parameters of the Pravega chart and their default values.
+The following table lists the configurable parameters of the pravega chart and their default values.
 
 | Parameter | Description | Default |
 | ----- | ----------- | ------ |
@@ -70,8 +82,8 @@ The following table lists the configurable parameters of the Pravega chart and t
 | `segmentStore.resources.limits.memory` | Memory limits for segmentStore | `4Gi` |
 | `segmentStore.service.type` | Override the segmentStore service type, if external access is enabled (LoadBalancer/NodePort) | |
 | `segmentStore.service.annotations` | Annotations to add to the segmentStore service, if external access is enabled | `{}` |
-| `segmentStore.service.segmentStoreLoadBalancerIP` |It is used to provide a LoadBalancerIP | |
-| `segmentStore.service.segmentStoreExternalTrafficPolicy` | It is used to provide segmentStoreExternalTrafficPolicy  |  |  
+| `segmentStore.service.loadBalancerIP` |It is used to provide a LoadBalancerIP for the segmentStore service | |
+| `segmentStore.service.externalTrafficPolicy` | It is used to provide ExternalTrafficPolicy for the segmentStore service |  |
 | `segmentStore.jvmOptions` | JVM Options for segmentStore | `[]` |
 | `storage.longtermStorage.type` | Type of long term storage backend to be used (filesystem/ecs/hdfs) | `filesystem` |
 | `storage.longtermStorage.filesystem.pvc` | Name of the pre-created PVC, if long term storage type is filesystem | `pravega-tier2` |

charts/pravega/templates/pravega.yaml

@@ -9,9 +9,15 @@ spec:
   {{- if .Values.tls }}
   tls:
     static:
+      {{- if .Values.tls.secret.controller }}
       controllerSecret: {{ .Values.tls.secret.controller }}
+      {{- end }}
+      {{- if .Values.tls.secret.segmentStore }}
       segmentStoreSecret: {{ .Values.tls.secret.segmentStore }}
+      {{- end }}
+      {{- if .Values.tls.caBundle }}
       caBundle: {{ .Values.tls.caBundle }}
+      {{- end }}
   {{- end }}
   authentication:
     enabled: {{ .Values.authentication.enabled }}
@@ -33,10 +39,12 @@ spec:
     {{- if .Values.externalAccess.enabled }}
     controllerServiceAccountName: {{ .Values.serviceAccount.name }}
     segmentStoreServiceAccountName: {{ .Values.serviceAccount.name }}
-    {{- if .Values.segmentStore.service.segmentStoreLoadBalancerIP }}
-    segmentStoreLoadBalancerIP: {{ .Values.segmentStore.service.segmentStoreLoadBalancerIP  }}
+    {{- if .Values.segmentStore.service.loadBalancerIP }}
+    segmentStoreLoadBalancerIP: {{ .Values.segmentStore.service.loadBalancerIP  }}
+    {{- end }}
+    {{- if .Values.segmentStore.service.externalTrafficPolicy }}
+    segmentStoreExternalTrafficPolicy:  {{ .Values.segmentStore.service.externalTrafficPolicy }}
     {{- end }}
-    segmentStoreExternalTrafficPolicy:  {{ .Values.segmentStore.service.segmentStoreExternalTrafficPolicy }}
     {{- if .Values.controller.service.type }}
     controllerExtServiceType: {{ .Values.controller.service.type }}
     {{- end }}
@@ -70,11 +78,9 @@ spec:
     {{- end }}
     {{- if .Values.segmentStore.secret }}
     segmentStoreSecret:
-      {{- if .Values.segmentStore.secret.name }}
-      secret: {{ .Values.segmentStore.name }}
-      {{- end }}
+      secret: {{ .Values.segmentStore.secret.name }}
       {{- if .Values.segmentStore.secret.path }}
-      mountPath: {{ .Values.segmentStore.path }}
+      mountPath: {{ .Values.segmentStore.secret.path }}
       {{- end }}
     {{- end }}
     {{- if .Values.controller.jvmOptions }}
@@ -86,28 +92,34 @@ spec:
 {{ toYaml .Values.segmentStore.jvmOptions | indent 6 }}
     {{- end }}
     debugLogging: {{ .Values.debugLogging }}
+    {{- if .Values.storage.cache }}
     cacheVolumeClaimTemplate:
       accessModes: [ "ReadWriteOnce" ]
+      {{- if .Values.storage.cache.className }}
       storageClassName: {{ .Values.storage.cache.className }}
+      {{- end }}
+      {{- if .Values.storage.cache.size }}
       resources:
         requests:
           storage: {{ .Values.storage.cache.size }}
+      {{- end }}
+    {{- end }}
     longtermStorage:
-      {{- if eq .Values.storage.longtermStorage.type "ecs"}}
+      {{- if eq (default .Values.storage.longtermStorage.type "filesystem") "ecs" }}
       ecs:
         configUri: {{ .Values.storage.longtermStorage.ecs.configUri }}
         bucket: {{ .Values.storage.longtermStorage.ecs.bucket }}
         prefix: {{ .Values.storage.longtermStorage.ecs.prefix }}
         credentials: {{ .Values.storage.longtermStorage.ecs.credentials }}
-      {{- else if eq .Values.storage.longtermStorage.type "hdfs"}}
+      {{- else if eq (default .Values.storage.longtermStorage.type "filesystem") "hdfs"}}
       hdfs:
         uri: {{ .Values.storage.longtermStorage.hdfs.uri }}
         root: {{ .Values.storage.longtermStorage.hdfs.root }}
         replicationFactor: {{ .Values.storage.longtermStorage.hdfs.replicationFactor }}
       {{- else }}
       filesystem:
         persistentVolumeClaim:
-          claimName: {{ .Values.storage.longtermStorage.filesystem.pvc }}
+          claimName: {{ default "pravega-tier2" .Values.storage.longtermStorage.filesystem.pvc }}
       {{- end }}
     {{- if .Values.options }}
     options: