Update README

README.md

@@ -1,52 +1,75 @@
-# CSI Snapshotter
+# CSI Snapshot Feature
 
-The CSI external-snapshotter is part of Kubernetes implementation of [Container Storage Interface (CSI)](https://github.com/container-storage-interface/spec).
+The CSI snapshot feature is part of Kubernetes implementation of [Container Storage Interface (CSI)](https://github.com/container-storage-interface/spec).
 
-The volume snapshot feature supports CSI v1.0 and it has been an Alpha feature in Kubernetes since v1.12.
+The volume snapshot feature supports CSI v1.0 and higher. It was introduced as an Alpha feature in Kubernetes v1.12 and has been promoted to an Beta feature in Kubernetes 1.17.
 
 ## Overview
 
-CSI Snapshotter is an external controller that watches Kubernetes Snapshot CRD objects and triggers CreateSnapshot/DeleteSnapshot against a CSI endpoint. Full design can be found at Kubernetes proposal at [here](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/csi-snapshot.md)
+With the promotion of Volume Snapshot to beta, the feature is now enabled by default on standard Kubernetes deployments instead of being opt-in.
 
-## Design
+The move of the Kubernetes Volume Snapshot feature to beta also means:
+* A revamp of volume snapshot APIs.
+* The CSI external-snapshotter sidecar is split into two controllers, a snapshot controller and a CSI external-snapshotter sidecar.
 
-External snapshotter follows [controller](https://github.com/kubernetes/community/blob/master/contributors/devel/controllers.md) pattern and uses informers to watch for `VolumeSnapshot` and `VolumeSnapshotContent` create/update/delete events. It filters out these objects with `Snapshotter==<CSI driver name>` specified in the associated VolumeSnapshotClass object and then processes these events in workqueues with exponential backoff.
+The snapshot controller is deployed by the Kubernetes distributions and is responsible for watching the VolumeSnapshot CRD objects.
 
-### Snapshotter
+The CSI external-snapshotter sidecar watches Kubernetes VolumeSnapshotContent CRD objects and triggers CreateSnapshot/DeleteSnapshot against a CSI endpoint.
 
-Snapshotter talks to CSI over socket (/run/csi/socket by default, configurable by -csi-address). The snapshotter then:
+Blog post for the beta feature can be found [here](https://kubernetes.io/blog/2019/12/09/kubernetes-1-17-feature-cis-volume-snapshot-beta)
 
-* Discovers the supported snapshotter name by `GetDriverName` call.
+## Controller Workflow
 
-* Uses ControllerGetCapabilities for find out if CSI driver supports `ControllerServiceCapability_RPC_CREATE_DELETE_SNAPSHOT` and `ControllerServiceCapability_RPC_LIST_SNAPSHOTS` calls. Otherwise, the controller will not start.
+Both the snapshot controller and CSI external-snapshotter sidecar follow [controller](https://github.com/kubernetes/community/blob/master/contributors/devel/controllers.md) pattern and uses informers to watch for events. The snapshot controller watches for `VolumeSnapshot` and `VolumeSnapshotContent` create/update/delete events.
 
-* Processes new/updated/deleted `VolumeSnapshots`: The snapshotter only processes `VolumeSnapshot` that has `snapshotter` specified in its `VolumeSnapshotClass` matches its driver name. The process workflow is as follows
-  * If the snapshot status is `Ready`, the controller checks whether the snapshot and its content still binds correctly. If there is any problem with the binding (e.g., snapshot points to a non-exist snapshot content), update the snapshot status and emit event.
-  * If the snapshot status is not ready, there are two cases.
-    * `SnapshotContentName` is not empty: the controller verifies whether the snapshot content exists and also binds to the snapshot. If verification passes, the controller binds the snapshot and its content objects and marks it is ready. Otherwise, it updates the error status of the snapshot.
-    * `SnapshotContentName` is set empty: the controller will first check whether there is already a content object which binds the snapshot correctly with snapshot uid (`VolumeSnapshotRef.UID`) specified. If so, the controller binds these two objects. Otherwise, the controller issues a create snapshot operation. Please note that if the error status shows that snapshot creation already failed before, it will not try to create snapshot again.
+The CSI external-snapshotter sidecar only watches for `VolumeSnapshotContent` create/update/delete events. It filters out these objects with `Driver==<CSI driver name>` specified in the associated VolumeSnapshotClass object and then processes these events in workqueues with exponential backoff.
 
-* Processes new/updated/deleted `VolumeSnapshotContents`: The snapshotter only processes `VolumeSnapshotContent` in which the CSI driver specified in the spec matches the controller's driver name.
-  * If the `VolumeSnapshotRef` is set to nil, skip this content since it is not bound to any snapshot object.
-  * Otherwise, the controller verifies whether the content object is correctly bound to a snapshot object. In case the `VolumeSnapshotRef.UID` is set but it does not match its snapshot object or snapshot no long exists, the content object and its associated snapshot will be deleted.
+The CSI external-snapshotter sidecar talks to CSI over socket (/run/csi/socket by default, configurable by -csi-address).
 
-## Usage
+### Dynamically Creating a New Snapshot
 
-### Running on command line
+The VolumeSnapshot object must specify the following source type: persistentVolumeClaimName - The name of the PVC to snapshot. Please note that the source PVC, PV, and VolumeSnapshotClass for a VolumeSnapshot object must point to the same CSI driver.
 
-For debugging, it is possible to run snapshotter on command line. For example,
+When volume snapshot creation is invoked, the snapshot controller first creates a VolumeSnapshotContent object with the volumeSnapshotRef, source volumeHandle, volumeSnapshotClassName if specified, driver, and deletionPolicy.
 
-```bash
-csi-snapshotter -kubeconfig ~/.kube/config -v 5 -csi-address /run/csi/socket
-```
+The CSI external-snapshotter sidecar then passes the VolumeSnapshotClass parameters, the source volume ID, and any referenced secret(s) to the CSI driver via a CSI CreateSnapshot call. In response, the CSI driver creates a new snapshot for the specified volume, and returns the ID for that snapshot. The CSI external-snapshotter sidecar then updates the snapshotHandle, creationTime, restoreSize, and readyToUse in the status field of the VolumeSnapshotContent object that represents the new snapshot. For a storage system that needs to upload the snapshot after it is being cut, the CSI external-snapshotter sidecar will keep calling the CSI CreateSnapshot to check the status until upload is complete and readyToUse is true.
 
-### Running in a statefulset
+The snapshot controller binds the VolumeSnapshotContent object to the VolumeSnapshot (sets BoundVolumeSnapshotContentName), updating the creationTime, restoreSize, and readyToUse in the status field of the VolumeSnapshot object based on the status field of the VolumeSnapshotContent object.
 
-It is necessary to create a new service account and give it enough privileges to run the snapshotter. We provide .yaml files that deploy for use together with the hostpath example driver. A real production deployment must customize them:
+If no volumeSnapshotClassName is specified, one is automatically selected as follows: The StorageClass from PVC or PV of the source volume is fetched. The default VolumeSnapshotClass is fetched, if available. A default VolumeSnapshotClass is a snapshot class created by the admin with the snapshot.storage.kubernetes.io/is-default-class annotation. If the Driver field of the default VolumeSnapshotClass is the same as the Provisioner field in the StorageClass, the default VolumeSnapshotClass is used. If there is no default VolumeSnapshotClass or more than one default VolumeSnapshotClass for a snapshot, an error will be returned.
+
+### Importing an Existing Snapshot
+
+You can always expose a pre-existing volume snapshot in Kubernetes by manually creating a VolumeSnapshotContent object to represent the existing volume snapshot. Because VolumeSnapshotContent is a non-namespace API object, only a cluster admin may have the permission to create it. By specifying the volumeSnapshotRef the cluster admin specifies exactly which user can use the snapshot.
+
+Once a VolumeSnapshotContent object is created, a user can create a VolumeSnapshot object pointing to the VolumeSnapshotContent object. The name and namespace of the VolumeSnapshot object must match the name/namespace specified in the volumeSnapshotRef of the VolumeSnapshotContent. It specifies the following fields: volumeSnapshotContentName - name of the volume snapshot content specified above. This field is required. volumeSnapshotClassName - name of the volume snapshot class. This field is optional.
+
+Once both objects are created, the snapshot controller verifies the binding between VolumeSnapshot and VolumeSnapshotContent objects is correct and marks the VolumeSnapshot as ready (if the CSI driver supports the ListSnapshots call, the controller also validates that the referenced snapshot exists). The CSI external-snapshotter sidecar checks if the snapshot exists if ListSnapshots CSI method is implemented, otherwise it assumes the snapshot exists. The external-snapshotter sidecar sets readyToUse to true in the status field of VolumeSnapshotContent. The snapshot controller marks the snapshot as ready accordingly.
+
+## Deployment
+
+The Volume Snapshot feature now depends on a new, volume snapshot controller in addition to the volume snapshot CRDs. Both the volume snapshot controller and the CRDs are independent of any CSI driver. Regardless of the number CSI drivers deployed on the cluster, there must be only one instance of the volume snapshot controller running and one set of volume snapshot CRDs installed per cluster.
+
+Therefore, it is strongly recommended that Kubernetes distributors bundle and deploy the controller and CRDs as part of their Kubernetes cluster management process (independent of any CSI Driver).
+
+If your cluster does not come pre-installed with the correct components, you may manually install these components by executing the following steps.
+
+Install Snapshot Beta CRDs:
+* kubectl create -f config/crd
+* https://github.com/kubernetes-csi/external-snapshotter/tree/master/config/crd
+* Do this once per cluster
+
+Install Common Snapshot Controller:
+* kubectl create -f deploy/kubernetes/snapshot-controller
+* https://github.com/kubernetes-csi/external-snapshotter/tree/master/deploy/kubernetes/snapshot-controller
+* Do this once per cluster
+
+Install CSI Driver:
+* Follow instructions provided by your CSI Driver vendor.
+* Here is an example to install the sample hostpath CSI driver
+  * kubectl create -f deploy/kubernetes/csi-snapshotter
+  * https://github.com/kubernetes-csi/external-snapshotter/tree/master/deploy/kubernetes/csi-snapshotter
 
-```bash
-for i in $(find deploy/kubernetes -name '*.yaml'); do kubectl create -f $i; done
-```
 
 ### Running with Leader Election
 
@@ -55,12 +78,24 @@ If you want to run external-snapshotter with higher availability, you can enable
 --leader-election=true
 ```
 
+### Upgrade from Alpha to Beta
+
+The change from Alpha to Beta snapshot APIs is not backward compatible.
+
+If you have already deployed Alpha snapshot APIs and controller and want to upgrade to Beta, you need to do the following:
+* Delete snapshots created using Alpha snapshot CRDs and controller.
+* Uninstall Alpha snapshot CRDs, controller, and CSI driver.
+* Install Beta snapshot CRDs, controller, and CSI driver.
+
+
 ## Testing
 
 Running Unit Tests:
 
 ```bash
-go test -timeout 30s  github.com/kubernetes-csi/external-snapshotter/pkg/controller
+go test -timeout 30s  github.com/kubernetes-csi/external-snapshotter/pkg/common-controller
+
+go test -timeout 30s  github.com/kubernetes-csi/external-snapshotter/pkg/sidecar-controller
 ```
 
 ## Dependency Management