From b6f7d0ebda0c91156633e87a0972c91f8fdf18c4 Mon Sep 17 00:00:00 2001 From: Fernando Alfaro Campos Date: Tue, 14 Feb 2023 13:27:28 -0500 Subject: [PATCH] Powerflex Replication (#445) * Add PowerFlex replication documentation * Update PowerFlex replication docs * Fix documentation * Updated compatible versions --- content/docs/replication/_index.md | 47 +-- .../replication/deployment/installation.md | 3 +- .../docs/replication/deployment/powerflex.md | 268 ++++++++++++++++++ .../replication/deployment/storageclasses.md | 2 +- content/docs/replication/release/_index.md | 1 + .../docs/replication/replication-actions.md | 20 +- 6 files changed, 306 insertions(+), 35 deletions(-) create mode 100644 content/docs/replication/deployment/powerflex.md diff --git a/content/docs/replication/_index.md b/content/docs/replication/_index.md index c926dd7266..3602e5c9ea 100644 --- a/content/docs/replication/_index.md +++ b/content/docs/replication/_index.md @@ -18,40 +18,40 @@ CSM for Replication provides the following capabilities: {{}} | Capability | PowerMax | PowerStore | PowerScale | PowerFlex | Unity | | ----------------------------------------------------------------------------------------------------------------------------------- | :------: | :--------: | :--------: | :-------: | :---: | -| Replicate data using native storage array based replication | yes | yes | yes | no | no | +| Replicate data using native storage array based replication | yes | yes | yes | yes | no | | Asynchronous file volume replication | yes | no | yes | no | no | -| Asynchronous block volume replication | yes | yes | n/a | no | no | +| Asynchronous block volume replication | yes | yes | n/a | yes | no | | Synchronous file volume replication | yes | no | no | no | no | | Synchronous block volume replication | yes | no | n/a | no | no | | Active-Active (Metro) block volume replication | yes | no | n/a | no | no | | Active-Active (Metro) file volume replication | yes | no | no | no | no | -| Create `PersistentVolume` objects in the cluster representing the replicated volume | yes | yes | yes | no | no | -| Create `DellCSIReplicationGroup` objects in the cluster | yes | yes | yes | no | no | -| Failover & Reprotect applications using the replicated volumes | yes | yes | yes | no | no | -| Online Volume Expansion for replicated volumes | yes | no | no | no | no | -| Provides a command line utility - [repctl](tools) for configuring & managing replication related resources across multiple clusters | yes | yes | yes | no | no | +| Create `PersistentVolume` objects in the cluster representing the replicated volume | yes | yes | yes | yes | no | +| Create `DellCSIReplicationGroup` objects in the cluster | yes | yes | yes | yes | no | +| Failover & Reprotect applications using the replicated volumes | yes | yes | yes | yes | no | +| Online Volume Expansion for replicated volumes | yes | no | no | yes | no | +| Provides a command line utility - [repctl](tools) for configuring & managing replication related resources across multiple clusters | yes | yes | yes | yes | no | {{
}} ## Supported Operating Systems/Container Orchestrator Platforms {{}} -| COP/OS | PowerMax | PowerStore | PowerScale | -| ----------------- | ---------------- | ---------------- | ---------------- | -| Kubernetes | 1.24, 1.25, 1.26 | 1.24, 1.25, 1.26 | 1.24, 1.25, 1.26 | -| Red Hat OpenShift | 4.10, 4.11 | 4.9, 4.10 | 4.9, 4.10 | -| RHEL | 7.x, 8.x | 7.x, 8.x | 7.x, 8.x | -| CentOS | 7.8, 7.9 | 7.8, 7.9 | 7.8, 7.9 | -| Ubuntu | 20.04 | 20.04 | 20.04 | -| SLES | 15SP4 | 15SP2 | 15SP2 | +| COP/OS | PowerMax | PowerStore | PowerScale | PowerFlex | +| ----------------- | ---------------- | ---------------- | ---------------- | ---------------- | +| Kubernetes | 1.24, 1.25, 1.26 | 1.24, 1.25, 1.26 | 1.24, 1.25, 1.26 | 1.24, 1.25, 1.26 | +| Red Hat OpenShift | 4.10, 4.11 | 4.9, 4.10 | 4.9, 4.10 | 4.10, 4.11 | +| RHEL | 7.x, 8.x | 7.x, 8.x | 7.x, 8.x | 7.x, 8.x | +| CentOS | 7.8, 7.9 | 7.8, 7.9 | 7.8, 7.9 | 7.8, 7.9 | +| Ubuntu | 20.04 | 20.04 | 20.04 | 20.04 | +| SLES | 15SP4 | 15SP2 | 15SP2 | 15SP3 | {{
}} ## Supported Storage Platforms {{}} -| | PowerMax | PowerStore | PowerScale | -| ------------- | :------------------------------------------------------------------------------------------------------: | :-----------------: | :--------------------------------: | -| Storage Array | PowerMax 2000/8000
PowerMax 2500/8500
5978.479.479, 5978.711.711, 6079.xxx.xxx, Unisphere 10.0 | 1.0.x, 2.0.x, 2.1.x | OneFS 8.1, 8.2, 9.0, 9.1, 9.2, 9.3 | +| | PowerMax | PowerStore | PowerScale | PowerFlex | +| ------------- | :------------------------------------------------------------------------------------------------------: | :-----------------: | :--------------------------------: | :-------: | +| Storage Array | PowerMax 2000/8000
PowerMax 2500/8500
5978.479.479, 5978.711.711, 6079.xxx.xxx, Unisphere 10.0 | 1.0.x, 2.0.x, 2.1.x | OneFS 8.1, 8.2, 9.0, 9.1, 9.2, 9.3 | 3.6.x, 4.0 | {{
}} ## Supported CSI Drivers @@ -63,6 +63,7 @@ CSM for Replication supports the following CSI drivers and versions. | CSI Driver for Dell PowerMax | [csi-powermax](https://github.com/dell/csi-powermax) | v2.0 + | | CSI Driver for Dell PowerStore | [csi-powerstore](https://github.com/dell/csi-powerstore) | v2.0 + | | CSI Driver for Dell PowerScale | [csi-powerscale](https://github.com/dell/csi-powerscale) | v2.2 + | +| CSI Driver for Dell PowerFlex | [csi-powerflex](https://github.com/dell/csi-powerflex) | v2.6 + | {{}} ## Details @@ -89,11 +90,11 @@ the objects still exist in pairs. The following matrix provides a list of all supported versions for each Dell Storage product. -| Platforms | PowerMax | PowerStore | PowerScale | -| ---------------- | ------------------------------ | ---------------- | ---------------- | -| Kubernetes | 1.24, 1.25, 1.26 | 1.24, 1.25, 1.26 | 1.24, 1.25, 1.26 | -| RedHat Openshift | 4.10, 4.11 | 4.9, 4.10 | 4.9, 4.10 | -| CSI Driver | 2.x(k8s),
2.2+(OpenShift) | 2.x | 2.2+ | +| Platforms | PowerMax | PowerStore | PowerScale | PowerFlex | +| ---------------- | ------------------------------ | ---------------- | ---------------- | ---------------- | +| Kubernetes | 1.24, 1.25, 1.26 | 1.24, 1.25, 1.26 | 1.24, 1.25, 1.26 | 1.24, 1.25, 1.26 | +| RedHat Openshift | 4.10, 4.11 | 4.9, 4.10 | 4.9, 4.10 | 4.10, 4.11 | +| CSI Driver | 2.x(k8s),
2.2+(OpenShift) | 2.x | 2.2+ | 2.6+ | For compatibility with storage arrays please refer to corresponding [CSI drivers](../csidriver/#features-and-capabilities) diff --git a/content/docs/replication/deployment/installation.md b/content/docs/replication/deployment/installation.md index 005637fac7..88f21ae2e3 100644 --- a/content/docs/replication/deployment/installation.md +++ b/content/docs/replication/deployment/installation.md @@ -75,8 +75,9 @@ The following CSI drivers support replication: 1. CSI driver for PowerMax 2. CSI driver for PowerStore 3. CSI driver for PowerScale +4. CSI driver for PowerFlex -Please follow the steps outlined in [PowerMax](../powermax), [PowerStore](../powerstore) or [PowerScale](../powerscale) pages during the driver installation. +Please follow the steps outlined in [PowerMax](../powermax), [PowerStore](../powerstore), [PowerScale](../powerscale), or [PowerFlex](../powerflex) pages during the driver installation. >Note: Please ensure that replication CRDs are installed in the clusters where you are installing the CSI drivers. These CRDs are generally installed as part of the CSM Replication controller installation process. diff --git a/content/docs/replication/deployment/powerflex.md b/content/docs/replication/deployment/powerflex.md new file mode 100644 index 0000000000..e7755154ae --- /dev/null +++ b/content/docs/replication/deployment/powerflex.md @@ -0,0 +1,268 @@ +--- +title: PowerFlex +linktitle: PowerFlex +weight: 6 +description: Enabling Replication feature for CSI PowerFlex +--- +## Enabling Replication In CSI PowerFlex + +Container Storage Modules (CSM) Replication sidecar is a helper container that +is installed alongside a CSI driver to facilitate replication functionality. +Such CSI drivers must implement `dell-csi-extensions` calls. + +CSI driver for Dell PowerFlex supports necessary extension calls from +`dell-csi-extensions`. To be able to provision replicated volumes you would need +to do the steps described in the following sections. + +### Before Installation + +#### On Storage Array + +Be sure to configure replication between multiple PowerFlex instances using instructions provided by PowerFlex storage. + +Ensure that the remote systems are configured by navigating to the `Protection` tab and choosing `Peer Systems` in the UI of the PowerFlex instance. + +There should be a list of remote systems with the `State` fields set to `Connected`. + +#### In Kubernetes +Ensure you installed CRDs and replication controller in your clusters. + +Run the following commands to verify that everything is installed correctly: + +* Check controller pods + ```shell + kubectl get pods -n dell-replication-controller + ``` + Pods should be `READY` and `RUNNING` +* Check that the controller config map is properly populated + ```shell + kubectl get cm -n dell-replication-controller dell-replication-controller-config -o yaml + ``` + `data` field should be properly populated with cluster-id of your choosing + and, if using multi-cluster installation, your `targets:` parameter should be + populated by a list of target cluster IDs. + + +If you don't have something installed or something is out-of-place, please refer +to installation instructions in [installation-repctl](../install-repctl) or +[installation](../installation). + +### Installing Driver With Replication Module + +To install the driver with replication enabled, you need to ensure you have set +helm parameter `replication.enabled` in your copy of example `values.yaml` file +(usually called `my-powerflex-settings.yaml`, `myvalues.yaml` etc.). + +Here is an example of how that would look +```yaml +... +# Set this to true to enable replication +replication: + enabled: true + image: dellemc/dell-csi-replicator:v1.2.0 + replicationContextPrefix: "powerflex" + replicationPrefix: "replication.storage.dell.com" +... +``` +You can leave other parameters like `image`, `replicationContextPrefix`, and +`replicationPrefix` as they are. + +After enabling the replication module you can continue to install the CSI driver +for PowerFlex following the usual installation procedure, just ensure you've added +necessary array connection information to secret. + +> **_NOTE:_** you need to install your driver at least on the source cluster, +> but it is recommended to install drivers on all clusters you will use for +> replication. + +### Creating Storage Classes + +To be able to provision replicated volumes you need to create properly +configured storage classes on both source and target clusters. + +Pair of storage classes on the source and target clusters would be essentially +`mirrored` copies of one another. You can create them manually or with help from +`repctl`. + +#### Manual Storage Class Creation + +You can find a sample of a replication enabled storage class in the driver repository +[here](https://github.com/dell/csi-powerflex/blob/main/samples/storageclass/vxflexos-replication.yaml). + +It will look like this: +```yaml +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: vxflexos-replication +provisioner: csi-vxflexos.dellemc.com +reclaimPolicy: Retain +allowVolumeExpansion: true +volumeBindingMode: Immediate +parameters: + replication.storage.dell.com/isReplicationEnabled: "true" + replication.storage.dell.com/remoteStorageClassName: "vxflexos-replication" + replication.storage.dell.com/remoteClusterID: + replication.storage.dell.com/remoteSystem: + replication.storage.dell.com/remoteStoragePool: + replication.storage.dell.com/rpo: 60 + replication.storage.dell.com/volumeGroupPrefix: "csi" + replication.storage.dell.com/consistencyGroupName: + replication.storage.dell.com/protectionDomain: + systemID: + storagepool: + protectiondomain: +``` + +Let's go through each parameter and what it means: +* `replication.storage.dell.com/isReplicationEnabled` if set to `true` will mark + this storage class as replication enabled, just leave it as `true`. +* `replication.storage.dell.com/remoteStorageClassName` points to the name of + the remote storage class. If you are using replication with the multi-cluster + configuration you can make it the same as the current storage class name. +* `replication.storage.dell.com/remoteClusterID` represents ID of a remote + cluster. It is the same id you put in the replication controller config map. +* `replication.storage.dell.com/remoteSystem` is the name of the remote system + as seen from the current PowerFlex instance. This parameter is the systemID of + the array. +* `replication.storage.dell.com/remoteStoragePool` is the name of the storage + pool on the remote system to be used for creating the remote volumes. +* `replication.storage.dell.com/rpo` is an acceptable amount of data, which is + measured in units of time, that may be lost due to a failure. +* `replication.storage.dell.com/volumeGroupPrefix` represents what string would + be appended to the volume group name to differentiate them. +* `replication.storage.dell.com/consistencyGroupName` represents the desired + name to give the consistency group on the PowerFlex array. If omitted, the + driver will generate a name for the consistency group. +* `replication.storage.dell.com/protectionDomain` represents the remote array's + protection domain to use. +* `systemID` represents the systemID of the PowerFlex array. +* `storagepool` represents the name of the storage pool to be used on the + PowerFlex array. +* `protectiondomain` represents the array's protection domain to be used. + +Let's follow up that with an example. Let's assume we have two Kubernetes +clusters and two PowerFlex storage arrays: +* Clusters have IDs of `cluster-1` and `cluster-2` +* Cluster `cluster-1` connected to array `000000000001` +* Cluster `cluster-2` connected to array `000000000002` +* For `cluster-1` we plan to use storage pool `pool1` and protection domain `domain1` +* For `cluster-2` we plan to use storage pool `pool1` and protection domain `domain1` + +And this is how our pair of storage classes would look: + +StorageClass to be created in `cluster-1`: +```yaml +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: "vxflexos-replication" +provisioner: "csi-vxflexos.dellemc.com" +reclaimPolicy: Retain +volumeBindingMode: Immediate +allowVolumeExpansion: true +parameters: + replication.storage.dell.com/isReplicationEnabled: "true" + replication.storage.dell.com/remoteStorageClassName: "vxflexos-replication" + replication.storage.dell.com/remoteClusterID: "cluster-2" + replication.storage.dell.com/remoteSystem: "000000000002" + replication.storage.dell.com/remoteStoragePool: pool1 + replication.storage.dell.com/protectionDomain: domain1 + replication.storage.dell.com/rpo: 60 + replication.storage.dell.com/volumeGroupPrefix: "csi" + arrayID: "000000000001" + storagepool: "pool1" + protectiondomain: "domain1" +``` + +StorageClass to be created in `cluster-2`: +```yaml +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: "vxflexos-replication" +provisioner: "csi-vxflexos.dellemc.com" +reclaimPolicy: Retain +volumeBindingMode: Immediate +allowVolumeExpansion: true +parameters: + replication.storage.dell.com/isReplicationEnabled: "true" + replication.storage.dell.com/remoteStorageClassName: "vxflexos-replication" + replication.storage.dell.com/remoteClusterID: "cluster-1" + replication.storage.dell.com/remoteSystem: "000000000001" + replication.storage.dell.com/remoteStoragePool: pool1 + replication.storage.dell.com/protectionDomain: domain1 + replication.storage.dell.com/rpo: 60 + replication.storage.dell.com/volumeGroupPrefix: "csi" + arrayID: "000000000002" + storagepool: "pool1" + protectiondomain: "domain1" +``` + +After figuring out how storage classes would look, you just need to go and apply +them to your Kubernetes clusters with `kubectl`. + +#### Storage Class Creation With repctl + +`repctl` can simplify storage class creation by creating a pair of mirrored +storage classes in both clusters (using a single storage class configuration) in +one command. + +To create storage classes with `repctl` you need to fill up the config with +the necessary information. You can find an example in +[here](https://github.com/dell/csm-replication/blob/main/repctl/examples/powerflex_example_values.yaml), +copy it, and modify it to your needs. + +If you open this example you can see a lot of similar fields and parameters you +can modify in the storage class. + +Let's use the same example from the manual installation and see how the config would +look +```yaml +sourceClusterID: "cluster-1" +targetClusterID: "cluster-2" +name: "vxflexos-replication" +driver: "vxflexos" +reclaimPolicy: "Retain" +replicationPrefix: "replication.storage.dell.com" +parameters: + storagePool: # populate with storage pool to use of arrays + source: "pool1" + target: "pool1" + protectionDomain: # populate with protection domain to use of arrays + source: "domain1" + target: "domain1" + arrayID: # populate with unique ids of storage arrays + source: "0000000000000001" + target: "0000000000000002" + rpo: "60" + volumeGroupPrefix: "csi" + consistencyGroupName: "" # optional name to be given to the rcg +``` + +After preparing the config you can apply it to both clusters with repctl. Just +make sure you've added your clusters to repctl via the `add` command before. + +To create storage classes just run `./repctl create sc --from-config ` and storage classes would be applied to both clusters. + +After creating storage classes you can make sure they are in place by using +`./repctl get storageclasses` command. + +### Provisioning Replicated Volumes + +After installing the driver and creating storage classes you are good to create +volumes using newly created storage classes. + +On your source cluster, create a PersistentVolumeClaim using one of the +replication enabled Storage Classes. The CSI PowerFlex driver will create a +volume on the array, add it to a VolumeGroup and configure replication using the +parameters provided in the replication enabled Storage Class. + +### Supported Replication Actions +The CSI PowerFlex driver supports the following list of replication actions: +- FAILOVER_REMOTE +- UNPLANNED_FAILOVER_LOCAL +- REPROTECT_LOCAL +- SUSPEND +- RESUME +- SYNC diff --git a/content/docs/replication/deployment/storageclasses.md b/content/docs/replication/deployment/storageclasses.md index df85a44833..aab483bff9 100644 --- a/content/docs/replication/deployment/storageclasses.md +++ b/content/docs/replication/deployment/storageclasses.md @@ -29,7 +29,7 @@ This should contain the name of the storage class on the remote cluster which is >Note: You still need to create a pair of storage classes even while using a single stretched cluster ### Driver specific parameters -Please refer to the driver specific sections for [PowerMax](../powermax/#creating-storage-classes), [PowerStore](../powerstore/#creating-storage-classes) or [PowerScale](../powerscale/#creating-storage-classes) for a detailed list of parameters. +Please refer to the driver specific sections for [PowerMax](../powermax/#creating-storage-classes), [PowerStore](../powerstore/#creating-storage-classes), [PowerScale](../powerscale/#creating-storage-classes) or [PowerFlex](../powerflex#creating-storage-classes) for a detailed list of parameters. ### PV sync Deletion diff --git a/content/docs/replication/release/_index.md b/content/docs/replication/release/_index.md index 1a24ded178..4d01e34196 100644 --- a/content/docs/replication/release/_index.md +++ b/content/docs/replication/release/_index.md @@ -13,6 +13,7 @@ Description: > - [PowerScale - Implement Failback functionality](https://github.com/dell/csm/issues/558) - [PowerScale - Implement Reprotect functionality](https://github.com/dell/csm/issues/532) - [PowerScale - SyncIQ policy improvements](https://github.com/dell/csm/issues/573) + - [PowerFlex - Initial Replication Support](https://github.com/dell/csm/issues/618) ### Fixed Issues diff --git a/content/docs/replication/replication-actions.md b/content/docs/replication/replication-actions.md index 5bc89fccac..0a8fd639eb 100644 --- a/content/docs/replication/replication-actions.md +++ b/content/docs/replication/replication-actions.md @@ -34,11 +34,11 @@ For e.g. - The following table lists details of what actions should be used in different Disaster Recovery workflows & the equivalent operation done on the storage array: {{}} -| Workflow | Actions | PowerMax | PowerStore | PowerScale | -| ------------------- | ----------------------------------------------------- | ---------------------- | -------------------------------------- | ------------------------------------------------ | -| Planned Migration | FAILOVER_LOCAL
FAILOVER_REMOTE | symrdf failover -swap | FAILOVER (no REPROTECT after FAILOVER) | allow_writes on target, disable local policy | -| Reprotect | REPROTECT_LOCAL
REPROTECT_REMOTE | symrdf resume/est | REPROTECT | Delete policy on source, create policy on target | -| Unplanned Migration | UNPLANNED_FAILOVER_LOCAL
UNPLANNED_FAILOVER_REMOTE | symrdf failover -force | FAILOVER (at target site) | allow_writes on target | +| Workflow | Actions | PowerMax | PowerStore | PowerScale | PowerFlex | +| ------------------- | ----------------------------------------------------- | ---------------------- | -------------------------------------- | ------------------------------------------------ | -------------------------------------- | +| Planned Migration | FAILOVER_LOCAL
FAILOVER_REMOTE | symrdf failover -swap | FAILOVER (no REPROTECT after FAILOVER) | allow_writes on target, disable local policy | FAILOVER (no REPROTECT after FAILOVER) | +| Reprotect | REPROTECT_LOCAL
REPROTECT_REMOTE | symrdf resume/est | REPROTECT | Delete policy on source, create policy on target | REPROTECT | +| Unplanned Migration | UNPLANNED_FAILOVER_LOCAL
UNPLANNED_FAILOVER_REMOTE | symrdf failover -force | FAILOVER (at target site) | allow_writes on target | FAILOVER (at target site) | {{
}} ### Maintenance Actions @@ -46,11 +46,11 @@ These actions can be run at any site and are used to change the replication link The following table lists the supported maintenance actions and the equivalent operation done on the storage arrays {{}} -| Action | Description | PowerMax | PowerStore | PowerScale | -| ------- | -------------------------------------------------- | ---------------- | --------------- | -------------------- | -| SUSPEND | Temporarily suspend
replication | symrdf suspend | PAUSE | disable local policy | -| RESUME | Resume replication | symrdf resume | RESUME | enable local policy | -| SYNC | Synchronize all changes
from source to target | symrdf establish | SYNCHRONIZE NOW | start syncIQ job | +| Action | Description | PowerMax | PowerStore | PowerScale | PowerFlex | +| ------- | -------------------------------------------------- | ---------------- | --------------- | -------------------- | --------- | +| SUSPEND | Temporarily suspend
replication | symrdf suspend | PAUSE | disable local policy | PAUSE | +| RESUME | Resume replication | symrdf resume | RESUME | enable local policy | RESUME | +| SYNC | Synchronize all changes
from source to target | symrdf establish | SYNCHRONIZE NOW | start syncIQ job | SYNC NOW | {{
}} ### How to perform actions