[cinder-csi-plugin] Programatically create cinder backups from Kubernetes via CSI

[Sebastian-RG]

Is this a BUG REPORT or FEATURE REQUEST?:
/kind feature

What happened:
Currently, there is no method to programmatically create volume backups from K8S.
This disallows for Kubernetes administrators to easily make sure their data is redundantly stored.

What you expected to happen:

I believe the best option would be to do something similar to what Longhorn does.
By setting the value on the volumesnapshotclass' parameters.type to snapshot or backup, you can choose between linking
the volumesnapshot object to a volume snapshot or a backup.

kind: VolumeSnapshotClass
apiVersion: snapshot.storage.k8s.io/v1
driver: driver.longhorn.io
deletionPolicy: Delete
parameters:
  type: <type>

How to reproduce it:
Not applicable

Anything else we need to know?:
I would love to contribute the necessary changes to the project

Environment:

• openstack-cloud-controller-manager(or other related binary) version:
• OpenStack version: openstack 6.3.0
• Others:

[zetaab]

https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/cinder-csi-plugin/features.md#volume-snapshots is this what you are looking for?

[zetaab]

conversation in slack: currently cpo supports volume snapshots. Instead we could have also volume backups.

So the full workflows could be:

take snapshot:

1. take volume snapshot from cinder volume
2. make volume backup using cinder volume snapshot (done in 1.)
3. delete volume snapshot

restore snapshot:

1. restore volume backup

just in case: what is difference between volume backup and volume snapshot?

The difference is that volume snapshot is stored in block storage and volume backup can be stored to object storage (swift/s3 etc).

[jichenjc]

just in case: what is difference between volume backup and volume snapshot?
The difference is that volume snapshot is stored in block storage and volume backup can be stored to object storage (swift/s3 etc).

I knew nova backup is different to nova snapshot, but not sure volume snapshot/backup
does it means we need change existing API call to cinder ? or need CSI level design (e.g backup differnet to snapshot) ?

[kayrus]

backup has its own API: https://docs.openstack.org/api-ref/block-storage/v3/index.html#backups-backups
I recently implemented it in velero openstack plugin: Lirt/velero-plugin-for-openstack#89

[jichenjc]

@Sebastian-RG will you submit a PR for this (with the above info we got from @zetaab and @kayrus )?

[Sebastian-RG]

Hi! Yes, I will submit it tomorrow

[Sebastian-RG]

Hi. I've been having some issues with creating a volume from a backup. The flow goes like this:

1. A PVC is created that points to a VolumeSnapshot with type backup.
2. cinder-csi-plugin creates a volume that points to a backup with the gophercloud API.
3. Volume creation hits this this in cinder-volume.
4. Volume gets created, then cinder-volume tells cinder-backup to copy the data from the backup to the newly created volume
5. The data from the backup gets copied to the new volume
6. cinder-csi-plugin for some reason decides to delete the volume
7. Backup tries to restore the data again to the volume, but it does not exist anymore. Backup gets stuck in state Restoring

If anyone has any idea why the last part happens I would be very grateful for the help. I've been stuck on this bit for almost a week 😓

Some things I think may cause this:

1. GRPC repeatedly calling create volume somehow causes an issue. Probably because the backup gets set to available at the very beginning and then changes status when the backup starts restoring.
2. Some other component (csi-provisioner or external snapshotter) deciding to delete the volume because there isn't a correponding snapshot in cinder

This is my current code master...Sebastian-RG:cloud-provider-openstack:master

[Sebastian-RG]

As it turns out, when you create a volume from a backup via Gophercloud, the returned volume object does not include the backupID. This caused the ContentSource field in CreateVolumeResponse to come back empty. This eventually caused external-snapshotter to delete the volume.

Sending the PR now :)