From 5a22c0f0faae659b782dc3adb91d1e4cdbb31022 Mon Sep 17 00:00:00 2001 From: carlory Date: Tue, 11 Jun 2024 15:32:27 +0800 Subject: [PATCH] update VolumeAttributesClass Deletion Protection --- .../3751-volume-attributes-class/README.md | 42 ++++++++----------- 1 file changed, 17 insertions(+), 25 deletions(-) diff --git a/keps/sig-storage/3751-volume-attributes-class/README.md b/keps/sig-storage/3751-volume-attributes-class/README.md index a2c2b0c8ffa..4e210760d0e 100644 --- a/keps/sig-storage/3751-volume-attributes-class/README.md +++ b/keps/sig-storage/3751-volume-attributes-class/README.md @@ -437,46 +437,38 @@ Operation metrics from [csiOperationsLatencyMetric](https://github.com/kubernete #### VolumeAttributesClass Deletion Protection -While a VolumeAttributesClass is referenced by any PVC, we will prevent the object from being deleted by adding a finalizer([reference](https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/admission/storage/storageobjectinuseprotection/admission.go)). +While a VolumeAttributesClass is referenced by any PVC, we will prevent the object from being deleted by adding a finalizer `kubernetes.io/vac-protection`. It's a best effort to prevent users from making mistakes. It may not be accurate in all cases. -The **vac_finalizer_controller** sets/removes the finalizer on all VolumeAttributesClasses. This prevents users from deleting a VolumeAttributesClass that's used by a PVC. The vac_finalizer_controller will maintain a cache of a map of VolumeAttributesClass to the list of PVCs that are using the VolumeAttributesClass. +The **vac_finalizer_controller** sets/removes the finalizer on all VolumeAttributesClasses. It has PVC/PV informers with a custom indexer which is used to list all PVCs/PVs for a given VolumeAttributesClass name. -There are a few conditions that will trigger add/remove pvc finalizers in the VolumeAttributesClass: +There are a few conditions that will trigger add/remove the finalizer in the VolumeAttributesClass: 1. PVC created with a VolumeAttributesClass The **vac_finalizer_controller**: - * If the VolumeAttributesClassName is nil or empty, the VolumeAttributesClass will not be added to the vac_finalizer_controller cache - * Check if the VolumeAttributesClass exists. If not, the VolumeAttributesClass will not be added to the vac_finalizer_controller cache + * If the VolumeAttributesClassName is nil or empty, do nothing. + * Check if the VolumeAttributesClass exists. If not, do nothing. * Check if this VolumeAttributesClass already has a protection finalizer * Add the finalizer to the VolumeAttributesClass if there is none -2. PVC created with a VolumeAttributesClass being deleted - The **vac_finalizer_controller**: - * Check VolumeAttributesClass is being deleted and PVC creation failed -3. PVC updated to a different VolumeAttributesClass - * The **vac_finalizer_controller** will remove finalizer in the VolumeAttributesClass only if after listing all the PVCs/PVs and confirm that this PVC/PV is the last one that is consuming the VolumeAttributesClass in the vac_finalizer_controller cache +2. PVC updated to a different VolumeAttributesClass + * The **vac_finalizer_controller** will remove finalizer in the VolumeAttributesClass only if after listing all the PVCs/PVs from informers via a custom indexer and confirm that this PVC/PV is the last one that is consuming the VolumeAttributesClass in the vac_finalizer_controller cache * Check if the new VolumeAttributesClass already has a protection finalizer - * Add the finalizer to the new VolumeAttributesClass if there is none -4. PVC updated to a different VolumeAttributesClass that is being deleted - The **vac_finalizer_controller**: - * Check VolumeAttributesClass is being deleted and PVC update failed -5. PVC has a VolumeAttributesClass and this PVC is deleted - * The **vac_finalizer_controller** will remove finalizer in the VolumeAttributesClass only if after listing all the PVCs/PVs and confirm that this PVC/PV is the last one that is consuming the VolumeAttributesClass in the informer(a cache of VolumeAttributesClass) **only** -6. Delete a VolumeAttributesClass while there is **kubernetes.io/vac-protection** finalizer associated with this VolumeAttributesClass - * Deletion will not return an error but it will add a deletionTimestamp and wait for the finalizer being removed, then remove the VolumeAttributesClass -7. Delete a VolumeAttributesClass without any finalizers - * Deletion succeed + * Add the finalizer to the new VolumeAttributesClass if there is nones +3. PVC has a VolumeAttributesClass and this PVC is deleted + * The **vac_finalizer_controller** will remove finalizer in the VolumeAttributesClass only if after listing all the PVCs/PVs from informers via a custom indexer and confirm that this PVC/PV is the last one that is consuming the VolumeAttributesClass in the informer(a cache of VolumeAttributesClass) **only** +4. Delete a VolumeAttributesClass while there is **kubernetes.io/vac-protection** finalizer associated with this VolumeAttributesClass + * Remove the finalizer only if there is no PVC/PV using this VolumeAttributesClass anymore. The check logic is the same as above. -For unbound PVs referencing a VAC: +For PVs referencing a VAC: 1. Unbound PV created with a VolumeAttributesClass The **vac_finalizer_controller**: - * If the VolumeAttributesClassName is nil or empty, the VolumeAttributesClass will not be added to the vac_finalizer_controller cache + * If the VolumeAttributesClassName is nil or empty, do nothing * Check if this VolumeAttributesClass already has a protection finalizer * Add the finalizer to the VolumeAttributesClass if there is none 2. PV has a VolumeAttributesClass and this PV is deleted - * The **vac_finalizer_controller** will remove finalizer in the VolumeAttributesClass only if after listing all the PVCs/PVs and confirm that this PVC/PV is the last one that is consuming the VolumeAttributesClass in the informer(a cache of VolumeAttributesClass) **only** + * The **vac_finalizer_controller** will remove finalizer in the VolumeAttributesClass only if after listing all the PVCs/PVs from informers via a custom indexer and confirm that this PVC/PV is the last one that is consuming the VolumeAttributesClass in the informer(a cache of VolumeAttributesClass) **only** -Only the **vac_finalizer_controller** will remove finalizers on VolumeAttributesClass. If the **vac_finalizer_controller** fails at the step of removing finalizer even there is no PVC/PV using the VolumeAttributesClass anymore, the **vac_finalizer_controller** should retry the deletion as a separate go routine. +Only the **vac_finalizer_controller** will remove finalizers on VolumeAttributesClass. If the **vac_finalizer_controller** fails at the step of removing finalizer even there is no PVC/PV using the VolumeAttributesClass anymore, the **vac_finalizer_controller** should retry it in next reconcile loop. The vac_finalizer_controller will use only informers and therefore it may remove the finalizer too early. One scenario is: @@ -888,7 +880,7 @@ For each of them, fill in the following information by copying the below templat levels that could help debug the issue? Not required until feature graduated to beta. - Testing: Are there any tests for failure mode? If not, describe why. ---> + --> ###### What steps should be taken if SLOs are not being met to determine the problem?