This is same as closed issue #404 -- pvc/pv are removed, but not from heketi or the kubernetes dynamic service

[amgads]

Versions:
k8s: v1.15.4 -- heketi: v9.0.0, glusterfs: 6.5+

We are facing the same problem as in #404 when undeploying an application with tens of pvcs/pvs. It looks like pvc and pv are gone.
kubectl get pvc or kubectl get pv ---> no resources found

But both volumes in heketi as well as - the corresponding glusterfs-dynamic services are not still showing up -- not removed:
heketi-cli volume list or heketi-cli topology info --> show the volumes there. Also gluster volume list
and for dynamic services:

kubectl get svc --all-namespaces
NAMESPACE NAME TYPE CLUSTE
R-IP EXTERNAL-IP PORT(S) AGE
default glusterfs-dynamic-067b2708-8ffc-4f39-a5ee-67b05675fb73 ClusterIP 10.254
.1.187 1/TCP 3d18h
default glusterfs-dynamic-08ae158a-0064-42ed-b24b-8713186379ef ClusterIP 10.254
.160.55 1/TCP 3d18h
default glusterfs-dynamic-125a572f-d9a1-4ce1-b948-f1c1676cec16 ClusterIP 10.254
.180.65 1/TCP 3d18h
.....

[beat]

The SVC and the EP are still there too
kubectl get svc,ep
show entries still there too.

Looks like related to heketi bug heketi/heketi#1329 effecting that the user needs to be the admin user in the StorageClasss definition.