Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Controller looks like trying to delete PV not belongs to him #1511

Closed
4 tasks done
shaikatz opened this issue Jun 8, 2022 · 11 comments
Closed
4 tasks done

Controller looks like trying to delete PV not belongs to him #1511

shaikatz opened this issue Jun 8, 2022 · 11 comments

Comments

@shaikatz
Copy link

shaikatz commented Jun 8, 2022

Controller Version

0.24.0

Helm Chart Version

0.19.0

Deployment Method

Helm

Checks

  • This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
  • I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
  • My actions-runner-controller version (v0.x.y) does support the feature
  • I've already upgraded ARC to the latest and it didn't fix the issue

Resource Definitions

I don't have any resources set yet, just finished installing the controller.

To Reproduce

Have PV attached to PVC for any pod in the cluster.
Install helm chart 0.19.0 with default settings.
Check the controller logs, you'll see:
-06-08T10:27:37Z    DEBUG   actions-runner-controller.runnerpersistentvolume        Retrying sync until pvc gets removed    {"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}

Describe the bug

When installing fresh controller, without any runners attached, it finds used PV in the cluster, and tries to delete it.

Describe the expected behavior

The controller should not touch PVs not belongs to him.

Controller Logs

2022-06-08T10:20:37Z	INFO	controller-runtime.metrics	Metrics server is starting to listen	{"addr": "127.0.0.1:8080"}
2022-06-08T10:20:37Z	INFO	actions-runner-controller	Initializing actions-runner-controller	{"github-api-cache-duration": "50s", "default-scale-down-delay": "10m0s", "sync-period": "1m0s", "default-runner-image": "summerwind/actions-runner:latest", "default-docker-image": "docker:dind", "common-runnner-labels": null, "leader-election-enabled": true, "leader-election-id": "actions-runner-controller", "watch-namespace": ""}
2022-06-08T10:20:37Z	INFO	controller-runtime.builder	Registering a mutating webhook	{"GVK": "actions.summerwind.dev/v1alpha1, Kind=Runner", "path": "/mutate-actions-summerwind-dev-v1alpha1-runner"}
2022-06-08T10:20:37Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/mutate-actions-summerwind-dev-v1alpha1-runner"}
2022-06-08T10:20:37Z	INFO	controller-runtime.builder	Registering a validating webhook	{"GVK": "actions.summerwind.dev/v1alpha1, Kind=Runner", "path": "/validate-actions-summerwind-dev-v1alpha1-runner"}
2022-06-08T10:20:37Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/validate-actions-summerwind-dev-v1alpha1-runner"}
2022-06-08T10:20:37Z	INFO	controller-runtime.builder	Registering a mutating webhook	{"GVK": "actions.summerwind.dev/v1alpha1, Kind=RunnerDeployment", "path": "/mutate-actions-summerwind-dev-v1alpha1-runnerdeployment"}
2022-06-08T10:20:37Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/mutate-actions-summerwind-dev-v1alpha1-runnerdeployment"}
2022-06-08T10:20:37Z	INFO	controller-runtime.builder	Registering a validating webhook	{"GVK": "actions.summerwind.dev/v1alpha1, Kind=RunnerDeployment", "path": "/validate-actions-summerwind-dev-v1alpha1-runnerdeployment"}
2022-06-08T10:20:37Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/validate-actions-summerwind-dev-v1alpha1-runnerdeployment"}
2022-06-08T10:20:37Z	INFO	controller-runtime.builder	Registering a mutating webhook	{"GVK": "actions.summerwind.dev/v1alpha1, Kind=RunnerReplicaSet", "path": "/mutate-actions-summerwind-dev-v1alpha1-runnerreplicaset"}
2022-06-08T10:20:37Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/mutate-actions-summerwind-dev-v1alpha1-runnerreplicaset"}
2022-06-08T10:20:37Z	INFO	controller-runtime.builder	Registering a validating webhook	{"GVK": "actions.summerwind.dev/v1alpha1, Kind=RunnerReplicaSet", "path": "/validate-actions-summerwind-dev-v1alpha1-runnerreplicaset"}
2022-06-08T10:20:37Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/validate-actions-summerwind-dev-v1alpha1-runnerreplicaset"}
2022-06-08T10:20:37Z	INFO	controller-runtime.webhook	Registering webhook	{"path": "/mutate-runner-set-pod"}
2022-06-08T10:20:37Z	INFO	actions-runner-controller	starting manager
2022-06-08T10:20:37Z	INFO	controller-runtime.webhook.webhooks	Starting webhook server
2022-06-08T10:20:37Z	INFO	Starting server	{"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"}
2022-06-08T10:20:37Z	INFO	controller-runtime.certwatcher	Updated current TLS certificate
2022-06-08T10:20:37Z	INFO	controller-runtime.webhook	Serving webhook server	{"host": "", "port": 9443}
2022-06-08T10:20:37Z	INFO	controller-runtime.certwatcher	Starting certificate watcher
I0608 10:20:37.736661       1 leaderelection.go:248] attempting to acquire leader lease actions-runner-system/actions-runner-controller...
I0608 10:20:37.753528       1 leaderelection.go:258] successfully acquired lease actions-runner-system/actions-runner-controller
2022-06-08T10:20:37Z	DEBUG	events	Normal	{"object": {"kind":"ConfigMap","namespace":"actions-runner-system","name":"actions-runner-controller","uid":"4b5eaaf9-76e5-465e-9b15-892f3d353b0c","apiVersion":"v1","resourceVersion":"1864900"}, "reason": "LeaderElection", "message": "gh-runners-actions-runner-controller-84c48d6c9-dkmv6_ce120535-a6e2-4c0e-a1b2-0b4fca995854 became leader"}
2022-06-08T10:20:37Z	DEBUG	events	Normal	{"object": {"kind":"Lease","namespace":"actions-runner-system","name":"actions-runner-controller","uid":"bddabdaf-cb13-4905-bcdd-08fc652bd81b","apiVersion":"coordination.k8s.io/v1","resourceVersion":"1864901"}, "reason": "LeaderElection", "message": "gh-runners-actions-runner-controller-84c48d6c9-dkmv6_ce120535-a6e2-4c0e-a1b2-0b4fca995854 became leader"}
2022-06-08T10:20:37Z	INFO	controller.runner-controller	Starting EventSource	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "Runner", "source": "kind source: *v1alpha1.Runner"}
2022-06-08T10:20:37Z	INFO	controller.runner-controller	Starting EventSource	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "Runner", "source": "kind source: *v1.Pod"}
2022-06-08T10:20:37Z	INFO	controller.runner-controller	Starting Controller	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "Runner"}
2022-06-08T10:20:37Z	INFO	controller.runnerreplicaset-controller	Starting EventSource	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerReplicaSet", "source": "kind source: *v1alpha1.RunnerReplicaSet"}
2022-06-08T10:20:37Z	INFO	controller.runnerreplicaset-controller	Starting EventSource	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerReplicaSet", "source": "kind source: *v1alpha1.Runner"}
2022-06-08T10:20:37Z	INFO	controller.runnerreplicaset-controller	Starting Controller	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerReplicaSet"}
2022-06-08T10:20:37Z	INFO	controller.runnerset-controller	Starting EventSource	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerSet", "source": "kind source: *v1alpha1.RunnerSet"}
2022-06-08T10:20:37Z	INFO	controller.runnerset-controller	Starting EventSource	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerSet", "source": "kind source: *v1.StatefulSet"}
2022-06-08T10:20:37Z	INFO	controller.runnerdeployment-controller	Starting EventSource	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerDeployment", "source": "kind source: *v1alpha1.RunnerDeployment"}
2022-06-08T10:20:37Z	INFO	controller.runnerset-controller	Starting Controller	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerSet"}
2022-06-08T10:20:37Z	INFO	controller.runnerdeployment-controller	Starting EventSource	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerDeployment", "source": "kind source: *v1alpha1.RunnerReplicaSet"}
2022-06-08T10:20:37Z	INFO	controller.runnerdeployment-controller	Starting Controller	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerDeployment"}
2022-06-08T10:20:37Z	INFO	controller.runnerpod-controller	Starting EventSource	{"reconciler group": "", "reconciler kind": "Pod", "source": "kind source: *v1.Pod"}
2022-06-08T10:20:37Z	INFO	controller.runnerpod-controller	Starting Controller	{"reconciler group": "", "reconciler kind": "Pod"}
2022-06-08T10:20:37Z	INFO	controller.horizontalrunnerautoscaler-controller	Starting EventSource	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "HorizontalRunnerAutoscaler", "source": "kind source: *v1alpha1.HorizontalRunnerAutoscaler"}
2022-06-08T10:20:37Z	INFO	controller.horizontalrunnerautoscaler-controller	Starting Controller	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "HorizontalRunnerAutoscaler"}
2022-06-08T10:20:37Z	INFO	controller.runnerpersistentvolumeclaim-controller	Starting EventSource	{"reconciler group": "", "reconciler kind": "PersistentVolumeClaim", "source": "kind source: *v1.PersistentVolumeClaim"}
2022-06-08T10:20:37Z	INFO	controller.runnerpersistentvolumeclaim-controller	Starting Controller	{"reconciler group": "", "reconciler kind": "PersistentVolumeClaim"}
2022-06-08T10:20:37Z	INFO	controller.runnerpersistentvolume-controller	Starting EventSource	{"reconciler group": "", "reconciler kind": "PersistentVolume", "source": "kind source: *v1.PersistentVolume"}
2022-06-08T10:20:37Z	INFO	controller.runnerpersistentvolume-controller	Starting Controller	{"reconciler group": "", "reconciler kind": "PersistentVolume"}
2022-06-08T10:20:37Z	INFO	controller.runnerdeployment-controller	Starting workers	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerDeployment", "worker count": 1}
2022-06-08T10:20:37Z	INFO	controller.runnerpod-controller	Starting workers	{"reconciler group": "", "reconciler kind": "Pod", "worker count": 1}
2022-06-08T10:20:37Z	INFO	controller.runnerpersistentvolume-controller	Starting workers	{"reconciler group": "", "reconciler kind": "PersistentVolume", "worker count": 1}
2022-06-08T10:20:37Z	INFO	controller.horizontalrunnerautoscaler-controller	Starting workers	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "HorizontalRunnerAutoscaler", "worker count": 1}
2022-06-08T10:20:37Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:20:37Z	INFO	controller.runnerreplicaset-controller	Starting workers	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerReplicaSet", "worker count": 1}
2022-06-08T10:20:37Z	INFO	controller.runnerpersistentvolumeclaim-controller	Starting workers	{"reconciler group": "", "reconciler kind": "PersistentVolumeClaim", "worker count": 1}
2022-06-08T10:20:37Z	INFO	controller.runnerset-controller	Starting workers	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "RunnerSet", "worker count": 1}
2022-06-08T10:20:37Z	INFO	controller.runner-controller	Starting workers	{"reconciler group": "actions.summerwind.dev", "reconciler kind": "Runner", "worker count": 1}
2022-06-08T10:20:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:20:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:21:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:21:17Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:21:27Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:21:37Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:21:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:21:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:22:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:22:17Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:22:27Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:22:37Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:22:44Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:22:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:22:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:23:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:23:17Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:23:27Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:23:37Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:23:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:23:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:23:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:24:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:24:17Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:24:27Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:24:37Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:24:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:24:50Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:24:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:25:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:25:17Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:25:27Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:25:37Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:25:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:25:54Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:25:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:26:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:26:17Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:26:27Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:26:37Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:26:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:26:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:26:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:27:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:27:17Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:27:27Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:27:37Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:27:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:27:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:28:00Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:28:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:28:17Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:28:27Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:28:37Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:28:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:28:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:29:03Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:29:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:29:17Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:29:27Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:29:37Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:29:47Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:29:57Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:30:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}
2022-06-08T10:30:07Z	DEBUG	actions-runner-controller.runnerpersistentvolume	Retrying sync until pvc gets removed	{"pv": "/pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0", "requeueAfter": "10s"}

Runner Pod Logs

No runner yet.

Additional Context

No response

@shaikatz shaikatz added the bug Something isn't working label Jun 8, 2022
@rjeczalik
Copy link

This reproduces for me even if I set scope.singleNamespace=true, the pv-controller does not ignore cluster-scoped pv's. I think @shaikatz can confirm that in his case the /pvc-66f7edd0-5a6c-4b4c-81e4-26d9fdf886c0 is also cluster-scoped.

@ionutzaharia
Copy link

I also have the same issue, it's trying to delete a PVC that is from a non-related namespace.

@mumoshu
Copy link
Collaborator

mumoshu commented Jun 9, 2022

Hey everyone. In short, it's not trying to delete any resource. It's just a debug message to let you know that ARC did recognize the PV and still waiting to decide whether it needs to reconcile the PV or not (and it finally reconciles the PV only when the PV was created by ARC). See the relevant code linked below- you'll see it will never be removed, and will never modify unrelated PV.

https://github.com/actions-runner-controller/actions-runner-controller/blob/a62882d243fd8a3cd7901f72b99beeb92cc4e629/controllers/sync_volumes.go#L151

@mumoshu mumoshu removed the bug Something isn't working label Jun 9, 2022
@mumoshu mumoshu changed the title Controller trying to delete PV not belongs to him Controller looks like trying to delete PV not belongs to him Jun 9, 2022
@mumoshu
Copy link
Collaborator

mumoshu commented Jun 9, 2022

I believe this isn't actually a bug like reported(Feel free to retag it as "bug" if you still think so!). However I'm still open to suggestions like... should we rephrase the log message in a way so that it's less confusing, or hide it behind a more verbose log level so that it doesn't show up by default(btw, the default log level is DEBUG in ARC's official chart so that might be the reason why many folks are seeing it).

@rjeczalik
Copy link

It's just a debug message to let you know that ARC did recognize the PV and still waiting to decide whether it needs to reconcile the PV or not

@mumoshu Out of curiosity, why the pv reconciler even recognizes PV that do not belong to its namespace? (a case where the controller is configured with scope.singleNamespace=true)

@mumoshu
Copy link
Collaborator

mumoshu commented Jun 10, 2022

@rjeczalik AFAIK, PVs are not namespaced. Perhaps I'm missing something and you might be successfully using PVs with namespaces?

@rjeczalik
Copy link

@mumoshu Right, you're correct. Thanks for the explanation 🤗

@nuteee
Copy link

nuteee commented Jun 10, 2022

@mumoshu rephrasing the log message would be great as it currently suggests that it either gets removed by ARC, or I should manually remove those PVCs, while neither is true.

@mumoshu
Copy link
Collaborator

mumoshu commented Jun 13, 2022

@nuteee Thanks for your feedback! Would it look better with #1526?

@nuteee
Copy link

nuteee commented Jun 13, 2022

@mumoshu much better, thank you!

mumoshu added a commit that referenced this issue Jun 13, 2022
@mumoshu
Copy link
Collaborator

mumoshu commented Jun 15, 2022

Closing as resolved. Please feel free to open another issue if necessary! Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants