Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: PVC in wf.status should be reset when retrying workflow #10685

Merged

Conversation

terrytangyuan
Copy link
Member

@terrytangyuan terrytangyuan commented Mar 15, 2023

This fixes a bug where a retried workflow would fail with the error persistentvolumeclaim xxx not found if the PVC is deleted previously (e.g. volumeClaimGC strategy is set to OnWorkflowCompletion). We can fix this by resetting the PVC in workflow status so that the operator could recreate the PVC.

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
@terrytangyuan terrytangyuan marked this pull request as ready for review March 15, 2023 18:52
Copy link
Member

@jessesuen jessesuen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this instead be reset when the pvc is deleted? In the controller code?

@terrytangyuan
Copy link
Member Author

terrytangyuan commented Mar 16, 2023

Should this instead be reset when the pvc is deleted? In the controller code?

@jessesuen I thought about that as well. This is the simplest fix without introducing additional API calls and complicating controller code. I cannot think of other use cases other than workflow retry. Another consideration is that when an engineer accidentally deletes the PVC, we want their workflows to fail and be aware of mistakes instead of silently recreating PVC. It's intuitive to recreate the PVC when retrying a workflow.

@terrytangyuan terrytangyuan merged commit ec85683 into argoproj:master Mar 17, 2023
@terrytangyuan terrytangyuan deleted the fix-pvc-creation-when-retry branch March 17, 2023 16:33
terrytangyuan added a commit that referenced this pull request Mar 29, 2023
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
JPZ13 pushed a commit to pipekit/argo-workflows that referenced this pull request Jul 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants