Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(artifacts): only retry on transient S3 errors #5579

Merged
merged 10 commits into from
Apr 13, 2021

Conversation

dinever
Copy link
Member

@dinever dinever commented Apr 2, 2021

Closes #5220

Hi,

This PR fixes the issue of executor retrying non-transient S3 errors.

I took the list of retryable errors from the minio-go repo.

As of today, this package doesn't have any unit tests. As the logic is getting more complex it's kind of risky to have no tests. I'll see if I can add some. (Might need some minor refactoring)

Thanks!

workflow/artifacts/s3/s3.go Outdated Show resolved Hide resolved
workflow/artifacts/s3/s3.go Outdated Show resolved Hide resolved
workflow/artifacts/s3/s3.go Outdated Show resolved Hide resolved
workflow/artifacts/s3/s3.go Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Apr 3, 2021

Codecov Report

Merging #5579 (c40a2d0) into master (54f4c26) will increase coverage by 0.03%.
The diff coverage is 76.19%.

❗ Current head c40a2d0 differs from pull request most recent head 513a4b5. Consider uploading reports for the commit 513a4b5 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5579      +/-   ##
==========================================
+ Coverage   47.08%   47.12%   +0.03%     
==========================================
  Files         240      242       +2     
  Lines       15050    15137      +87     
==========================================
+ Hits         7087     7133      +46     
- Misses       7059     7098      +39     
- Partials      904      906       +2     
Impacted Files Coverage Δ
cmd/argoexec/commands/root.go 1.72% <0.00%> (+0.02%) ⬆️
workflow/artifacts/s3/errors.go 60.00% <60.00%> (ø)
workflow/artifacts/s3/s3.go 41.66% <70.73%> (ø)
workflow/controller/operator.go 71.11% <100.00%> (+0.17%) ⬆️
workflow/metrics/server.go 12.50% <0.00%> (-4.17%) ⬇️
cmd/argo/commands/get.go 56.95% <0.00%> (+0.64%) ⬆️
cmd/argoexec/commands/emissary.go 50.00% <0.00%> (+1.56%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 54f4c26...513a4b5. Read the comment docs.

@dinever dinever force-pushed the fix/5220/s3-transient-errors branch from 51c9175 to 0c5cd16 Compare April 3, 2021 02:40
@dinever
Copy link
Member Author

dinever commented Apr 8, 2021

Will rebase after #5601

dinever added 4 commits April 11, 2021 22:10
Signed-off-by: Peixuan Ding <dingpeixuan911@gmail.com>
Signed-off-by: Peixuan Ding <dingpeixuan911@gmail.com>
Signed-off-by: Peixuan Ding <dingpeixuan911@gmail.com>
Signed-off-by: Peixuan Ding <dingpeixuan911@gmail.com>
@dinever dinever force-pushed the fix/5220/s3-transient-errors branch from 0c5cd16 to 721e0f8 Compare April 12, 2021 03:14
Signed-off-by: Peixuan Ding <dingpeixuan911@gmail.com>
@dinever dinever force-pushed the fix/5220/s3-transient-errors branch from b638d66 to 35cc086 Compare April 12, 2021 03:56
Signed-off-by: Peixuan Ding <dingpeixuan911@gmail.com>
workflow/artifacts/s3/s3.go Outdated Show resolved Hide resolved
dinever added 2 commits April 12, 2021 20:55
Signed-off-by: Peixuan Ding <dingpeixuan911@gmail.com>
Signed-off-by: Peixuan Ding <dingpeixuan911@gmail.com>
@dinever dinever force-pushed the fix/5220/s3-transient-errors branch from f5c55fc to fdc209a Compare April 13, 2021 00:56
Copy link
Contributor

@alexec alexec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I request one final small change. Once you've done that - dismiss this review so I get notified.

workflow/artifacts/s3/s3.go Outdated Show resolved Hide resolved
Signed-off-by: Peixuan Ding <dingpeixuan911@gmail.com>
@dinever dinever force-pushed the fix/5220/s3-transient-errors branch from 5b9917f to 513a4b5 Compare April 13, 2021 15:57
@alexec alexec merged commit b7d6905 into argoproj:master Apr 13, 2021
@dinever dinever deleted the fix/5220/s3-transient-errors branch April 15, 2021 19:17
@simster7 simster7 mentioned this pull request Apr 19, 2021
50 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

All S3 put errors get retried, not only transient ones
3 participants