Garbage collect with provided retention options #638
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aryan9600
changed the title
pjbgf
reviewed
Mar 25, 2022
pjbgf
reviewed
Mar 25, 2022
pjbgf
reviewed
Mar 25, 2022
aryan9600
force-pushed
the
garbage-retention
branch
from
9d6da04 to
18c2f29
Compare
aryan9600
force-pushed
the
garbage-retention
branch
4 times, most recently
from
be78b48 to
d9cf6de
Compare
pjbgf
reviewed
Mar 30, 2022
pjbgf
reviewed
Mar 30, 2022
pjbgf
reviewed
Mar 30, 2022
hiddeco
reviewed
Apr 1, 2022
aryan9600
force-pushed
the
garbage-retention
branch
2 times, most recently
from
5e6e6c8 to
68ef63d
Compare
hiddeco
reviewed
Apr 1, 2022
pjbgf
reviewed
Apr 1, 2022
| if err := testStorage.MkdirAll(*obj.Status.Artifact); err != nil { | ||
| return err | ||
| } | ||
| if err := testStorage.AtomicWriteFile(obj.Status.Artifact, strings.NewReader(v), 0644); err != nil { |
There was a problem hiding this comment.
Suggested change
| if err := testStorage.AtomicWriteFile(obj.Status.Artifact, strings.NewReader(v), 0644); err != nil { | |
| if err := testStorage.AtomicWriteFile(obj.Status.Artifact, strings.NewReader(v), 0o644); err != nil { |
pjbgf
reviewed
Apr 1, 2022
| if err := testStorage.MkdirAll(*obj.Status.Artifact); err != nil { | ||
| return err | ||
| } | ||
| if err := testStorage.AtomicWriteFile(obj.Status.Artifact, strings.NewReader("file"), 0644); err != nil { |
There was a problem hiding this comment.
Suggested change
| if err := testStorage.AtomicWriteFile(obj.Status.Artifact, strings.NewReader("file"), 0644); err != nil { | |
| if err := testStorage.AtomicWriteFile(obj.Status.Artifact, strings.NewReader("file"), 0o644); err != nil { |
pjbgf
reviewed
Apr 4, 2022
darkowlzz
reviewed
Apr 4, 2022
aryan9600
force-pushed
the
garbage-retention
branch
4 times, most recently
from
d03d02b to
d98e93c
Compare
stefanprodan
reviewed
Apr 5, 2022
pjbgf
reviewed
Apr 6, 2022
aryan9600
force-pushed
the
garbage-retention
branch
from
9b97975 to
391d559
Compare
aryan9600
force-pushed
the
garbage-retention
branch
2 times, most recently
from
c46dd51 to
dc6b9e5
Compare
hiddeco
approved these changes
Apr 7, 2022
There was a problem hiding this comment.
This LGTM if the commits are smashed. Thank you @aryan9600 🥇
aryan9600
force-pushed
the
garbage-retention
branch
from
dc6b9e5 to
641bc19
Compare
Introduce two new flags to configure the ttl of an artifact and the max no. of files to retain for an artifact. Modify the gc process to consider the options and use timeouts to prevent the controller from hanging. This helps in situations when the SC has already garbage collected the current artifact but the advertised artifact url is still the same, which leads to the server returning a 404. Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
aryan9600
force-pushed
the
garbage-retention
branch
from
641bc19 to
f8c27a8
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds two ways (a ttl for each artifact and max number of artifact per objects) to customize garbage collection. Adds a timeout context to cancel garbage collection after a duration.
Ref: #602
Signed-off-by: Sanskar Jaiswal jaiswalsanskar078@gmail.com