Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[receiver/k8sobjects] Improve handling of response code 410 #24903

Closed
TylerHelmuth opened this issue Aug 4, 2023 · 3 comments · Fixed by #26098
Closed

[receiver/k8sobjects] Improve handling of response code 410 #24903

TylerHelmuth opened this issue Aug 4, 2023 · 3 comments · Fixed by #26098
Assignees
@TylerHelmuth
Labels
enhancement New feature or request priority:p2 Medium receiver/k8sobjects

Comments

@TylerHelmuth
Copy link
Member

TylerHelmuth commented Aug 4, 2023 •
edited
Loading

Component(s)

No response

Is your feature request related to a problem? Please describe.

The watch capability of the K8s API will return an http status code of 410 and a status object if the resourceVersion supplied to the API is too old. At the moment the receiver handles these situations by logging that they occurred and then stopping the watch for the object in question.

This situation is most likely to occur if a user passes in a resource_version itself, but could occur during normal option as well (although that would be strange).

Describe the solution you'd like

Instead of giving up, the receiver could do a new List to get a new resourceVersion. See https://kubernetes.io/docs/reference/using-api/api-concepts/#410-gone-responses for details.
@TylerHelmuth TylerHelmuth added enhancement New feature or request needs triage New item requiring triage priority:p2 Medium receiver/k8sobjects and removed needs triage New item requiring triage labels Aug 4, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Aug 4, 2023

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 4, 2023

Pinging code owners for receiver/k8sobjects: @dmitryax @hvaghani221. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@TylerHelmuth TylerHelmuth self-assigned this Aug 23, 2023
@TylerHelmuth
Copy link
Member Author

I have started working on this.

@TylerHelmuth TylerHelmuth mentioned this issue Aug 24, 2023
Merged
TylerHelmuth added a commit that referenced this issue Sep 11, 2023
@TylerHelmuth @evan-bradley
[receiver/k8sobjects] Add logic to properly handle 410 responses (#26098
a4999eb 
)

**Description:** 
Updates the receiver to properly handle 410 response code. The
expectations for what clients should do when a 410 is received can be
found here:
https://kubernetes.io/docs/reference/using-api/api-concepts/#410-gone-responses.

I originally implemented this feature directly in `startWatch`,
rebuilding the watcher and res channel within the for loop, but I grew
concerned about making sure everything stopped correctly. I took a look
at the retry watcher's implementation and reused its concepts for this
implementation. If it is overcomplicated we can go back to my original
idea.

**Link to tracking Issue:** <Issue number if applicable>

Closes
#24903

**Testing:** <Describe what testing was performed and which tests were
added.>

Tested locally. Unit tests proved to be extremely challenging since I
couldn't figure out how to get the mock to produce a 410. We really need
e2e tests
(#18395).

---------

Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@TylerHelmuth TylerHelmuth

Labels
enhancement New feature or request priority:p2 Medium receiver/k8sobjects
Projects
None yet
Milestone
No milestone
1 participant
@TylerHelmuth