Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exposes the endpointSliceWatchTimeout value #53

Merged
merged 8 commits into from
Aug 10, 2022

Conversation

sonnysideup
Copy link
Contributor

@sonnysideup sonnysideup commented Aug 4, 2022

@steved I found this interesting article that describes some of the issues we see in environments where buildkit pods are frequently terminated and re-created.

My conclusion is that (a) my static 90s endpoint watch timeout is too low in certain instances where the buildkit pods churn a lot and (b) this value should be externally configurable so that it can serviced without code changes.

Changes

  • Exposes fetchAndExtractTimeout value to control archive processing time and bumps default fetch retry considerably
  • Exposes poolEndpointWatchTimeout to allow the configuration of this value
  • Upgrades to go 1.19
  • Upgrades vector to 0.23
  • Upgrades golangci-lint
  • Adds diagnostic logging whenever we fail to acquire a buildkit pod endpoint

this was a const before but now it's configurable via the helm chart
values -> config

also pass a pod to be processed by value to avoid the variable
re-declaration with a for loop
@sonnysideup sonnysideup requested a review from steved August 4, 2022 22:15
if rErr := p.releasePod(ctx, pod); rErr != nil {
log.Error(err, "Failed to release pod")
if rErr := p.releasePod(ctx, &pod); rErr != nil {
log.Error(rErr, "Failed to release pod")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was logging the wrong error.

the value should be an int, not a string
@@ -375,8 +376,7 @@ func (p *workerPool) updateWorkers(ctx context.Context) error {
if req := p.requests.Dequeue(); req != nil {
log.Info("Found pending pod request, processing")

pod := pod
if p.processPodRequest(ctx, req, &pod) {
if p.processPodRequest(ctx, req, pod) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just passing the pod value instead of a pointer to avoid implicit memory aliasing and variable re-declaration.

these will be printed when we fail to resolve an endpoint for buildkit
pod
and fixes linting errors
and corrects previous value for poolEndpointWatchTimeout
@sonnysideup sonnysideup marked this pull request as ready for review August 10, 2022 16:48
@sonnysideup sonnysideup merged commit fd48cdf into main Aug 10, 2022
@sonnysideup sonnysideup deleted the expose-endpoint-watch-timeout branch August 10, 2022 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants