-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
builder: add retries to composer API calls #103
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a bunch of unrelated whitespace and import reordering changes.
The status calls are sometimes failing on: upstream connect error or disconnect/reset before headers. reset reason: connection termination Since all requests are going through the company proxy, I think that the networking isn't working 100% reliably. This commit adds a retry mechanism provided by the urllib3 library. It will retry on all networking issues and also on some 5xx errors that makes sense to retry (like gateway failures). A test is added that runs the compose waiting code against a mock server that fails every second request. This is imho sufficient to mimick a flaky networking.
These IDEs are smart... Fixed, thanks for noticing it! :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
How long (in seconds) will we retry for? In fedora Koji we may have outages of up to 30sec. |
|
Thanks, won't really make a difference for he firewall issue then. |
Could make it a config option. Or just increase it? |
The status calls are sometimes failing on:
upstream connect error or disconnect/reset before headers. reset reason:
connection termination
Since all requests are going through the company proxy, I think that the
networking isn't working 100% reliably. This commit adds a retry mechanism
provided by the urllib3 library. It will retry on all networking issues and
also on some 5xx errors that makes sense to retry (like gateway failures).
A test is added that runs the compose waiting code against a mock server
that fails every second request. This is imho sufficient to mimick a flaky
networking.