Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Figure out retry behavior across the board #1001

Open
wolfv opened this issue Dec 23, 2024 · 0 comments
Open

Figure out retry behavior across the board #1001

wolfv opened this issue Dec 23, 2024 · 0 comments

Comments

@wolfv
Copy link
Contributor

wolfv commented Dec 23, 2024

We currently have the following situation with regards to network retries:

  • We use the ClientWithMiddleware everywhere, and there can be a retry-middleware installed.
  • We cannot inspect the middlewares that are installed. Once inside the ClientWithMiddleware it becomes opaque. We need the authentication middlewares for sure.
  • The package cache has it's own retry mechanism that retries under certain conditions. These conditions differ from reqwest-retry. For example, reqwest-retry also retries on a connect issue under certain conditions: https://github.com/TrueLayer/reqwest-middleware/blob/f310cb8604ada3c992ffab370fbdbd50ff108a21/reqwest-retry/src/retryable_strategy.rs#L140
  • We do not use a retry strategy by default when downloading sharded repodata and we have hit some issues in the pixi tests (they became a little flaky). Retrying would definitely be appropriate.

Now, in pixi and rattler-build we install the (default) retry middleware in the ClientWithMiddleware. However, now we are in a situation where we might retry more in the package cache, since we woudl do 3x3 retries.

We should decide if we want to have custom retry handling in rattler, or a custom retry middleware. However, that might also be tricky, because the package cache passes the stream to the extraction functions, and we migth have to do more "cleanup" when retrying.

We could also create a new wrapper type around the ClientWithMiddleware that would leave it open to the functions to use the client with builtin-retry mechanism or the client without.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant