-
Notifications
You must be signed in to change notification settings - Fork 13
refactor: Remove redundant retries from batch_add_requests
#391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: Remove redundant retries from batch_add_requests
#391
Conversation
Align sync and async version to both swallow exception, but return unprocessed requets Add tests
batch_add_requests
batch_add_requests
059d5ce
to
e411ca4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR refactors the request queue client by removing redundant retry logic from the batch_add_requests methods (both sync and async) and aligning exception handling to rely on the HTTP client’s built-in retry capabilities. Additionally, new tests have been added to verify that exceptions are raised when appropriate and that partially processed batches are handled correctly.
- Removed retry logic and associated arguments from batch_add_requests.
- Aligned sync and async methods to surface HTTP client exceptions.
- Added unit tests covering exception and partial processing scenarios.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
File | Description |
---|---|
tests/unit/test_client_request_queue.py | New tests added for both async and sync behaviors after refactoring. |
src/apify_client/clients/resource_clients/request_queue.py | Removed the redundant retry logic; updated type annotations and docstrings accordingly. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Breaking change? Are we planning to release client v2? 🙂
We'll probably need to make this backward compatible for now, with a follow-up issue to clean things up for v2.0.
Good point. Let me just raise deprecation warning when someone uses no longer needed arguments and we can remove them in the far future in v2 |
batch_add_requests
batch_add_requests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks
Description
Remove retries from
batch_add_requests
(such retries already exist on the http client level).Add deprecation warnings when retires related arguments are used.
Align sync and async version to not ignore exceptions from the http client.
Add tests.
Issues
RequestQueueClient(Async).batch_add_requests
#390