refactor: Replace machinery with tasqueue #32
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This MR aims to replace the underlying job library (machinery) with tasqueue. Apart from internal changes that do not affect this libraries' clients, the following changes/suggestions are made:
Manage the runtime cleanly via contexts. Currently I've passed a simple context to the
Start()
method. We can discuss how to handle the cancellation (this is omitted as of now).Change queue/concurrency configs. Tasqueue expects the concurrency parameter to be passed when registering the task (with some certain queue), unlike the existing way where concurrency is being passed when a group job is supposed to be queued. (We can discuss how to make this cleaner for clients)
Log management: Tasqueue uses
zerodha/logf
internally, maybe we can use this across the projectOrdering of pending jobs: The current testcases assume that the pending jobs returned will be in the same order as submission. This is implemented in a reverse manner in Tasqueue
Change in config: Instead of using DSNs to pass configs, have added a declarative config for both the results and broker options for Tasqueue.
PS: Any readme related changes can be made once the MR is finalized