Improve extensibility

[egor-georgiev]

No description provided.

[egor-georgiev]

Hello, thanks for your work on Huey. I have added several ways to facilitate extensibility of Huey components, here are the use cases I added them for:

1. Moving consumer event loop logic to a separate method allows to insert custom operations into every iteration of event loop without checking that updates do not break custom logic (by calling super()). For instance, in our codebase, we make sure that every iteration of consumer event loop touches a file as a means of health-checking Consumer process in container runtime.
2. Adding consumer_class, task_wrapper_class allows to use custom classes for Huey Consumer and TaskWrapper without overriding whole task, create_consumer methods. To illustrate the use case, in our codebase we add several new convenience methods to Consumer class in order to facilitate health-check logic described in 1. We also plan on overriding TaskWrapper's s() and schedule() methods in order to register task id <-> task name mapping in redis storage.

[egor-georgiev]

I brought task_wrapper_class as a @property in order not to toss around class declarations (property is lazy, but attribute is not, therefore, adding it as an attribute and not re-ordering declarations would cause a NameError).

[coleifer]

Some of these changes are now implemented in:

• 383331e
• 8cfe4cf

It seemed silly to make the consumer class an attribute and task wrapper a property. I'll retain the convention of requiring people to override the get_consumer() method if they want to use a custom Consumer implementation, and have also added a new get_task_wrapper_class() method that can be overridden to provide a custom TaskWrapper.

For the consumer, I've factored the main loop into its own method similar to how you have it, although I pulled the exception into the consumer module, as it's not something I expect people to generally attempt to catch.

I believe these changes should make it possible for you to accomplish what you've described.

[egor-georgiev]

@coleifer thank you for your quick response! Do you have in mind a timeline for the next release?

[coleifer]

Not particularly, no, feel free to run on master in the meantime - I generally keep it stable & tests passing.