Feature/multithreading

[davidheitman]

This allows multithreading for list_consumer_groups() and describe_consumer_groups(), so brokers are access in parallel rather than sequentially, greatly improving performance.

---

This change is 

[jeffwidman]

I understand what you're trying to do here, I literally ran into this problem just two days ago at work.

However, I wouldn't solve it at this layer... the underlying KafkaClient instance already runs an event loop and returns a future. So it might be cleaner to leverage that to make the requests non-blocking/parallel. That's how the Java client does it.

The biggest problem with this approach is that it's a bit more invasive because all the rest of these methods have to manage futures/callbacks, but that is probably the right way to go long term.

Because the alternative here of layering on threading on top of an event loop feels a bit convoluted... like we took non-blocking, made it blocking, then tried to make it non-blocking again with threads.

Would you be interesting in trying to convert KafkaAdminClient to use futures instead?

[davidheitman]

Certainly, great suggestion. Not something I've worked with before, but I'll do some reading and see what I can do.

[davidheitman]

Haven't tested this yet, but you're talking about something like this I think: davidheitman@ba16a65#diff-c893e6ff8a43607853864c15224c1802

[davidheitman]

And tested. Works like a charm, I'll open a PR for the new branch.

[davidheitman]

#1807