Remove old SimpleClient / SimpleProducer / SimpleConsumer

[jeffwidman]

A bunch of old code cleanup as part of bumping to version 2.

• Remove old SimpleClient / SimpleProducer / SimpleConsumer and all associated code.
• Clearly document the removal of these so that anyone who comes along knows they need to use the 1.x branch for these old clients. --> Moved to Document the removal of Simple* and that the KafkaClient is the new one, not the old one #1926
• Update tests from the old Simple* stuff to use the new KafkaClient / KafkaProducer / KafkaConsumer

Notably this will drop our test runs from ~6-7 minutes to ~2-3 minutes per run... really convenient.

Fix #1193

---

This change is 

[jeffwidman]

I spent a while updating this last night, there are a few tests of the new KafkaConsumer / KafkaProducer that depend on fixtures using the old SimpleClient that still need updating, once that's complete this will be close to done.

It is refreshing poking through the codebase on this branch as it's immediately obvious where things are w/o the confusion of wondering if I'm looking at code used by the old clients or the new clients.

[dpkp]

This is definitely a big one. Re sequencing, I would love to get a final 1.4.X patch release out with any remaining bugfixes for stability. Then we can release this as either 1.5.0 or 2.0.

[jeffwidman]

IMHO this should be a 2.0 release. Massive breakage everywhere. It would pair well with the new Admin client for the 2.0 release.

And 👍 on a bugfix release for the 1.4.X series... I am actually just about to start profiling some code that hit a massive performance regression after switching from SimpleProducer to KafkaProducer, so may have one more more items to land in that 1.4.x series... I think it'd be fine to sneak in the initial admin client as well into this 1.4.x release but the big emphasis of it would be the 2.0 release.

[jeffwidman]

I think tests should now fully pass for this, assuming Travis passes for #1886.

There is one commit of TODO's highlighting possible additional code deletions... anything there should probably be included in this PR as well...

Also, still need to update the docs to clearly mention the deprecation/deletion of the old clients.

[jeffwidman]

Tests are green.

[jeffwidman]

Just rebased to pickup the new updates from the 1.4.7 release.

Hoping we can get this merged in the next few weeks so that it doesn't keep hanging out there needing to be rebased... and so we don't have folks accidentally creating unittest-based tests.

[jeffwidman]

The tests are now failing because more unittest-style tests crept in here: #1833

So those need to be migrated for this to work. Perhaps @ulrikjohansson can handle it when he fixes up his tests.

[jeffwidman]

Many thanks to @ulrikjohansson for migrating the KafkaAdminClient tests to pytest.

I just rebased this, hopefully Travis works fine.

[jeffwidman]

It bothers me that we've had two folks nicely submit PR's and then I had to ask them to update tests to drop the Unittest support so that they didn't block this... so I'm planning to merge this.

I think at this point we should do a 2.x version, as it's a big change... I realize we don't religiously follow Semvar, but I don't think 1.5 is a big enough bump.

I filed #1926 to track the documentation of the change for old users.

[mmenbawy]

@jeffwidman @dpkp
Why were the following 3 files removed by this PR?

• kafka/partitioner/base.py
• kafka/partitioner/hashed.py
• kafka/partitioner/roundrobin.py

[ekeric13]

@jeffwidman @dpkp yeah i am trying to use a round robin partitioner and it is gone now? how come it was removed? was it buggy?