Add kafka 0.10.2.1 into integration testing version

[jianbin-wei]

No description provided.

[dpkp]

Adding another kafka version means we add 6x more test runs to travis (one for each python version). I am not super excited about making tests take longer. Thoughts on possibly removing an existing version from the test suite?

[jianbin-wei]

Do you mean the tests increase from 30 to 36 instead of 6 times?

I would deprecate python 2.6 as Python 2.6.9 final was released on October 29, 2013 and 2.6.x has official retired.

Regarding kafka, I would prefer 0.8.0 and 0.8.1.1 to be deprecated. 0.8.0 has about 0.5% emails in kafka-user mailing list. 0.8.1.1 has about 1% emails.

That would reduce the tests need to run from 30 to 20.

[jeffwidman]

I agree with Jianbin.

I'd even consider going a step further and stop testing against anything below Kafka 0.9x. That's when the consumer group APIs showed up that are used by KafkaConsumer. The SimpleConsumer stack is already deprecated, so don't need to continue to guarantee support. Plus 0.8x was notoriously buggy, so while I know there are plenty of existing installations of it, by this point they should be seriously thinking about upgrading.

Also, for Python, I'd probably drop Python 3.3 and 3.4. Most folks on the Python 3 bandwagon are self-selecting toward those hungry to be on the bleeding edge. Additionally Ubuntu 16.04 LTS shipped with 3.5 and RHEL is likely shipping such an ancient version it's not worth worrying about. There's a decent chance anyone using Python 3 on REHL is using https://ius.io/ to get either 3.5 or 3.6 these days.

[jianbin-wei]

I think if some deployment on 0.8x it is very likely to be on 0.8.2. It should be ok to keep testing for 0.8.2.2 for that unless some feature breaks that backward compatibility and would need some discussion.

Regarding python 3.x, http://blog.jelastic.com/2016/04/14/software-stacks-market-share-first-quarter-of-2016/ shows that globally 3.4 is 7%, 3.5 6%, 2.7 87%, 3.3 is 0%. But in US 3.3 is 5%. Therefore, I would not drop the test cases for them. Some feature backward compatibility is another story when that happens.

[dpkp]

Debian 8.0 / stable is still shipping with python3.4 and also 3.5 adds some fairly significant new features that I want to make sure we are not relying on unintentionally. But python2.6 I am totally willing to drop. For kafka versions I think 0.8.0 and 0.8.1.1 can probably also go. We can always run tests for these manually before releases.

[jianbin-wei]

Agreed with @dpkp

[jeffwidman]

Looks like there's some py26 specific code in tox.ini (search for py26). And setup.py has an explicit 2.6 reference plus a sneaky one:

if sys.version_info < (2, 7):
    test_require.append('unittest2')

Can you include these with the previous commit? git commit --amend followed by force pushing to this branch should do it.

[dpkp]

I'd prefer that we not delete the resources files for old kafka brokers or tox configuration for py26. These would still be useful to test locally / debug with, even if we aren't doing continuous testing or making any compatibility guarantees.

[jianbin-wei]

For each minor version of kafka, I left one resource directory in place, i.e.,

0.10.0.0 0.10.0.1 0.10.1.1 0.10.2.1 0.8.0 0.8.1.1 0.8.2.2 0.9.0.0 0.9.0.1 dist trunk

Is that good enough? I am ok with either way you prefer.

Regarding python 2.6, do you really need it? I searched the code and there are some changes would break the support, for example the use of gzip context manager in kafka/codec.py.

[tvoinarovskyi]

Am totally for this patch, but agree with @dpkp, there's just no need to remove any of the servers folders. Let them stay there, kafka's old versions are still used.

EDIT: as for the python2.6, maybe we should release new version as 1.4 and say that we dropped py2.6. So that 1.3 is last release with python2.6 support.

[jianbin-wei]

@dpkp, I have reverted the deletion of server resource files. What is your opinion about py26? I kind of agree what @Drizzt1991 said to bump the version for dropping python 2.6 support.

[jeffwidman]

Bump @dpkp your thoughts?

We're starting to evaluate upgrading some clusters to 0.10.2.1, but we're a bit hesitant to do so until this is merged...

[dpkp]

I doubt anyone on python2.6 is running master, so it's probably ok to drop now and make the next release 1.4 .

Also should be aware that 0.11.0.0 is up for release vote, and we will need to start testing against it as well.

[dpkp]

I decided not to keep the tox.ini / setup.py changes that disabled running tests on py26 . I think we should still be able to run the tests, even if it is no longer supported.