Problems while connecting to ZK over TLS

[Vobo]

I'd like to use TSL while connecting to ZK. I've done all needed configuration changes to ZK (secureClientPort, keystore, truststore) and tested it with a golang client. That's how I'm creating a connection in python (I'm using the same certs/keys/ca as for the golang client):

    self.client = KazooClient(
            use_ssl=True,
            verify_certs=True,
            ca='zookeeper.crt',
            certfile='cert.pem',
            keyfile='key.pem',
    self.client.start()

In most cases the connection fails with messages:

Connection dropped: socket connection error: The operation did not complete (read) (_ssl.c:2305)
Connection dropped: socket connection error: The operation did not complete (read) (_ssl.c:2305)
Connection dropped: socket connection error: The operation did not complete (read) (_ssl.c:2305)
Connection dropped: socket connection error: The operation did not complete (read) (_ssl.c:2305)
Failed connecting to Zookeeper within the connection retry policy.
Traceback (most recent call last):
  File "/home/vovabogatyrev/PycharmProjects/experiments/zookeeper.py", line 79, in <module>
    print(read_node('/test_schema'))
  File "/home/vovabogatyrev/PycharmProjects/experiments/zookeeper.py", line 53, in read_node
    with KazooContext(hosts=ZK_HOSTS) as zk_client:
  File "/home/vovabogatyrev/PycharmProjects/experiments/zookeeper.py", line 35, in __enter__
    self.client.start(timeout=20)
  File "/home/vovabogatyrev/PycharmProjects/experiments/venv/lib/python3.6/site-packages/kazoo/client.py", line 582, in start
    raise self.handler.timeout_exception("Connection time-out")
kazoo.handlers.threading.KazooTimeoutError: Connection time-out

But sometimes it succeeds.
That's what I see in Zookeeper logs:

2019-11-22 12:42:51,406 [myid:1] - INFO  [epollEventLoopGroup-7-43:X509AuthenticationProvider@172] - Authenticated Id 'O=Internet Widgits Pty Ltd,ST=Some-State,C=AU' for Scheme 'x509'
2019-11-22 12:42:51,972 [myid:1] - INFO  [epollEventLoopGroup-4-95:NettyServerCnxn@272] - Processing mntr command from /2a02:6b8:c01:b7d:0:1589:7220:bd18%0:58480
2019-11-22 12:42:52,486 [myid:1] - INFO  [epollEventLoopGroup-7-44:NettyServerCnxnFactory@336] - SSL handler added for channel: [id: 0x7f4c7dd2, L:/2a02:6b8:c01:b7d:0:1589:7220:bd18%0:2182 - R:/2a02:6b8:c02:45c:0:1589:d1fb:dcd2%0:39768]
2019-11-22 12:42:52,666 [myid:1] - INFO  [epollEventLoopGroup-7-44:X509AuthenticationProvider@172] - Authenticated Id 'O=Internet Widgits Pty Ltd,ST=Some-State,C=AU' for Scheme 'x509'
2019-11-22 12:42:52,735 [myid:1] - INFO  [epollEventLoopGroup-7-45:NettyServerCnxnFactory@336] - SSL handler added for channel: [id: 0x4f36a06f, L:/2a02:6b8:c01:b7d:0:1589:7220:bd18%0:2182 - R:/2a02:6b8:c02:45c:0:1589:d1fb:dcd2%0:39796]
2019-11-22 12:42:52,931 [myid:1] - INFO  [epollEventLoopGroup-7-45:X509AuthenticationProvider@172] - Authenticated Id 'O=Internet Widgits Pty Ltd,ST=Some-State,C=AU' for Scheme 'x509'
2019-11-22 12:42:52,998 [myid:1] - INFO  [epollEventLoopGroup-7-46:NettyServerCnxnFactory@336] - SSL handler added for channel: [id: 0x2a8ec569, L:/2a02:6b8:c01:b7d:0:1589:7220:bd18%0:2182 - R:/2a02:6b8:c02:45c:0:1589:d1fb:dcd2%0:39834]

Kazoo: 2.6.1
Python: 3.6.8
Zookeeper: 3.5.5

[StephenSorriaux]

Hi @Vobo,

Can you please share a snippet that would allow us to reproduce the issue?

[jeffwidman]

nudge @Vobo