Enable ZooKeeper client to establish connection in read-only mode

[massakam]

Motivation

If the system property readonlymode.enabled is set to true on a ZooKeeper server, read-only mode is enabled. Data can be read from the server in read-only mode even if that server is split from the quorum.
https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#Experimental+Options%2FFeatures

To connect to the server in read-only mode, the client must also allow read-only mode. The ZooKeeperClient class in the bookkeeper repository also has an option called allowReadOnlyMode.

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperClient.java

Lines 219 to 222 in 15171e1

	public Builder allowReadOnlyMode(boolean allowReadOnlyMode) {
	this.allowReadOnlyMode = allowReadOnlyMode;
	return this;
	}

However, even if this option is set to true, the connection to the server in read-only mode will actually fail. The cause is in the ZooKeeperWatcherBase class. When the ZooKeeperWatcherBase class receives the SyncConnected event, it releases clientConnectLatch and assumes that the connection is complete.

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperWatcherBase.java

Lines 128 to 144 in 15171e1

	switch (event.getState()) {
	case SyncConnected:
	LOG.info("ZooKeeper client is connected now.");
	clientConnectLatch.countDown();
	break;
	case Disconnected:
	LOG.info("ZooKeeper client is disconnected from zookeeper now,"
	+ " but it is OK unless we received EXPIRED event.");
	break;
	case Expired:
	clientConnectLatch = new CountDownLatch(1);
	LOG.error("ZooKeeper client connection to the ZooKeeper server has expired!");
	break;
	default:
	// do nothing
	break;
	}

However, if the server is in read-only mode, it will receive ConnectedReadOnly instead of SyncConnected. This causes the connection to time out without being completed.

Changes

Modified the switch statement in the ZooKeeperWatcherBase class to release clientConnectLatch when ConnectedReadOnly is received if the allowReadOnlyMode option is true.

By the way, allowReadOnlyMode is never set to true in BookKeeper. So this change would be useless for BookKeeper. However, it is useful for Pulsar. Because Pulsar also uses ZooKeeperWatcherBase and needs to be able to connect to ZooKeeper in read-only mode.
https://github.com/apache/pulsar/blob/cba1600d0f6a82f1ea194f3214a80f283fe8dc27/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/PulsarZooKeeperClient.java#L242-L244

[massakam]

Ran git rebase master.

[massakam]

@shoothzj @dlg99 @hangc0276
All checks have passed so would you please merge this?