[Bug] collision while using ConsistentHashing & consumer name that end by a number

[jbarotin]

Search before asking

• I searched in the issues and found nothing similar.

Version

pulsar v3.1.1 & ubuntu 22.04

Minimal reproduce step

Consistent hashing enabled in brocker config :
subscriptionKeySharedUseConsistentHashing=true

Create Pulsar consumers in a loop like this :

for (int i = 0; i < 40; i++) {

  ConsumerBuilder<GenericRecord> consumerBuilder = client.newConsumer(myschema)
         .topic("mytopic")
         .consumerName(hostname+"-"+i)
         .subscriptionName("mysubcription")
         .subscriptionMode(SubscriptionMode.Durable)
         .subscriptionType(SubscriptionType.Key_Shared)
         .keySharedPolicy(KeySharedPolicy.autoSplitHashRange().setAllowOutOfOrderDelivery(true))
         .subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
         .receiverQueueSize(1000)
         .maxAcknowledgmentGroupSize(100)
         .acknowledgmentGroupTime(100, TimeUnit.MILLISECONDS)
         .messageListener((MessageListener<GenericRecord>) (consumer, msg) -> {
            // my consumer code
          }
}  

What did you expect to see?

While I'm executing
pulsar-admin topics stats public/default/mytopic-partition-0

if I look keyHashRanges field, I found that the following range "[1932476396, 1932750362]" is associated to two different consumers host2-2 and host2-25 .

What did you see instead?

While I'm executing
pulsar-admin topics stats public/default/mytopic

if I look keyHashRanges field, I want to found every range associated to only one consumer.

Anything else?

On slack @lhotari told me that hash collision that happens with my use case where Ihave a consumerName ending with a number, and pointed me to this code

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ConsistentHashingStickyKeyConsumerSelector.java

Lines 61 to 63 in 08b28f5

	for (int i = 0; i < numberOfPoints; i++) {
	String key = consumer.consumerName() + i;
	int hash = Murmur3_32Hash.getInstance().makeHash(key.getBytes());

A simple workaround is to use .consumerName(hostname+"-"+i+"-"), ie add some character other than a number to the end of the consumer name.

Are you willing to submit a PR?

• I'm willing to submit a PR!