Kafka consume more than 1 message per time

[higor-correa]

Hi.

I used the basic example to consume and publish to kafka, it worked. I realized that it executes one message per time.
How do I make it execute more than one than one message per time.

[BEagle1984]

With Kafka you will always only get one message after the other (per consumer).

Silverback creates "a thread" per each topic partition and tries to parallelize a bit, but the degree of parallelism greatly depends on the amount of messages you consume and how they are spread across the partitions.

I generally suggest to add more partitions if you need to parallelize more, but I'm talking like 12/15 anyway, not hundreds.

Another way to improve the performance is processing the messages in batches. You still consume sequentially but the commit will be batched and you can process multiple messages concurrently (if you want, if you build your parallelization logic). The idea is that you can for example batch multiple inserts into the database and execute them all at once, greatly reducing the connection overhead.

A batch per partition will be created.

Have a look at https://silverback-messaging.net/concepts/broker/inbound.html#batch-processing.

[higor-correa]

Thank you so much for the explanation!