What is the purpose of OutboxWorker EnforceMessageOrder? Why not rely on Confluent idempotent producer configuration?

[cookie-bytes]

Hi,

I’m curious about the use of EnforceMessageOrder on the OutboxWorker. When should it be used and what is its purpose? We have set our producer to EnableIdempotence and Acks All, which seems to guarantee message order for a topic and partition in the event of a message error. We are satisfied with this capability. Is EnforceMessageOrder intended for situations where EnableIdempotence cannot be relied upon for a topic and message order must be enforced across different topics?

Thanks

[BEagle1984]

Hi, as written in the field's documentation: specifies whether the messages must be produced in the same order as they were added to the queue. If set to true the message order will be ensured, retrying the same message until it can be successfully produced.

It's intended as overall ordering, across all topics/endpoints.

If you set that to false it simply means that the worker will simply skip the message in case of failure and retry that one later. It might seem a good idea, in case you are unable to write in a certain topic (permission issue, or if using multiple brokers) but you might end up messing the ordering inside a topic as well, in case of a transitory issue (imagine that there are 10 messages to be sent to topic A, if the first 3 fails to be sent, the worker would still continue and try with the next ones, potentially violating the ordering).

[cookie-bytes]

If we use the EnableIdempotence set as True from our testing it seemed to not send later events to the topic\partition which had the issue with the event. Is this not the case? Do we need to test further to prove this?

[cookie-bytes]

Hi, thank you for your quick response.

When we set EnforceMessageOrder to True and we do encounter an exception processing a message in the batch, for example, message size too big, we have found that the outbox worker gets stuck in the WaitAllAsync while loop.
It gets stuck because we have exited but _pendingProduceOperations is greater than zero.
Is this a bug or by design?

Thanks again.

[BEagle1984]

Probably. 🤕

[BEagle1984]

v4.3.2 is out. Hopefully fixed for real now...

[cookie-bytes]

Just had a look. I think the lastProduced should go after the ProcessMessageAsync.
I am running the ProcessQueue_ProduceError_Retried test and 3 events are removed from the outbox and not just the 2 processed.
I am looking at that test. Is there a way to assert that only the expected events are removed by the IOutboxReader?
Thanks

[BEagle1984]

Theoretically that test ensures that all the messages are produced eventually. It seemed to work correctly now.
I'll check again.

[cookie-bytes]

@BEagle1984 I have had a chance to write a test to assert with the IOutboxReader and the current implementation is fine. I am not sure what happened when I was debugging last Friday. Thank you