Message IDs are not the same across sources

[sekaiwish]

After using this plugin to feed an API, I discovered that when using multiple sources, the IDs are not the same across them, thus defeating the purpose of using it to remove duplicated messages.

In my instance, the same message was generated with the ID 1610613727408 and 1610613727421.

[sekaiwish]

One potential solution to this would be to generate a hash using any algorithm without taking the zoned timestamp into account.

[hex-agon]

This might have changed with the clans update but the id used is what the client itself uses to de-duplicate them and, in the past, it was guaranteed to be unique at the time of generation. Note that the id is basically worldId << 32 + sequence where sequence is a simple sequence that increases by 1 starting from 0 for each cross-world message sent in the world. I'll see if i can find some time to re-validate it.

Time hashing is hard because the user clock can be out of sync, so you'd have to add some tolerance to it and by adding tolerance, you might discard two equal messages sent in quick succession.

[hex-agon]

I've re-checked and it seems like they can diff now between accounts:

Account 1:

MESSAGE_FRIENDCHANNEL
worldId=489
sequence=806461
MESSAGE_FRIENDCHANNEL
worldId=301
sequence=857930

Account 2:

MESSAGE_FRIENDCHANNEL
worldId=489
sequence=806462
MESSAGE_FRIENDCHANNEL
worldId=301
sequence=857931

I don't see a quick solution for this at the moment.

[KiwiZ0]

I just see if the ID's between the same message from the same sender are within 1000 in the ID field, works very well