Ref counting MFiles #887
Ref counting MFiles #887
Conversation
probably wont do much against race conditions, but at least in theory it could be better
Can cause seg faults if a follower is still trying to replicate it.
So that the errors can be caught in specs. In lavinmq.cr is the exit done
Add a random delay to the stream queue GC loops, so that not all loops are executing at the same time.
So that the files are closed after each spec
There was a problem hiding this comment.
I tested and it crashed:
mmap /tmp/lavinmq-cluster/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8/96e488d035f3c5aec50b494f6a7973ff287b6bdd/msgs.0000000001
unmap /tmp/lavinmq-cluster/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8/96e488d035f3c5aec50b494f6a7973ff287b6bdd/msgs.0000000001
mmap /tmp/lavinmq-cluster/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8/96e488d035f3c5aec50b494f6a7973ff287b6bdd/msgs.0000000001
unmap /tmp/lavinmq-cluster/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8/96e488d035f3c5aec50b494f6a7973ff287b6bdd/msgs.0000000001
2024-12-17T07:52:37.467231Z INFO lmq.message_store[queue: "perf-test", vhost: "/"] Loaded 1 segments, 0 messages
mmap /tmp/lavinmq-cluster/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8/96e488d035f3c5aec50b494f6a7973ff287b6bdd/msgs.0000000001
mmap /tmp/lavinmq-cluster/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8/96e488d035f3c5aec50b494f6a7973ff287b6bdd/acks.0000000001
mmap /tmp/lavinmq-cluster/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8/96e488d035f3c5aec50b494f6a7973ff287b6bdd/msgs.0000000002
unmap /tmp/lavinmq-cluster/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8/96e488d035f3c5aec50b494f6a7973ff287b6bdd/acks.0000000001
mmap /tmp/lavinmq-cluster/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8/96e488d035f3c5aec50b494f6a7973ff287b6bdd/acks.0000000002
unmap /tmp/lavinmq-cluster/42099b4af021e53fd8fd4e056c2568d7c2e3ffa8/96e488d035f3c5aec50b494f6a7973ff287b6bdd/msgs.0000000001
[1] 70930 segmentation fault ./bin/lavinmq-ref-counting-mfiles -c clustering.ini
Thoughts:
- May there be a case when a MFile has been closed and
borrowis called? - Shouldn't ByteMessage borrow? Or, what about we pass a "lease" with
Envelope?
WHAT is this pull request doing?
Keep track of usage of MFiles, and only mmap when needed
HOW can this pull request be tested?
Specs