Federation sender doesn't start federating for 20+ minutes after coming up

[turt2live]

The logs indicate that it is acquiring a ton of locks for joined room hosts, and this is observed on t2bot.io. Given those two variables, it's probably voyager or something sending some sort of event right at startup that causes the whole thing to deadlock federation while it calculates hosts in all 3000 rooms.

[richvdh]

can you share logs demonstrating the problem pleeeease?

[richvdh]

Having spent a while looking at @turt2live's logs and metrics, I have the following conclusions:

The outgoing transaction rate doesn't actually drop to zero, but certainly it sits around 1Hz (with the odd spike) for some time after restart, compared to a more normal rate of around 30-40Hz.

However, if you exclude transactions which do not contain any events (ie, they are only presence, to_device messages, device list updates, etc), then the transaction rate is consistently 1Hz, with no significant drop after a restart.

My suspicion is therefore that what we're actually seeing is the server transmitting unnecessary presence storms as per #3962 most of the time. Accordingly my inclination is to fix #3962 first, and then see how this looks afterwards.

[turt2live]

#3962 fixed this