Add local_current_membership table

[erikjohnston]

The idea here is to track membership state for local users separately from current_state_events, as currently synapse may delete the contents of current_state_events for that room if its no longer joined to that room (e.g. if the server tries and rejoins and fails). This breaks things like the leave section in /sync responses and exporting user data.

While that is a bit of an edge case, I do plan on changing synapse to delete the contents of current_state_events when the server leaves the room (to make it easy to answer the question "does the server share a room with a given remote user", and generally to remove footguns), which would then properly break those features.

Two things:

1. The initial insertion of local_current_membership happens in the delta script, which means it'll block start up. On a test on matrix.org this took ~40s, which is a while but not the end of the world. Doing it synchronously means we don't have to keep around various iterations of functions depending on the state of the various background updates, which got a bit fiddly.

2. There is a bug where if a user gets state reset out of a room that room doesn't appear in the leave section of sync (due to us just deleting the row in current_state_events), which I haven't attempted to fix as I don't really know what we want to do there (since we don't have a leave event to pass down to clients).

Note: The renaming of the functions are in a separate commit.

(This is part of the work for #6399)

[richvdh]

a few questions and suggestions below, but generally seems plausible.

It feels odd though. It feels like we're essentially duplicating a significant part of current_state_events mostly in order to act as a backup for when it gets cleared. Are there not easier ways to solve the problems that we are trying to solve by deleting stuff from current_state_events ?

[erikjohnston]

It feels odd though. It feels like we're essentially duplicating a significant part of current_state_events mostly in order to act as a backup for when it gets cleared. Are there not easier ways to solve the problems that we are trying to solve by deleting stuff from current_state_events ?

We want to answer the question of is the server in a room with a remote user efficiently, basically. (Which is fundamentally trying to solve the problem of figuring out if we need to resync keys for a remote user when they leave/join rooms).

I agree it feels odd, but a) I think not clearing out current_state_events is masking bugs as per PR description and b) its not obvious that having stale data in there isn't causing problems, e.g. get_rooms_for_user is called against remote users in the e2e key code.

Given the above I'm relatively happy that we should be clearing out current_state_events, but how we then track room membership for local users is a more open question. We could only track ban/leaves for rooms the server is no longer in in a separate table to avoid duplication, but not sure if that just makes things more fiddly or not.

[richvdh]

lgtm now modulo some nits about comments and the merge conflict.