Hopefully most of these changes are explained through the added comments and error
messages. The changes are also described conceptually in the MSC:
https://github.com/Sorunome/matrix-doc/blob/soru/knock/proposals/2403-knock.md#join-rules

We're ditching the usual idea of having two endpoints for each
membership-related endpoint as per the MSC. Thus knocking only gets the
more powerful variant (the one that supports room aliases as well as
IDs. The reason is also optional.

The other small change is just to ensure displaynames get added to the
content of this particular membership event.

This option serves the same purpose as the existing
room_invite_state_types option, which defines what state events are sent
over to a user that is invited to a room. This information is necessary
for the user - who isn't in the room yet - to get some metadata about
the room in order to display it in a pretty fashion in the user's
pending-invites list.

It includes information such as the room's name, avatar, topic,
canonical alias, room encryption state etc. as well as the invite
membership event which the invited user's homeserver can reference.

This new option is the exact same, but is sent by a homeserver in the
room to the knocker during the knock process. This option will actually
be utilised in a later commit.

Most of this is explained in the linked MSC (and don't miss the sequence
diagram in the MSC comments), but roughly knocking takes inspiration from
room joins and room invites. This commit is the room join stuff.

First the knocking homeserver POSTs to the make_knock endpoint on
another homeserver. The other homeserver will send back a knock event
that is valid for the knocking user and the room that they are knocking
on. The knocking homeserver will sign the event and send it back, before
the other homeserver takes that event and then sends it into the room on
the knocking homeserver's behalf.

It's worth noting that the accepting/rejecting knocks all happen over
existing room invite/leave flows. A homeserver rescinding its knock as
well is also just sending a leave.

Once the event has been inserted into the room, the homeserver that's in
the room will send back a 200 and an empty JSON dict to confirm
everything went well to the knocker. In a future commit, this dict will
instead be filled with some stripped state events from the room which
the knocking homeserver will pass back to the knocking user.

And yes, the logging statements in this commit are intentional. They're
consistent with the rest of the file :)

Here we finally send the stripped state events back to the knocking
homeserver, which then ingests and stores those events.

A future commit will actually start sending those events down /sync to
the relevant user.

So we've got federation so that homeservers can communicate knocking
information between them - but how does that information actually get
down to the client? The client knows that it knocked successfully from a
200 in its original request, but what else does it need? This commit
adds a new "knock" section to /sync (in addition to "invite", "join",
and "leave") all help give the client the information it needs.

The new "knock" section is used for sending down the stripped state
events we collected earlier. The client will use these to display the
room and its metadata in a little "pending knocks" section or similar.

This is all this commit adds. If the user's knock has been accepted or
rejected, they will receive that information in the "join" or "leave"
sections of /sync.

Most of this code is just cribbing off the invite and join sync code yet
again, with some minor differences. For instance, we don't need to
exclude knock events from sync if the sender is in your ignore list, as
you are the only ones that can send knocks for yourself.

The structure of the "knock" dict in sync is modeled after "invite", as
clients also receive stripped state in that. The structure can be viewed
in the linked MSC.

Tiny commit to just bring knocking up to feature parity.

This just adds some database methods that we'll need to use when
implementing rescinding of knocks of clients. They are equivalent to
their invite-related counterparts.

As mentioned in the MSC, a user can rescind (take back) a knock while it
is pending by sending a leave event to the room. This will set their
membership to leave instead of knock.

Now, this functionality already worked before this commit for rooms that
the homeserver was already in. What didn't work was:

* Rescinding a knock over federation to a room with active homeservers
* Rescinding a knock over federation to a room with inactive homeservers

This commit addresses the second bullet point, and leaves the first
bullet point as a TODO (as it is an edge case an not immediately obvious
how it would be done).

What this commit does is crib off the same functionality as locally
rejecting an invite. That occurs when we are unable to contact the
homeserver that originally sent us an invite. Instead an out-of-band
leave membership event will be generated and sent to clients locally.

The same is happening here. You can mostly ignore the new
generate_local_out_of_band_membership methods, those are just some
structural bits to allow us to call that method from RoomMemberHandler.

The real meat of this commit is moving about and adding some logic in
`update_membership_locked`, specifically for when we're updating a
user's membership to "leave". There was already some code in there to
check whether the room to send the leave to was a room the homeserver is
not currently a part of. In that case, we'd remote reject the knock.
This commit just extends that to also rescind knocks if the user's
membership in the room is currently "knock". We skip the remote attempt
for now and go straight to generating a local leave membership event.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Turns out there was some extra boilerplate around replication that is necessary
to rescind knocks on worker setups. Workers need to call out to the master
process to help remotely accept/reject invites, and the same is true for
accepting/rescinding knocks.

This commit adds replication client functions and handlers for remote_knock and
remote_rescind_knock.

As part of this, I turned generate_local_out_of_band_leave back into a private
function. It was originally being called by
RoomMemberHandler.update_membership_locked as it was simply a shortcut to
generating a local leave event (aka rescinding a knock locally, but not
informing any other servers about it). This shortcut was temporary while
actually remote rescinding is left as a TODO. Instead of setting up all the
boilerplate code for generate_local_out_of_band_leave, only to later replace it
with remote_rescind_knock (which would call generate_local_out_of_band_leave if
it failed to rescind a knock remotely), we just set up replication for
remote_rescind_knock now.

In the future, remote_rescind_knock will do more than just immediately giving
up and rescinding the knock locally.

…e_string_from_args

Clients can tell if knocking is available by checking for the presence of a room
version containing knocking in /_matrix/client/r0/capabilities.

…update.

It was pointed out that the method in
RoomMemberHandler.update_membership_locked for retrieving the previous invite
or knock event was overly complex. We already knew the ID of the room, yet got
a list of all rooms that the user has been invited to/knocked on from the
database. We then filtered that list, and extracted some metadata from it that
allowed us to get the invite/knock event ID.

This was quite ineffecient, and I've updated both methods to simply get the
current state of the room given the room ID, extract the latest m.room.member
event for the user, and check that it has invite/knock membership.

There's still some inefficiency between pulling out the knock event, passing
the event ID to remote_rescind_knock, to which remote_rescind_knock then
pulls the associated event out of the database again - however I think this
is necessary given that we may need to call remote_rescind_knock over
replication, where passing an event ID is easier than a serialised event.
It also more closely resembles remote_reject_invite's function signature.

I assume the same does not need to happen to replication endpoints as they are not
client or federation facing.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

We introduced this option in this PR to mirror room_knock_invite_types, but ended up
questioning the worth of both, and ultimately decided that neither were very useful.

Context: https://github.com/matrix-org/synapse/pull/6739/files#r527027645 and
matrix-org#8807

This file had a minor edit to update the wording.

This is another PR that grew out of matrix-org#6739.

The existing code for checking whether a user is currently invited to a room when they want to leave the room looks like the following:

https://github.com/matrix-org/synapse/blob/f737368a26bb9eea401fcc3a5bdd7e0b59e91f09/synapse/handlers/room_member.py#L518-L540

It calls `get_invite_for_local_user_in_room`, which will actually query *all* rooms the user has been invited to, before iterating over them and matching via the room ID. It will then return a tuple of a lot of information which we pull the event ID out of.

I need to do a similar check for knocking, but this code wasn't very efficient. I then tried to write a different implementation using `StateHandler.get_current_state` but this actually didn't work as we haven't *joined* the room yet - we've only been invited to it. That means that only certain tables in Synapse have our desired `invite` membership state. One of those tables is `local_current_membership`.

So I wrote a store method that just queries that table instead

Additionally clean up the logic of this invite/knock code a bit, and it was
originally a little tricky to follow the flow.

… functions"

This reverts commit 3fb055c. It was found that
these arguments are necessary and nice to keep around following a discussion
in matrix-org#8809.

…tate

The type change was necessary as DEFAULT_ROOM_STATE_TYPES is actually
a list of str.

This specifically tests the stripped state events that come down /sync, as
at the current stage in the MSC the exact types to send are not fixed.

There's also a tiny type fix included in a nearby test.

shorthand being on has no effect if the path starts with /_matrix

…e included

The changes included here cover a few things:

* A new testcase was added that performs federated make_knock and send_knock calls,
  checking that all expected stripped room state is included in the response to
  send_knock.
* Abstracting the stripped room state checks from SyncKnockTestCase into generic
  functions that have been moved into FederationKnockingTestCase. We then import
  these into SyncKnockTestCase instead.
    * Two functions were created. One that sends the test state into a room, and
      another that takes that test state + the stripped state events received
      while knocking and compares them.
* The abstracted function for checking state now also checks that not all state
  in the room is given to the knocker. For instance, we probably don't want to
  give widget URLs to someone knocking on a room before we've accepted them!

  Thus we include a "secret state event" in the room and check that the knocker
  does not receive it.

When writing the federation tests it took quite a while for me to figure out how
to get a homeserver to accept calls from another homeserver that didn't exist.
To do so, I ended up mocking out the event signature and auth checking. These
checks aren't relavent to this test, and are instead checked by those in
Complement.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

…hip and room versions

Unfortunately the diff here came out a lot more awful than I would've liked, but
really we're just moving these two methods into a mixin class and then using that
instead of the two methods separately.

I also ensured that the top-level SECRET_STATE_EVENT_TYPE was used instead of
creating an identical variable.

Kept this change in a separate commit in order to not make the last commit's
diff even more confusing.

… var

I thought these were necessary but apparently not so. They were
originally cribbed from the federation catchip unit tests, and
apparently not necessary for ours. I suppose because we do not
hit the federation ratelimit, and the auth bypassing done via
mocks in the test itself is already sufficient.

…uth.check for all tests

We needed to disable event_auth checking for this test. However, the method in
which we were doing so was disabling event_auth.check across all tests. Thus
after this test ran, other unit tests failed in weird and wonderful ways.

Take a different approach which acts on the homeserver created for this test,
rather than mocking a file directly.

…ck_fixed

… instead

This informs the remote server of the room versions we support. If the room we're trying to
knock on has a version that is not one of our supported room versions, the remote server
will return an unsupported room version error.

Noticed in matrix-org/matrix-spec-proposals#2403 (comment)