From 8fd9ac31dc0e064399f4c3cf45bb05c0b64806cb Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 7 Feb 2022 14:54:31 -0500 Subject: [PATCH 01/13] Add MSC for dir & filter on /relations. --- proposals/3715-relations-parity-messages.md | 47 +++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 proposals/3715-relations-parity-messages.md diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md new file mode 100644 index 00000000000..9d3bc589352 --- /dev/null +++ b/proposals/3715-relations-parity-messages.md @@ -0,0 +1,47 @@ +# MSC3715: `/relations` partiy with `/messages` + +[MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) introduced the +`/relations` API as an endpoint to paginate over the relations of an event. It +is described as behaving like the `/messages` API, but is missing parameters +to fully achieve that. + + +## Proposal + +Two new parameters be added to the `/relations` API for parity with `/messages`, +both [parameters have the same as definition as for `/messages`](https://spec.matrix.org/v1.2/client-server-api/#get_matrixclientv3roomsroomidmessages), +which are copied below: + +* `dir`: The direction to return events from. If this is set to `f`, events will + be returned in chronological order starting at `from`. If it is set to `b`, + events will be returned in *reverse* chronological order, again starting at `from`. + + One of: `[b f]`. +* `filter`: A JSON RoomEventFilter to filter returned events with. + +The rationale for both of these is parity between endpoints, which makes the +overall specification easier to reason about. Additionally, these parameters +make it easier to request missing relation information without needed to paginate +through known information -- this is particularly needed for mobile or +low-bandwidth devices where it is desired to keep the round-trips to the server +to a minimum. + + +## Potential issues + +The `dir` parameter needs to be optional (defaulting to `b`) to be backwards +compatible with MSC2675 (and Synapse's legacy implementation). + + +## Alternatives + +None. + + +## Security considerations + +None. + +## Unstable prefix + +None, assuming that this lands in the same Matrix specification version as MSC2675. From afb882b0ee93d744bfbb5c179071d10eae68bd32 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 7 Feb 2022 15:06:56 -0500 Subject: [PATCH 02/13] Fix typo. --- proposals/3715-relations-parity-messages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index 9d3bc589352..f49bfc45099 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -1,4 +1,4 @@ -# MSC3715: `/relations` partiy with `/messages` +# MSC3715: `/relations` parity with `/messages` [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) introduced the `/relations` API as an endpoint to paginate over the relations of an event. It From 17cc7800f03f0b3d74b3edfa00b9fc4b4ec51298 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 8 Feb 2022 09:54:45 -0500 Subject: [PATCH 03/13] Simplify additional parameters. --- proposals/3715-relations-parity-messages.md | 36 ++++++++++++--------- 1 file changed, 20 insertions(+), 16 deletions(-) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index f49bfc45099..36d4b26a011 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -8,29 +8,33 @@ to fully achieve that. ## Proposal -Two new parameters be added to the `/relations` API for parity with `/messages`, -both [parameters have the same as definition as for `/messages`](https://spec.matrix.org/v1.2/client-server-api/#get_matrixclientv3roomsroomidmessages), -which are copied below: +It is proposed to add the `dir`to the `/relations` API for parity with `/messages`, +it will [have the same as definition as for `/messages`](https://spec.matrix.org/v1.2/client-server-api/#get_matrixclientv3roomsroomidmessages), +which is copied below: -* `dir`: The direction to return events from. If this is set to `f`, events will - be returned in chronological order starting at `from`. If it is set to `b`, - events will be returned in *reverse* chronological order, again starting at `from`. +> The direction to return events from. If this is set to `f`, events will +> be returned in chronological order starting at `from`. If it is set to `b`, +> events will be returned in *reverse* chronological order, again starting at `from`. +> +> One of: `[b f]`. - One of: `[b f]`. -* `filter`: A JSON RoomEventFilter to filter returned events with. +Including the `dir` parameter will make it easier to request missing relation +information without needed to paginate through known information -- this is +particularly needed for mobile or low-bandwidth devices where it is desired to +keep the round-trips to the server to a minimum. -The rationale for both of these is parity between endpoints, which makes the -overall specification easier to reason about. Additionally, these parameters -make it easier to request missing relation information without needed to paginate -through known information -- this is particularly needed for mobile or -low-bandwidth devices where it is desired to keep the round-trips to the server -to a minimum. +It is additionally useful to unify similar endpoints as much as possible to avoid +surprises for developers. ## Potential issues -The `dir` parameter needs to be optional (defaulting to `b`) to be backwards -compatible with MSC2675 (and Synapse's legacy implementation). +Unlike for `/messages`, the `dir` parameter for `/relations` needs to be optional +(defaulting to `b`) to be backwards compatible with MSC2675 (and Synapse's +legacy implementation). + +`/messages` does have one additional parameter (`filter`) which would still not +be implemented for `/relations`. It is unclear if this parameter is useful here. ## Alternatives From f7f92a1429ab51b1c703ed12a877cd25613468d5 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 8 Feb 2022 14:45:10 -0500 Subject: [PATCH 04/13] Add alternative. --- proposals/3715-relations-parity-messages.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index 36d4b26a011..28c447f4c27 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -39,7 +39,10 @@ be implemented for `/relations`. It is unclear if this parameter is useful here. ## Alternatives -None. +The endpoint could be replaced with the `/event_relationships` API proposed in +[MSC2836](https://github.com/matrix-org/matrix-doc/pull/2836). That API would +add significant complexity over the current `/relations` API (e.g. arbitrary +of relations) and is not necessary to simply iterate events in the reverse ordering. ## Security considerations From 08f85e33bbacd807e4767b4ae4e4d82a5a0e81e1 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 11 Feb 2022 10:29:37 -0500 Subject: [PATCH 05/13] Add an unstable prefix. --- proposals/3715-relations-parity-messages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index 28c447f4c27..90416e77021 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -51,4 +51,4 @@ None. ## Unstable prefix -None, assuming that this lands in the same Matrix specification version as MSC2675. +Before standardization, `org.matrix.msc3715.dir` should be used in place of `dir`. From b3007ce692112da1abb7f4dee9394fc4bdd3f4a9 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 9 Mar 2022 12:37:50 -0500 Subject: [PATCH 06/13] Move note about backwards compat. --- proposals/3715-relations-parity-messages.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index 90416e77021..89fa244ea49 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -18,6 +18,11 @@ which is copied below: > > One of: `[b f]`. +In order to be backwards compatible with MSC2675 (and Synapse's legacy +implementation), the `dir` parameter must be optional (defaulting to `b`). This +differs from the specification of `/messages` where the `dir` parameter is +required.[^1] + Including the `dir` parameter will make it easier to request missing relation information without needed to paginate through known information -- this is particularly needed for mobile or low-bandwidth devices where it is desired to @@ -29,10 +34,6 @@ surprises for developers. ## Potential issues -Unlike for `/messages`, the `dir` parameter for `/relations` needs to be optional -(defaulting to `b`) to be backwards compatible with MSC2675 (and Synapse's -legacy implementation). - `/messages` does have one additional parameter (`filter`) which would still not be implemented for `/relations`. It is unclear if this parameter is useful here. @@ -52,3 +53,6 @@ None. ## Unstable prefix Before standardization, `org.matrix.msc3715.dir` should be used in place of `dir`. + +[^1]: Note that Synapse's implementation does not require a `dir` parameter for +the `/messages` API and defaults to `f` if it is not provided. From d80dad9264b160af24c5b514e45a2d0ec74ef300 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 9 Mar 2022 12:39:29 -0500 Subject: [PATCH 07/13] Add a link. --- proposals/3715-relations-parity-messages.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index 89fa244ea49..d88fb5f4229 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -54,5 +54,6 @@ None. Before standardization, `org.matrix.msc3715.dir` should be used in place of `dir`. -[^1]: Note that Synapse's implementation does not require a `dir` parameter for -the `/messages` API and defaults to `f` if it is not provided. +[^1]: Note that [Synapse's implementation](https://github.com/matrix-org/synapse/blob/10a88ba91cb16ccf757984f0a7d41ddf8b4dc07f/synapse/streams/config.py#L48) +does not require a `dir` parameter for the `/messages` API and defaults to `f` +if it is not provided. From b343651e0c2013650c8ede5f940829d20759cc3a Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 5 Apr 2022 09:27:52 -0400 Subject: [PATCH 08/13] Clarify proposal. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3715-relations-parity-messages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index d88fb5f4229..623554335ba 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -8,7 +8,7 @@ to fully achieve that. ## Proposal -It is proposed to add the `dir`to the `/relations` API for parity with `/messages`, +It is proposed to add the `dir` parameter to the `/relations` API for parity with `/messages`, it will [have the same as definition as for `/messages`](https://spec.matrix.org/v1.2/client-server-api/#get_matrixclientv3roomsroomidmessages), which is copied below: From 6f4a77c6606060dc6355f4fb2a344f8011e5e57b Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 6 Apr 2022 12:41:49 -0400 Subject: [PATCH 09/13] Re-title MSC Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3715-relations-parity-messages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index 623554335ba..58d4b33f650 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -1,4 +1,4 @@ -# MSC3715: `/relations` parity with `/messages` +# MSC3715: Add a pagination direction parameter to `/relations` [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) introduced the `/relations` API as an endpoint to paginate over the relations of an event. It From 53590c3bb3a0248eed62a97d40a0827900a9cd35 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 13 Jun 2022 14:27:51 -0400 Subject: [PATCH 10/13] Document another alternative. --- proposals/3715-relations-parity-messages.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index 58d4b33f650..f190e8c9fb8 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -45,6 +45,10 @@ The endpoint could be replaced with the `/event_relationships` API proposed in add significant complexity over the current `/relations` API (e.g. arbitrary of relations) and is not necessary to simply iterate events in the reverse ordering. +Instead of having a different default direction from `/messages`, a `v2` version +could be added which accepts the `dir` parameter with the same default directionality +as `/messages`, but the opposite of the current (`v1`) version of the endpoint. + ## Security considerations From 703ccc1287a4188b1afe36df6f93d58785b84271 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 13 Jun 2022 14:37:11 -0400 Subject: [PATCH 11/13] Add note about prev_batch token. --- proposals/3715-relations-parity-messages.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index f190e8c9fb8..0a1462e5b83 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -31,6 +31,14 @@ keep the round-trips to the server to a minimum. It is additionally useful to unify similar endpoints as much as possible to avoid surprises for developers. +Since this endpoint can now be used to paginate backwards over children events, +it is also useful for the `from` parameter to accept `prev_batch` values from +previous calls (as well as `next_batch` values, as is currently specified). The +[definition of the `from` parameter](https://spec.matrix.org/unstable/client-server-api/#get_matrixclientv1roomsroomidrelationseventid) +is updated: + +> Can be a `from_batch` token or `next_batch` token from a previous call, or a +> returned `start` token from `/messages`, or a `next_batch` token from `/sync`. ## Potential issues From 5620bb1cef968b989e16b56d8bc5f7d82ffc83ff Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 19 Jul 2022 08:49:43 -0400 Subject: [PATCH 12/13] Clarifications from review. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3715-relations-parity-messages.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index 0a1462e5b83..bd4597ccc40 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -8,8 +8,8 @@ to fully achieve that. ## Proposal -It is proposed to add the `dir` parameter to the `/relations` API for parity with `/messages`, -it will [have the same as definition as for `/messages`](https://spec.matrix.org/v1.2/client-server-api/#get_matrixclientv3roomsroomidmessages), +It is proposed to add the `dir` parameter to the `/relations` API for parity with `/messages`. +It will [have the same as definition as for `/messages`](https://spec.matrix.org/v1.2/client-server-api/#get_matrixclientv3roomsroomidmessages), which is copied below: > The direction to return events from. If this is set to `f`, events will @@ -37,9 +37,10 @@ previous calls (as well as `next_batch` values, as is currently specified). The [definition of the `from` parameter](https://spec.matrix.org/unstable/client-server-api/#get_matrixclientv1roomsroomidrelationseventid) is updated: -> Can be a `from_batch` token or `next_batch` token from a previous call, or a +> Can be a `from_batch` token **or `next_batch`** token from a previous call, or a > returned `start` token from `/messages`, or a `next_batch` token from `/sync`. +(Bold indicates new text.) ## Potential issues `/messages` does have one additional parameter (`filter`) which would still not From defb0dc1821509fb528f25a6c7444493ccd59cb5 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 14 Sep 2022 13:46:06 -0400 Subject: [PATCH 13/13] Flesh out description. --- proposals/3715-relations-parity-messages.md | 46 +++++++++++++++++++-- 1 file changed, 42 insertions(+), 4 deletions(-) diff --git a/proposals/3715-relations-parity-messages.md b/proposals/3715-relations-parity-messages.md index bd4597ccc40..6eaeda4dd36 100644 --- a/proposals/3715-relations-parity-messages.md +++ b/proposals/3715-relations-parity-messages.md @@ -1,10 +1,48 @@ # MSC3715: Add a pagination direction parameter to `/relations` [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675) introduced the -`/relations` API as an endpoint to paginate over the relations of an event. It -is described as behaving like the `/messages` API, but is missing parameters -to fully achieve that. - +`/relations` API as an endpoint to paginate over the relations of an event. Unfortunately, +it is missing a `dir` parameter which is necessary to properly load thread timelines +at an arbitrary location. + +Consider a long thread (e.g. consisting of 300 events), if a user receives a link +to an event somewhere in the middle it is desirable for the client to only display +a few dozen messages: the event itself and some context before and after it. As +the user scrolls additional content from the thread should be fetched. + +Currently clients are forced to fetch *the entire thread* until the target +message (plus any surround context) is reached in order to implement this +functionality. A simpler implementation would allow the client to: + +1. Load events in a thread backwards from a pagination token, +2. Load events in a thread forwards from a pagination token, +3. Load context around an event. + +1 & 3 are currently possible; this MSC attempts to solve condition 2. + +With the `dir` paraemter on `/relations, it now becomes possible for a client to: + +1. Call `/context` on the target event to get a pagination token. +2. Call `/relations` twice (once with `dir=b` and once with `dir=f`) on the same + pagination token to collect any contextual events. +3. Store the resulting pagination tokens to paginate further, if necessary. + +This allows the thread to be fetched in arbitrary chunks: + +```mermaid +flowchart RL + subgraph /relations?dir=f + $2-->$1 + end + $1-->$0 + subgraph /context/$0 + $0 + end + $0-->$-1 + subgraph /relations?dir=b + $-1-->$-2 + end +``` ## Proposal