From 78ba8f23ca152213c7c9ca7eb09a24c9bc246f7c Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 16 Nov 2021 18:08:51 +0000 Subject: [PATCH 1/6] Clarify event format text in room version specs Split the text about event IDs and event formats into separate sections. This is largely to make it easier to link to, but I think the resulting text makes more sense too. --- content/rooms/fragments/v4-event-explainer.md | 19 ----------- content/rooms/fragments/v4-event-format.md | 3 ++ content/rooms/fragments/v4-event-ids.md | 14 ++++++++ content/rooms/v3.md | 34 +++++++++---------- content/rooms/v4.md | 6 ++-- content/rooms/v5.md | 8 +++-- content/rooms/v6.md | 10 +++--- content/rooms/v7.md | 8 +++-- 8 files changed, 52 insertions(+), 50 deletions(-) delete mode 100644 content/rooms/fragments/v4-event-explainer.md create mode 100644 content/rooms/fragments/v4-event-format.md create mode 100644 content/rooms/fragments/v4-event-ids.md diff --git a/content/rooms/fragments/v4-event-explainer.md b/content/rooms/fragments/v4-event-explainer.md deleted file mode 100644 index 37a37644d0e..00000000000 --- a/content/rooms/fragments/v4-event-explainer.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -toc_hide: true ---- - -The event ID is the [reference -hash](/server-server-api#calculating-the-reference-hash-for-an-event) of -the event encoded using a variation of [Unpadded -Base64](/appendices#unpadded-base64) which replaces the 62nd and -63rd characters with `-` and `_` instead of using `+` and `/`. This -matches [RFC4648's definition of URL-safe -base64](https://tools.ietf.org/html/rfc4648#section-5). Event IDs are -still prefixed with `$` and may result in looking like -`$Rqnc-F-dvnEYJTyHq_iKxU2bZ1CI92-kuZq3a5lr5Zg`. - -Just like in room version 3, event IDs should not be sent over -federation to servers when the room uses this room version. On the -receiving end of an event, the server should compute the relevant event -ID for itself. Room version 3 also changes the format of `auth_events` -and `prev_events` in a PDU. diff --git a/content/rooms/fragments/v4-event-format.md b/content/rooms/fragments/v4-event-format.md new file mode 100644 index 00000000000..be4e675fcdb --- /dev/null +++ b/content/rooms/fragments/v4-event-format.md @@ -0,0 +1,3 @@ +Events in rooms of this version have the following structure: + +{{% definition path="api/server-server/definitions/pdu_v4" %}} diff --git a/content/rooms/fragments/v4-event-ids.md b/content/rooms/fragments/v4-event-ids.md new file mode 100644 index 00000000000..2fad7afab61 --- /dev/null +++ b/content/rooms/fragments/v4-event-ids.md @@ -0,0 +1,14 @@ +--- +toc_hide: true +--- + +{{% added-in this=true %}} The event ID is the [reference +hash](/server-server-api#calculating-the-reference-hash-for-an-event) of +the event encoded using a variation of [Unpadded +Base64](/appendices#unpadded-base64) which replaces the 62nd and +63rd characters with `-` and `_` instead of using `+` and `/`. This +matches [RFC4648's definition of URL-safe +base64](https://tools.ietf.org/html/rfc4648#section-5). + +Event IDs are still prefixed with `$` and may result in looking like +`$Rqnc-F-dvnEYJTyHq_iKxU2bZ1CI92-kuZq3a5lr5Zg`. diff --git a/content/rooms/v3.md b/content/rooms/v3.md index c7fffa3baae..1cb23cd468b 100644 --- a/content/rooms/v3.md +++ b/content/rooms/v3.md @@ -49,35 +49,33 @@ the use of a dedicated event ID, servers are required to track the hashes on an event to determine its ID. {{% /boxes/rationale %}} -The event ID is the [reference +{{% added-in this=true %}} The event ID is the [reference hash](/server-server-api#calculating-the-reference-hash-for-an-event) of the event encoded using [Unpadded Base64](/appendices#unpadded-base64), prefixed with `$`. A resulting event ID using this approach should look similar to `$CD66HAED5npg6074c6pDtLKalHjVfYb2q4Q3LZgrW6o`. -Event IDs should not be sent over federation to servers when the room -uses this room version. On the receiving end of an event, the server -should compute the relevant event ID for itself. +### Event format -Additionally, the `auth_events` and `prev_events` have had a format -change compared to other room versions to make it easier to handle. -Instead of a tuple of values, they are now plain lists of events. +When events are sent over federation, the `event_id` field is no longer +included. A server receiving an event should compute the relevant +event ID for itself. -{{% definition path="api/server-server/definitions/pdu_v3" %}} +Additionally, the format of the `auth_events` and `prev_events` fields are +changed: instead of lists of `(event_id, hash)` pairs, they are now plain lists +of event IDs. + +These changes to the format of an event mean that servers must be aware of the +version of the room containing an incoming event, so that the event can be +correctly parsed and handled. This is facilitated via changes to the +server-server API (such as the inclusion of `room_version` in the response to +`GET /_matrix/federation/v1/make_join/{roomId}/{userId}`.) -### Changes to APIs +The complete structure of a event in a v3 room is shown below. -Due to the event ID being removed from the event, some APIs need to -change. All APIs which currently accept an event ID must do so with the -new format. Servers must append the calculated event ID to all events -sent to clients where an event ID would normally be expected. +{{% definition path="api/server-server/definitions/pdu_v3" %}} -Because the format of events has changed, servers must be aware of the -room version where the event resides so that the server may parse and -handle the event. The federation API has taken this concern into -consideration by ensuring that servers are aware of (or can find) the -room version during a request. ### Authorization rules for events diff --git a/content/rooms/v4.md b/content/rooms/v4.md index d1f3347fe14..785dc6642a1 100644 --- a/content/rooms/v4.md +++ b/content/rooms/v4.md @@ -46,7 +46,7 @@ being interpretted differently by some reverse proxy software, and generally made administration harder. {{% /boxes/rationale %}} -{{% rver-fragment name="v4-event-explainer" %}} +{{% rver-fragment name="v4-event-ids" withVersioning="true" %}} ## Unchanged from v3 @@ -75,8 +75,8 @@ completeness. ### Event format -The event format is the same as [room version 3](/rooms/v3#event-ids), +The event format is the same as [room version 3](/rooms/v3#event-format), however the event IDs in the following example are updated to reflect the changes in this room version. -{{% definition path="api/server-server/definitions/pdu_v4" %}} +{{% rver-fragment name="v4-event-format" %}} diff --git a/content/rooms/v5.md b/content/rooms/v5.md index 117e44b80c9..387a435f92b 100644 --- a/content/rooms/v5.md +++ b/content/rooms/v5.md @@ -47,11 +47,13 @@ completeness. {{% rver-fragment name="v3-handling-redactions" %}} -### Event format +### Event IDs + +{{% rver-fragment name="v4-event-ids" %}} -{{% rver-fragment name="v4-event-explainer" %}} +### Event format -{{% definition path="api/server-server/definitions/pdu_v4" %}} +{{% rver-fragment name="v4-event-format" %}} ### Canonical JSON diff --git a/content/rooms/v6.md b/content/rooms/v6.md index eede41d7909..e974edcd60f 100644 --- a/content/rooms/v6.md +++ b/content/rooms/v6.md @@ -204,12 +204,14 @@ completeness. {{% rver-fragment name="v2-state-res" %}} -### Event format +### Event IDs + +{{% rver-fragment name="v4-event-ids" %}} -{{% rver-fragment name="v4-event-explainer" %}} +### Event format -{{% definition path="api/server-server/definitions/pdu_v4" %}} +{{% rver-fragment name="v4-event-format" %}} ### Handling redactions -{{% rver-fragment name="v3-handling-redactions" %}} \ No newline at end of file +{{% rver-fragment name="v3-handling-redactions" %}} diff --git a/content/rooms/v7.md b/content/rooms/v7.md index 62c7fac6888..17ec223bc33 100644 --- a/content/rooms/v7.md +++ b/content/rooms/v7.md @@ -191,11 +191,13 @@ completeness. {{% rver-fragment name="v2-state-res" %}} -### Event format +### Event IDs + +{{% rver-fragment name="v4-event-ids" %}} -{{% rver-fragment name="v4-event-explainer" %}} +### Event format -{{% definition path="api/server-server/definitions/pdu_v4" %}} +{{% rver-fragment name="v4-event-format" %}} ### Canonical JSON From 31e92f59cbac90accc9952b70109e7cee2af576e Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 16 Nov 2021 18:25:05 +0000 Subject: [PATCH 2/6] Add a changelog directory for room versions --- changelogs/room_versions/newsfragments/.gitignore | 1 + layouts/shortcodes/changelog/changelog-changes.html | 1 + scripts/generate-changelog.sh | 3 ++- 3 files changed, 4 insertions(+), 1 deletion(-) create mode 100644 changelogs/room_versions/newsfragments/.gitignore diff --git a/changelogs/room_versions/newsfragments/.gitignore b/changelogs/room_versions/newsfragments/.gitignore new file mode 100644 index 00000000000..b722e9e13ef --- /dev/null +++ b/changelogs/room_versions/newsfragments/.gitignore @@ -0,0 +1 @@ +!.gitignore \ No newline at end of file diff --git a/layouts/shortcodes/changelog/changelog-changes.html b/layouts/shortcodes/changelog/changelog-changes.html index edd46bce20d..4995a6bd3ed 100644 --- a/layouts/shortcodes/changelog/changelog-changes.html +++ b/layouts/shortcodes/changelog/changelog-changes.html @@ -36,6 +36,7 @@

Changes since last release

{{ partial "render-api-changes" (dict "title" "Application Service API" "id" "application-service-api" "path" (path.Join $path "application_service")) }} {{ partial "render-api-changes" (dict "title" "Identity Service API" "id" "identity-service-api" "path" (path.Join $path "identity_service")) }} {{ partial "render-api-changes" (dict "title" "Push Gateway API" "id" "push-gateway-api" "path" (path.Join $path "push_gateway")) }} +{{ partial "render-api-changes" (dict "title" "Room Versions" "id" "room-versions" "path" (path.Join $path "room_versions")) }} {{ define "partials/render-api-changes" }}

{{ .title }}

diff --git a/scripts/generate-changelog.sh b/scripts/generate-changelog.sh index 6e2c2f1131e..4209f4a36aa 100644 --- a/scripts/generate-changelog.sh +++ b/scripts/generate-changelog.sh @@ -11,7 +11,8 @@ cd changelogs # Pre-cleanup just in case it wasn't done on the last run rm -f rendered.* -# Reversed order so the push gateway ends up on the bottom +# Reversed order so that room versions ends up on the bottom +towncrier --name "Room Versions" --dir "./room_versions" --config "./pyproject.toml" --yes towncrier --name "Push Gateway API" --dir "./push_gateway" --config "./pyproject.toml" --yes towncrier --name "Identity Service API" --dir "./identity_service" --config "./pyproject.toml" --yes towncrier --name "Application Service API" --dir "./application_service" --config "./pyproject.toml" --yes From 5734d898b2d7af8e30a49c972e84cba8428477af Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 16 Nov 2021 18:27:18 +0000 Subject: [PATCH 3/6] newsfile --- changelogs/room_versions/3501.clarification | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelogs/room_versions/3501.clarification diff --git a/changelogs/room_versions/3501.clarification b/changelogs/room_versions/3501.clarification new file mode 100644 index 00000000000..7b6abe1bc1c --- /dev/null +++ b/changelogs/room_versions/3501.clarification @@ -0,0 +1 @@ +Clarifications to sections on event IDs and event formats. From e72da2a9007c11490680c597c3abb9a955e35d79 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 16 Nov 2021 18:30:17 +0000 Subject: [PATCH 4/6] fix newsfragment --- changelogs/room_versions/{ => newsfragments}/3501.clarification | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename changelogs/room_versions/{ => newsfragments}/3501.clarification (100%) diff --git a/changelogs/room_versions/3501.clarification b/changelogs/room_versions/newsfragments/3501.clarification similarity index 100% rename from changelogs/room_versions/3501.clarification rename to changelogs/room_versions/newsfragments/3501.clarification From 60e8d9ab4e92937bec0363708f04bb1418fb61e3 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Fri, 19 Nov 2021 10:52:57 +0000 Subject: [PATCH 5/6] Update content/rooms/fragments/v4-event-format.md Co-authored-by: Travis Ralston --- content/rooms/fragments/v4-event-format.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/content/rooms/fragments/v4-event-format.md b/content/rooms/fragments/v4-event-format.md index be4e675fcdb..938f0f61743 100644 --- a/content/rooms/fragments/v4-event-format.md +++ b/content/rooms/fragments/v4-event-format.md @@ -1,3 +1,7 @@ +--- +toc_hide: true +--- + Events in rooms of this version have the following structure: {{% definition path="api/server-server/definitions/pdu_v4" %}} From dccf40c86b69617b7b2e5692a73a5cb4e2f83a02 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Fri, 19 Nov 2021 11:02:45 +0000 Subject: [PATCH 6/6] linkify reference to --- content/rooms/v3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rooms/v3.md b/content/rooms/v3.md index 1cb23cd468b..baa0fcb02a1 100644 --- a/content/rooms/v3.md +++ b/content/rooms/v3.md @@ -70,7 +70,7 @@ These changes to the format of an event mean that servers must be aware of the version of the room containing an incoming event, so that the event can be correctly parsed and handled. This is facilitated via changes to the server-server API (such as the inclusion of `room_version` in the response to -`GET /_matrix/federation/v1/make_join/{roomId}/{userId}`.) +[`GET /_matrix/federation/v1/make_join/{roomId}/{userId}`](/server-server-api/#get_matrixfederationv1make_joinroomiduserid).) The complete structure of a event in a v3 room is shown below.