Clarify event format text in room version specs

changelogs/room_versions/newsfragments/.gitignore

@@ -0,0 +1 @@
+!.gitignore

changelogs/room_versions/newsfragments/3501.clarification

@@ -0,0 +1 @@
+Clarifications to sections on event IDs and event formats.

content/rooms/fragments/v4-event-format.md

@@ -0,0 +1,7 @@
+---
+toc_hide: true
+---
+
+Events in rooms of this version have the following structure:
+
+{{% definition path="api/server-server/definitions/pdu_v4" %}}

content/rooms/fragments/v4-event-ids.md

@@ -0,0 +1,14 @@
+---
+toc_hide: true
+---
+
+{{% added-in this=true %}} The event ID is the [reference
+hash](/server-server-api#calculating-the-reference-hash-for-an-event) of
+the event encoded using a variation of [Unpadded
+Base64](/appendices#unpadded-base64) which replaces the 62nd and
+63rd characters with `-` and `_` instead of using `+` and `/`. This
+matches [RFC4648's definition of URL-safe
+base64](https://tools.ietf.org/html/rfc4648#section-5).
+
+Event IDs are still prefixed with `$` and may result in looking like
+`$Rqnc-F-dvnEYJTyHq_iKxU2bZ1CI92-kuZq3a5lr5Zg`.

content/rooms/v3.md

@@ -49,35 +49,33 @@ the use of a dedicated event ID, servers are required to track the
 hashes on an event to determine its ID.
 {{% /boxes/rationale %}}
 
-The event ID is the [reference
+{{% added-in this=true %}} The event ID is the [reference
 hash](/server-server-api#calculating-the-reference-hash-for-an-event) of
 the event encoded using [Unpadded
 Base64](/appendices#unpadded-base64), prefixed with `$`. A
 resulting event ID using this approach should look similar to
 `$CD66HAED5npg6074c6pDtLKalHjVfYb2q4Q3LZgrW6o`.
 
-Event IDs should not be sent over federation to servers when the room
-uses this room version. On the receiving end of an event, the server
-should compute the relevant event ID for itself.
+### Event format
 
-Additionally, the `auth_events` and `prev_events` have had a format
-change compared to other room versions to make it easier to handle.
-Instead of a tuple of values, they are now plain lists of events.
+When events are sent over federation, the `event_id` field is no longer
+included. A server receiving an event should compute the relevant
+event ID for itself.
 
-{{% definition path="api/server-server/definitions/pdu_v3" %}}
+Additionally, the format of the `auth_events` and `prev_events` fields are
+changed: instead of lists of `(event_id, hash)` pairs, they are now plain lists
+of event IDs.
+
+These changes to the format of an event mean that servers must be aware of the
+version of the room containing an incoming event, so that the event can be
+correctly parsed and handled. This is facilitated via changes to the
+server-server API (such as the inclusion of `room_version` in the response to
+[`GET /_matrix/federation/v1/make_join/{roomId}/{userId}`](/server-server-api/#get_matrixfederationv1make_joinroomiduserid).)
 
-### Changes to APIs
+The complete structure of a event in a v3 room is shown below.
 
-Due to the event ID being removed from the event, some APIs need to
-change. All APIs which currently accept an event ID must do so with the
-new format. Servers must append the calculated event ID to all events
-sent to clients where an event ID would normally be expected.
+{{% definition path="api/server-server/definitions/pdu_v3" %}}
 
-Because the format of events has changed, servers must be aware of the
-room version where the event resides so that the server may parse and
-handle the event. The federation API has taken this concern into
-consideration by ensuring that servers are aware of (or can find) the
-room version during a request.
 
 ### Authorization rules for events
 

content/rooms/v4.md

@@ -46,7 +46,7 @@ being interpretted differently by some reverse proxy software, and
 generally made administration harder.
 {{% /boxes/rationale %}}
 
-{{% rver-fragment name="v4-event-explainer" %}}
+{{% rver-fragment name="v4-event-ids" withVersioning="true" %}}
 
 ## Unchanged from v3
 
@@ -75,8 +75,8 @@ completeness.
 
 ### Event format
 
-The event format is the same as [room version 3](/rooms/v3#event-ids),
+The event format is the same as [room version 3](/rooms/v3#event-format),
 however the event IDs in the following example are updated to reflect
 the changes in this room version.
 
-{{% definition path="api/server-server/definitions/pdu_v4" %}}
+{{% rver-fragment name="v4-event-format" %}}

content/rooms/v5.md

@@ -47,11 +47,13 @@ completeness.
 
 {{% rver-fragment name="v3-handling-redactions" %}}
 
-### Event format
+### Event IDs
+
+{{% rver-fragment name="v4-event-ids" %}}
 
-{{% rver-fragment name="v4-event-explainer" %}}
+### Event format
 
-{{% definition path="api/server-server/definitions/pdu_v4" %}}
+{{% rver-fragment name="v4-event-format" %}}
 
 ### Canonical JSON
 

content/rooms/v6.md

@@ -204,12 +204,14 @@ completeness.
 
 {{% rver-fragment name="v2-state-res" %}}
 
-### Event format
+### Event IDs
+
+{{% rver-fragment name="v4-event-ids" %}}
 
-{{% rver-fragment name="v4-event-explainer" %}}
+### Event format
 
-{{% definition path="api/server-server/definitions/pdu_v4" %}}
+{{% rver-fragment name="v4-event-format" %}}
 
 ### Handling redactions
 
-{{% rver-fragment name="v3-handling-redactions" %}}
+{{% rver-fragment name="v3-handling-redactions" %}}

content/rooms/v7.md

@@ -191,11 +191,13 @@ completeness.
 
 {{% rver-fragment name="v2-state-res" %}}
 
-### Event format
+### Event IDs
+
+{{% rver-fragment name="v4-event-ids" %}}
 
-{{% rver-fragment name="v4-event-explainer" %}}
+### Event format
 
-{{% definition path="api/server-server/definitions/pdu_v4" %}}
+{{% rver-fragment name="v4-event-format" %}}
 
 ### Canonical JSON
 

layouts/shortcodes/changelog/changelog-changes.html

@@ -36,6 +36,7 @@ <h2 id=api-changes>Changes since last release</h2>
 {{ partial "render-api-changes" (dict "title" "Application Service API" "id" "application-service-api" "path" (path.Join $path "application_service")) }}
 {{ partial "render-api-changes" (dict "title" "Identity Service API" "id" "identity-service-api" "path" (path.Join $path "identity_service")) }}
 {{ partial "render-api-changes" (dict "title" "Push Gateway API" "id" "push-gateway-api" "path" (path.Join $path "push_gateway")) }}
+{{ partial "render-api-changes" (dict "title" "Room Versions" "id" "room-versions" "path" (path.Join $path "room_versions")) }}
 
 {{ define "partials/render-api-changes" }}
 <h3 id="{{.id}}">{{ .title }}</h3>

scripts/generate-changelog.sh

@@ -11,7 +11,8 @@ cd changelogs
 # Pre-cleanup just in case it wasn't done on the last run
 rm -f rendered.*
 
-# Reversed order so the push gateway ends up on the bottom
+# Reversed order so that room versions ends up on the bottom
+towncrier --name "Room Versions" --dir "./room_versions" --config "./pyproject.toml" --yes
 towncrier --name "Push Gateway API" --dir "./push_gateway" --config "./pyproject.toml" --yes
 towncrier --name "Identity Service API" --dir "./identity_service" --config "./pyproject.toml" --yes
 towncrier --name "Application Service API" --dir "./application_service" --config "./pyproject.toml" --yes