Distinguish 'client' from 'federation' events

changelogs/client_server/newsfragments/3658.clarification

@@ -0,0 +1 @@
+Distinguish between "raw" event format as exchanged by the Federation API, and the "serialised" event formats as used in the client-server and AS APIs.

content/client-server-api/_index.md

@@ -1440,32 +1440,6 @@ any given point in time:
 
     [E0]->[E1]->[E2]->[E3]->[E4]->[E5]
 
-{{% boxes/warning %}}
-The format of events can change depending on room version. Check the
-[room version specification](/rooms) for specific
-details on what to expect for event formats. Examples contained within
-the client-server specification are expected to be compatible with all
-specified room versions, however some differences may still apply.
-
-For this version of the specification, clients only need to worry about
-the event ID format being different depending on room version. Clients
-should not be parsing the event ID, and instead be treating it as an
-opaque string. No changes should be required to support the currently
-available room versions.
-{{% /boxes/warning %}}
-
-{{% boxes/warning %}}
-Event bodies are considered untrusted data. This means that any application using
-Matrix must validate that the event body is of the expected shape/schema
-before using the contents verbatim.
-
-**It is not safe to assume that an event body will have all the expected
-fields of the expected types.**
-
-See [MSC2801](https://github.com/matrix-org/matrix-doc/pull/2801) for more
-detail on why this assumption is unsafe.
-{{% /boxes/warning %}}
-
 ### Types of room events
 
 Room events are split into two categories:
@@ -1496,25 +1470,31 @@ sent by clients and other clients would receive it through Matrix,
 assuming the client has access to the `com.example` namespace.
 {{% /boxes/note %}}
 
-Note that the structure of these events may be different than those in
-the server-server API.
-
-#### Event fields
+### Room event format
 
-{{% event-fields event_type="event" %}}
+The "raw" format of a room event, which is used internally by homeservers
+and between homeservers via the Server-Server API, depends on the ["room
+version"](/rooms) in use by the room. See, for example, the defintions
+in [room version 1](/rooms/v1#event-format) and [room version
+3](/rooms/v3#event-format).
 
-#### Room event fields
+However, it is unusual that a Matrix client would encounter this raw event
+format. Instead, homeservers are responsible for "serialising" events into the
+format shown below so that they can be easily parsed by clients.
 
-{{% event-fields event_type="room_event" %}}
+{{% boxes/warning %}}
+Event bodies are considered untrusted data. This means that any application using
+Matrix must validate that the event body is of the expected shape/schema
+before using the contents verbatim.
 
-#### State event fields
+**It is not safe to assume that an event body will have all the expected
+fields of the expected types.**
 
-In addition to the fields of a Room Event, State Events have the
-following field:
+See [MSC2801](https://github.com/matrix-org/matrix-doc/pull/2801) for more
+detail on why this assumption is unsafe.
+{{% /boxes/warning %}}
 
-| Key          | Type         | Description                                                                                                  |
-|--------------|--------------|--------------------------------------------------------------------------------------------------------------|
-| state_key    | string       | **Required.** A unique key which defines the overwriting semantics for this piece of room state. This value is often a zero-length string. The presence of this key makes this event a State Event. State keys starting with an `@` are reserved for referencing user IDs, such as room members. With the exception of a few events, state events set with a given user's ID as the state key MUST only be set by that user.         |
+{{% definition path="api/client-server/definitions/serialised_event" %}}
 
 ### Stripped state
 
@@ -1580,8 +1560,7 @@ updates not being sent.
 ### Size limits
 
 The complete event MUST NOT be larger than 65536 bytes, when formatted
-as a [PDU for the Server-Server
-protocol](/server-server-api/#pdus), including any
+with the [raw event format](#room-event-format), including any
 signatures, and encoded as [Canonical
 JSON](/appendices#canonical-json).
 

data/api/application-service/transactions.yaml

@@ -65,8 +65,7 @@ paths:
                 description: |-
                   A list of events, formatted as per the Client-Server API.
                 items:
-                  type: object
-                  title: Event
+                  $ref: "../client-server/definitions/serialised_event.yaml"
             required: ["events"]
       responses:
         200:

data/api/client-server/definitions/serialised_event.yaml

@@ -0,0 +1,33 @@
+# Copyright 2016-2021 The Matrix.org Foundation C.I.C.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+title: SerialisedEvent
+description: |-
+  The format used to serialise events when they are returned from a homeserver to a client
+  via the Client-Server API, or sent to an Application Service via the Application Services API.
+type: object
+allOf:
+  - $ref: "serialised_event_without_room_id.yaml"
+  - properties:
+      room_id:
+        description: The ID of the room associated with this event.
+        type: string
+        example: '!jEsUZKDJdhlrceRyVU:example.org'
+
+      unsigned:
+        properties:
+            redacted_because:
+              title: SerialisedEvent
+    required:
+      - room_id

data/api/client-server/definitions/serialised_event_without_room_id.yaml

@@ -0,0 +1,94 @@
+# Copyright 2016-2021 The Matrix.org Foundation C.I.C.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+title: SerialisedEventWithoutRoomID
+description: |-
+  The format used to serialise events when they are returned from
+  API endpoints such as `/sync`, where the room_id is implied elsewhere
+  in the response.
+type: object
+required:
+  - event_id
+  - type
+  - sender
+  - origin_server_ts
+  - content
+properties:
+  event_id:
+    description: The globally unique identifier for this event.
+    type: string
+    example: '$143273582443PhrSn'
+  type:
+    description: The type of the event.
+    type: string
+    example: 'm.room.member'
+  state_key:
+    description: |-
+      Present if, and only if, this event is a *state* event. The key making
+      this piece of state unique in the room.
+    type: string
+    example: '@user:example.org'
+  sender:
+    description: Contains the fully-qualified ID of the user who sent this event.
+    type: string
+    example: "@example:example.org"
+  origin_server_ts:
+    description: |-
+      Timestamp (in milliseconds since the unix epoch) on originating homeserver
+      when this event was sent.
+    type: integer
+    format: int64
+    example: 1432735824653
+  content:
+    description: |-
+      The body of this event, as created by the client which sent it.
+    type: object
+    example: {
+      "membership": "join"
+    }
+  unsigned:
+    title: UnsignedData
+    type: object
+    description: Contains optional extra information about the event.
+    properties:
+      age:
+        description: The time in milliseconds that has elapsed since the event was
+          sent. This field is generated by the local homeserver, and may be incorrect
+          if the local time on at least one of the two servers is out of sync, which can
+          cause the age to either be negative or greater than it actually is.
+        type: integer
+      redacted_because:
+        description: The event that redacted this event, if any.
+        type: object
+        title: SerialisedEventWithoutRoomID
+      transaction_id:
+        description: |
+          The client-supplied transaction ID, for example, provided via
+          `PUT /_matrix/client/r0/rooms/{roomId}/send/{eventType}/{txnId}`,
+          if the client being given the event is the same one which sent it.
+        type: string
+      prev_content:
+        description: |
+          The previous `content` for this event. This field is generated
+          by the local homeserver, and is only returned if the event is a state event,
+          and the client has permission to see the previous content.
+        x-changedInMatrixVersion:
+          1.2: |
+            Previously, this field was specified at the top level of returned
+            events rather than in `unsigned` (with the exception of the [`GET
+            .../notifications`](/client-server-api/#get_matrixclientv3notifications)
+            endpoint), though in practice no known server implementations honoured
+            this.
+        title: EventContent
+        type: object

data/api/client-server/definitions/state_event_batch.yaml

@@ -15,15 +15,7 @@ properties:
   events:
     description: List of events.
     items:
-      allOf:
-      - $ref: ../../../event-schemas/schema/core-event-schema/sync_state_event.yaml
-      type: object
-      required:
-      - event_id
-      #- room_id - Not in /sync
-      - sender
-      - origin_server_ts
-      - state_key
+      $ref: serialised_event_without_room_id.yaml
     type: array
 type: object
 title: StateEventBatch

data/api/client-server/definitions/timeline_batch.yaml

@@ -12,8 +12,6 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-allOf:
-- $ref: room_event_batch.yaml
 properties:
   limited:
     description: True if the number of events returned was limited by the `limit`
@@ -27,5 +25,12 @@ properties:
       If no earlier events are available, this property may be omitted from
       the response.
     type: string
+  events:
+    description: List of events.
+    type: array
+    items:
+      $ref: "serialised_event_without_room_id.yaml"
 type: object
 title: TimelineBatch
+required:
+  - events

data/api/client-server/event_context.yaml

@@ -92,28 +92,25 @@ paths:
                   A list of room events that happened just before the
                   requested event, in reverse-chronological order.
                 items:
-                  allOf:
-                    - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml"
+                  $ref: "definitions/serialised_event.yaml"
               event:
                 description: |-
                   Details of the requested event.
                 allOf:
-                  - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml"
+                  - $ref: "definitions/serialised_event.yaml"
               events_after:
                 type: array
                 description: |-
                   A list of room events that happened just after the
                   requested event, in chronological order.
                 items:
-                  allOf:
-                    - "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml"
+                  $ref: "definitions/serialised_event.yaml"
               state:
                 type: array
                 description: |-
                   The state of the room at the last event returned.
                 items:
-                  allOf:
-                    - "$ref": "../../event-schemas/schema/core-event-schema/state_event.yaml"
+                  $ref: "definitions/serialised_event.yaml"
           examples:
             application/json: {
               "end": "t29-57_2_0_2",

data/api/client-server/message_pagination.yaml

@@ -125,7 +125,7 @@ paths:
                   are available. Clients should continue to paginate until no `end` property
                   is returned.
                 items:
-                  "$ref": "../../event-schemas/schema/core-event-schema/room_event.yaml"
+                  "$ref": "definitions/serialised_event.yaml"
               state:
                 type: array
                 description: |-
@@ -138,7 +138,7 @@ paths:
                   sent to the client in prior calls to this endpoint, assuming
                   the membership of those members has not changed.
                 items:
-                  $ref: "../../event-schemas/schema/core-event-schema/state_event.yaml"
+                  $ref: "definitions/serialised_event.yaml"
             required: [start, chunk]
           examples:
             application/json: {

data/api/client-server/notifications.yaml

@@ -112,8 +112,7 @@ paths:
                       type: object
                       title: Event
                       description: The Event object for the event that triggered the notification.
-                      allOf:
-                        - $ref: ../../event-schemas/schema/core-event-schema/sync_room_event.yaml
+                      "$ref": "definitions/serialised_event_without_room_id.yaml"
                     profile_tag:
                       type: string
                       description: The profile tag of the rule that matched this event.