Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Experimental support for MSC3266 Room Summary API #10394

Merged
merged 41 commits into from
Aug 16, 2021
Merged

Experimental support for MSC3266 Room Summary API #10394

merged 41 commits into from
Aug 16, 2021

Conversation

t3chguy
Copy link
Member

@t3chguy t3chguy commented Jul 14, 2021 •
edited by clokep
Loading

See MSC3266.

Pull Request Checklist

  • Pull request is based on the develop branch
  • Pull request includes a changelog file. The entry should:
    • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
    • Use markdown where necessary, mostly for code blocks.
    • End with either a period (.) or an exclamation mark (!).
    • Start with a capital letter.
  • Pull request includes a sign off
  • Code style is correct (run the linters)

t3chguy and others added 2 commits July 14, 2021 13:56
@t3chguy
Extract ResolveRoomIdMixin and reuse it for /_matrix/client/r0/join/{…
7d9c34d 
…roomIdOrAlias}

Signed-off-by: Michael Telatynski <michaelt@element.io>
@t3chguy
First cut of a local-only poc for MSC3266
ef295d1 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy t3chguy mentioned this pull request Jul 14, 2021
Open
clokep
clokep reviewed Jul 14, 2021
View reviewed changes
synapse/handlers/room_summary.py Outdated Show resolved Hide resolved
synapse/handlers/room_summary.py Outdated Show resolved Hide resolved
synapse/handlers/room_summary.py Outdated Show resolved Hide resolved
t3chguy added 7 commits July 15, 2021 08:45
@t3chguy
Update allowed_spaces to allowed_room_ids to match Space Summary API
85abffc 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
Extract _is_room_accessible into synapse.api.auth.Auth
b0c8e02 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
Extract build_room_entry into a mixin, updates Space Summary API allo…
d169cd8 
…wed_spaces->allowed_room_ids

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
remove unused variables
01b222a 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
delint
9227efc 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
fix types
c02485c 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
Fix typo in the msc3266 experimental config flag
0fa04fe 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
t3chguy added 9 commits July 19, 2021 16:59
@t3chguy
Merge branch 'develop' into msc3266
3982736
@t3chguy
Merge branch 'develop' of https://github.com/matrix-org/synapse into m…
957cb4f 
…sc3266

� Conflicts:
�	synapse/config/experimental.py
�	synapse/http/servlet.py

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
extract requester_can_see_room_entry to reuse in both space and room …
9994f12 
…summary federation

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
Add newsfragment
b2c8a7c 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
Add tests for the room_summary handler and make remote_room_hosts opt…
e6e7ecf 
…ional

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
Merge branch 'msc3266' of github.com:t3chguy/synapse into msc3266
85c166e 
� Conflicts:
�	synapse/config/experimental.py
@t3chguy
fix mypy lint and missing await
cc6c271 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
delint s'more
0c5952b 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
fix test_state by stubbing get_event_auth_handler
9e113aa 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy

This comment has been minimized.

Sign in to view
@t3chguy t3chguy marked this pull request as ready for review August 12, 2021 09:04
@t3chguy t3chguy marked this pull request as draft August 12, 2021 09:04
t3chguy added 5 commits August 12, 2021 10:21
@t3chguy
Merge branch 'develop' of https://github.com/matrix-org/synapse into m…
faa8bf1 
…sc3266

� Conflicts:
�	synapse/handlers/space_summary.py
�	synapse/rest/client/v1/room.py
�	tests/handlers/test_space_summary.py
@t3chguy
re-apply consolidation between room_summary and space_summary
caa8015 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
delint
f5c6740 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
Make requester optional to be more generically re-usable
2a9b961 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
update naming and comments
b277031 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@clokep clokep self-requested a review August 12, 2021 11:54
@t3chguy
Copy link
Member Author

t3chguy commented Aug 13, 2021

@clokep happy to break the refactoring out into a separate PR if that helps review

@t3chguy
delint
d4020d8 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@clokep clokep self-requested a review August 13, 2021 11:51
clokep
clokep reviewed Aug 13, 2021
View reviewed changes
synapse/handlers/room_summary.py Outdated Show resolved Hide resolved
synapse/handlers/room_summary.py Outdated Show resolved Hide resolved
tests/test_state.py Outdated Show resolved Hide resolved
synapse/api/auth.py Outdated Show resolved Hide resolved
t3chguy added 3 commits August 13, 2021 15:28
@t3chguy
revert bits of the PR
3fbf621 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
revert s'more
e99ca29 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy
minimise diff
2823459 
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
@t3chguy t3chguy requested a review from clokep August 13, 2021 14:33
@t3chguy t3chguy marked this pull request as ready for review August 13, 2021 15:52
clokep
clokep reviewed Aug 13, 2021
View reviewed changes
synapse/handlers/room_summary.py Outdated
Comment on lines 522 to 524
# If the room is not a space or the children don't matter, return just
# the room information.
if room_entry.get("room_type") != RoomTypes.SPACE or max_children == 0:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is from #10569 and means we don't need to use a separate method to summarize a single room.

@clokep
Copy link
Member

clokep commented Aug 13, 2021

I think this is good to go now. I ended up restructuring it a bit to fit in better with changes in #10569. I also stripped out a layer of unsupported federation code. I think it makes sense to add that all at once instead of having some methods which just always raise an exception.

@clokep
Copy link
Member

clokep commented Aug 16, 2021

I had to resolve some conflicts from develop, but I think this looks good.

@clokep clokep changed the title MSC3266 Room Summary API implementation Experimental support for MSC3266 Room Summary API Aug 16, 2021
@clokep clokep enabled auto-merge (squash) August 16, 2021 13:38
@clokep clokep disabled auto-merge August 16, 2021 14:12
@clokep clokep enabled auto-merge (squash) August 16, 2021 14:12
@clokep clokep merged commit 0ace38b into matrix-org:develop Aug 16, 2021
richvdh added a commit that referenced this pull request Aug 24, 2021
@richvdh
Merge tag 'v1.41.0rc1' into rav/sw1v-hotfixes
abe6a29 
Synapse 1.41.0rc1 (2021-08-18)
==============================

Features
--------

- Add `get_userinfo_by_id` method to ModuleApi. ([\#9581](#9581))
- Initial local support for [MSC3266](#10394), Room Summary over the unstable `/rooms/{roomIdOrAlias}/summary` API. ([\#10394](#10394))
- Experimental support for [MSC3288](matrix-org/matrix-spec-proposals#3288), sending `room_type` to the identity server for 3pid invites over the `/store-invite` API. ([\#10435](#10435))
- Add support for sending federation requests through a proxy. Contributed by @Bubu and @dklimpel. ([\#10475](#10475))
- Add support for "marker" events which makes historical events discoverable for servers that already have all of the scrollback history (part of [MSC2716](matrix-org/matrix-spec-proposals#2716)). ([\#10498](#10498))
- Add a configuration setting for the time a `/sync` response is cached for. ([\#10513](#10513))
- The default logging handler for new installations is now `PeriodicallyFlushingMemoryHandler`, a buffered logging handler which periodically flushes itself. ([\#10518](#10518))
- Add support for new redaction rules for historical events specified in [MSC2716](matrix-org/matrix-spec-proposals#2716). ([\#10538](#10538))
- Add a setting to disable TLS when sending email. ([\#10546](#10546))
- Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10549](#10549), [\#10560](#10560), [\#10569](#10569), [\#10574](#10574), [\#10575](#10575), [\#10579](#10579), [\#10583](#10583))
- Admin API to delete several media for a specific user. Contributed by @dklimpel. ([\#10558](#10558), [\#10628](#10628))
- Add support for routing `/createRoom` to workers. ([\#10564](#10564))
- Update the Synapse Grafana dashboard. ([\#10570](#10570))
- Add an admin API (`GET /_synapse/admin/username_available`) to check if a username is available (regardless of registration settings). ([\#10578](#10578))
- Allow editing a user's `external_ids` via the "Edit User" admin API. Contributed by @dklimpel. ([\#10598](#10598))
- The Synapse manhole no longer needs coroutines to be wrapped in `defer.ensureDeferred`. ([\#10602](#10602))
- Add option to allow modules to run periodic tasks on all instances, rather than just the one configured to run background tasks. ([\#10638](#10638))

Bugfixes
--------

- Add some clarification to the sample config file. Contributed by @Kentokamoto. ([\#10129](#10129))
- Fix a long-standing bug where protocols which are not implemented by any appservices were incorrectly returned via `GET /_matrix/client/r0/thirdparty/protocols`. ([\#10532](#10532))
- Fix exceptions in logs when failing to get remote room list. ([\#10541](#10541))
- Fix longstanding bug which caused the user "status" to be reset when the user went offline. Contributed by @dklimpel. ([\#10550](#10550))
- Allow public rooms to be previewed in the spaces summary APIs from [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10580](#10580))
- Fix a bug introduced in v1.37.1 where an error could occur in the asynchronous processing of PDUs when the queue was empty. ([\#10592](#10592))
- Fix errors on /sync when read receipt data is a string. Only affects homeservers with the experimental flag for [MSC2285](matrix-org/matrix-spec-proposals#2285) enabled. Contributed by @SimonBrandner. ([\#10606](#10606))
- Additional validation for the spaces summary API to avoid errors like `ValueError: Stop argument for islice() must be None or an integer`. The missing validation has existed since v1.31.0. ([\#10611](#10611))
- Revert behaviour introduced in v1.38.0 that strips `org.matrix.msc2732.device_unused_fallback_key_types` from `/sync` when its value is empty. This field should instead always be present according to [MSC2732](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2732-olm-fallback-keys.md). ([\#10623](#10623))

Improved Documentation
----------------------

- Add documentation for configuration a forward proxy. ([\#10443](#10443))
- Updated the reverse proxy documentation to highlight the homserver configuration that is needed to make Synapse aware that is is intentionally reverse proxied. ([\#10551](#10551))
- Update CONTRIBUTING.md to fix index links and the instructions for SyTest in docker. ([\#10599](#10599))

Deprecations and Removals
-------------------------

- No longer build `.deb` packages for Ubuntu 20.10 LTS Groovy Gorilla, which has now EOLed. ([\#10588](#10588))
- The `template_dir` configuration settings in the `sso`, `account_validity` and `email` sections of the configuration file are now deprecated in favour of the global `templates.custom_template_directory` setting. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\#10596](#10596))

Internal Changes
----------------

- Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\#10119](#10119))
- Reduce errors in PostgreSQL logs due to concurrent serialization errors. ([\#10504](#10504))
- Include room ID in ignored EDU log messages. Contributed by @ilmari. ([\#10507](#10507))
- Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10527](#10527), [\#10530](#10530))
- Fix CI to not break when run against branches rather than pull requests. ([\#10529](#10529))
- Mark all events stemming from the [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` endpoint as historical. ([\#10537](#10537))
- Clean up some of the federation event authentication code for clarity. ([\#10539](#10539), [\#10591](#10591))
- Convert `Transaction` and `Edu` objects to attrs. ([\#10542](#10542))
- Update `/batch_send` endpoint to only return `state_events` created by the `state_events_from_before` passed in. ([\#10552](#10552))
- Update contributing.md to warn against rebasing an open PR. ([\#10563](#10563))
- Remove the unused public rooms replication stream. ([\#10565](#10565))
- Clarify error message when failing to join a restricted room. ([\#10572](#10572))
- Remove references to BuildKite in favour of GitHub Actions. ([\#10573](#10573))
- Move `/batch_send` endpoint defined by [MSC2716](matrix-org/matrix-spec-proposals#2716) to the `/v2_alpha` directory. ([\#10576](#10576))
- Allow multiple custom directories in `read_templates`. ([\#10587](#10587))
- Re-organize the `synapse.federation.transport.server` module to create smaller files. ([\#10590](#10590))
- Flatten the `synapse.rest.client` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10600](#10600))
- Build Debian packages for Debian 12 (Bookworm). ([\#10612](#10612))
- Fix up a couple of links to the database schema documentation. ([\#10620](#10620))
- Fix a broken link to the upgrade notes. ([\#10631](#10631))
aaronraimist added a commit to aaronraimist/synapse that referenced this pull request Aug 31, 2021
@aaronraimist
Merge tag 'v1.41.0' into 2021-05
e1fe53a 
Synapse 1.41.0 (2021-08-24)
===========================

This release adds support for Debian 12 (Bookworm), but **removes support for Ubuntu 20.10 (Groovy Gorilla)**, which reached End of Life last month.

Note that when using workers the `/_synapse/admin/v1/users/{userId}/media` must now be handled by media workers. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information.

Features
--------

- Enable room capabilities ([MSC3244](matrix-org/matrix-spec-proposals#3244)) by default and set room version 8 as the preferred room version when creating restricted rooms. ([\matrix-org#10571](matrix-org#10571))

Synapse 1.41.0rc1 (2021-08-18)
==============================

Features
--------

- Add `get_userinfo_by_id` method to ModuleApi. ([\matrix-org#9581](matrix-org#9581))
- Initial local support for [MSC3266](matrix-org#10394), Room Summary over the unstable `/rooms/{roomIdOrAlias}/summary` API. ([\matrix-org#10394](matrix-org#10394))
- Experimental support for [MSC3288](matrix-org/matrix-spec-proposals#3288), sending `room_type` to the identity server for 3pid invites over the `/store-invite` API. ([\matrix-org#10435](matrix-org#10435))
- Add support for sending federation requests through a proxy. Contributed by @Bubu and @dklimpel. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\matrix-org#10596](matrix-org#10596)). ([\matrix-org#10475](matrix-org#10475))
- Add support for "marker" events which makes historical events discoverable for servers that already have all of the scrollback history (part of [MSC2716](matrix-org/matrix-spec-proposals#2716)). ([\matrix-org#10498](matrix-org#10498))
- Add a configuration setting for the time a `/sync` response is cached for. ([\matrix-org#10513](matrix-org#10513))
- The default logging handler for new installations is now `PeriodicallyFlushingMemoryHandler`, a buffered logging handler which periodically flushes itself. ([\matrix-org#10518](matrix-org#10518))
- Add support for new redaction rules for historical events specified in [MSC2716](matrix-org/matrix-spec-proposals#2716). ([\matrix-org#10538](matrix-org#10538))
- Add a setting to disable TLS when sending email. ([\matrix-org#10546](matrix-org#10546))
- Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10549](matrix-org#10549), [\matrix-org#10560](matrix-org#10560), [\matrix-org#10569](matrix-org#10569), [\matrix-org#10574](matrix-org#10574), [\matrix-org#10575](matrix-org#10575), [\matrix-org#10579](matrix-org#10579), [\matrix-org#10583](matrix-org#10583))
- Admin API to delete several media for a specific user. Contributed by @dklimpel. ([\matrix-org#10558](matrix-org#10558), [\matrix-org#10628](matrix-org#10628))
- Add support for routing `/createRoom` to workers. ([\matrix-org#10564](matrix-org#10564))
- Update the Synapse Grafana dashboard. ([\matrix-org#10570](matrix-org#10570))
- Add an admin API (`GET /_synapse/admin/username_available`) to check if a username is available (regardless of registration settings). ([\matrix-org#10578](matrix-org#10578))
- Allow editing a user's `external_ids` via the "Edit User" admin API. Contributed by @dklimpel. ([\matrix-org#10598](matrix-org#10598))
- The Synapse manhole no longer needs coroutines to be wrapped in `defer.ensureDeferred`. ([\matrix-org#10602](matrix-org#10602))
- Add option to allow modules to run periodic tasks on all instances, rather than just the one configured to run background tasks. ([\matrix-org#10638](matrix-org#10638))

Bugfixes
--------

- Add some clarification to the sample config file. Contributed by @Kentokamoto. ([\matrix-org#10129](matrix-org#10129))
- Fix a long-standing bug where protocols which are not implemented by any appservices were incorrectly returned via `GET /_matrix/client/r0/thirdparty/protocols`. ([\matrix-org#10532](matrix-org#10532))
- Fix exceptions in logs when failing to get remote room list. ([\matrix-org#10541](matrix-org#10541))
- Fix longstanding bug which caused the user's presence "status message" to be reset when the user went offline. Contributed by @dklimpel. ([\matrix-org#10550](matrix-org#10550))
- Allow public rooms to be previewed in the spaces summary APIs from [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10580](matrix-org#10580))
- Fix a bug introduced in v1.37.1 where an error could occur in the asynchronous processing of PDUs when the queue was empty. ([\matrix-org#10592](matrix-org#10592))
- Fix errors on /sync when read receipt data is a string. Only affects homeservers with the experimental flag for [MSC2285](matrix-org/matrix-spec-proposals#2285) enabled. Contributed by @SimonBrandner. ([\matrix-org#10606](matrix-org#10606))
- Additional validation for the spaces summary API to avoid errors like `ValueError: Stop argument for islice() must be None or an integer`. The missing validation has existed since v1.31.0. ([\matrix-org#10611](matrix-org#10611))
- Revert behaviour introduced in v1.38.0 that strips `org.matrix.msc2732.device_unused_fallback_key_types` from `/sync` when its value is empty. This field should instead always be present according to [MSC2732](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2732-olm-fallback-keys.md). ([\matrix-org#10623](matrix-org#10623))

Improved Documentation
----------------------

- Add documentation for configuring a forward proxy. ([\matrix-org#10443](matrix-org#10443))
- Updated the reverse proxy documentation to highlight the homserver configuration that is needed to make Synapse aware that is is intentionally reverse proxied. ([\matrix-org#10551](matrix-org#10551))
- Update CONTRIBUTING.md to fix index links and the instructions for SyTest in docker. ([\matrix-org#10599](matrix-org#10599))

Deprecations and Removals
-------------------------

- No longer build `.deb` packages for Ubuntu 20.10 Groovy Gorilla, which has now EOLed. ([\matrix-org#10588](matrix-org#10588))
- The `template_dir` configuration settings in the `sso`, `account_validity` and `email` sections of the configuration file are now deprecated in favour of the global `templates.custom_template_directory` setting. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\matrix-org#10596](matrix-org#10596))

Internal Changes
----------------

- Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\matrix-org#10119](matrix-org#10119))
- Reduce errors in PostgreSQL logs due to concurrent serialization errors. ([\matrix-org#10504](matrix-org#10504))
- Include room ID in ignored EDU log messages. Contributed by @ilmari. ([\matrix-org#10507](matrix-org#10507))
- Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10527](matrix-org#10527), [\matrix-org#10530](matrix-org#10530))
- Fix CI to not break when run against branches rather than pull requests. ([\matrix-org#10529](matrix-org#10529))
- Mark all events stemming from the [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` endpoint as historical. ([\matrix-org#10537](matrix-org#10537))
- Clean up some of the federation event authentication code for clarity. ([\matrix-org#10539](matrix-org#10539), [\matrix-org#10591](matrix-org#10591))
- Convert `Transaction` and `Edu` objects to attrs. ([\matrix-org#10542](matrix-org#10542))
- Update `/batch_send` endpoint to only return `state_events` created by the `state_events_from_before` passed in. ([\matrix-org#10552](matrix-org#10552))
- Update contributing.md to warn against rebasing an open PR. ([\matrix-org#10563](matrix-org#10563))
- Remove the unused public rooms replication stream. ([\matrix-org#10565](matrix-org#10565))
- Clarify error message when failing to join a restricted room. ([\matrix-org#10572](matrix-org#10572))
- Remove references to BuildKite in favour of GitHub Actions. ([\matrix-org#10573](matrix-org#10573))
- Move `/batch_send` endpoint defined by [MSC2716](matrix-org/matrix-spec-proposals#2716) to the `/v2_alpha` directory. ([\matrix-org#10576](matrix-org#10576))
- Allow multiple custom directories in `read_templates`. ([\matrix-org#10587](matrix-org#10587))
- Re-organize the `synapse.federation.transport.server` module to create smaller files. ([\matrix-org#10590](matrix-org#10590))
- Flatten the `synapse.rest.client` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\matrix-org#10600](matrix-org#10600))
- Build Debian packages for Debian 12 (Bookworm). ([\matrix-org#10612](matrix-org#10612))
- Fix up a couple of links to the database schema documentation. ([\matrix-org#10620](matrix-org#10620))
- Fix a broken link to the upgrade notes. ([\matrix-org#10631](matrix-org#10631))
babolivier added a commit to matrix-org/synapse-dinsic that referenced this pull request Sep 1, 2021
@babolivier
Merge tag 'v1.41.0' into babolivier/dinsic_1.41.0
ea37bf9 
Synapse 1.41.0 (2021-08-24)
===========================

This release adds support for Debian 12 (Bookworm), but **removes support for Ubuntu 20.10 (Groovy Gorilla)**, which reached End of Life last month.

Note that when using workers the `/_synapse/admin/v1/users/{userId}/media` must now be handled by media workers. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information.

Features
--------

- Enable room capabilities ([MSC3244](matrix-org/matrix-spec-proposals#3244)) by default and set room version 8 as the preferred room version when creating restricted rooms. ([\#10571](matrix-org/synapse#10571))

Synapse 1.41.0rc1 (2021-08-18)
==============================

Features
--------

- Add `get_userinfo_by_id` method to ModuleApi. ([\#9581](matrix-org/synapse#9581))
- Initial local support for [MSC3266](matrix-org/synapse#10394), Room Summary over the unstable `/rooms/{roomIdOrAlias}/summary` API. ([\#10394](matrix-org/synapse#10394))
- Experimental support for [MSC3288](matrix-org/matrix-spec-proposals#3288), sending `room_type` to the identity server for 3pid invites over the `/store-invite` API. ([\#10435](matrix-org/synapse#10435))
- Add support for sending federation requests through a proxy. Contributed by @Bubu and @dklimpel. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\#10596](matrix-org/synapse#10596)). ([\#10475](matrix-org/synapse#10475))
- Add support for "marker" events which makes historical events discoverable for servers that already have all of the scrollback history (part of [MSC2716](matrix-org/matrix-spec-proposals#2716)). ([\#10498](matrix-org/synapse#10498))
- Add a configuration setting for the time a `/sync` response is cached for. ([\#10513](matrix-org/synapse#10513))
- The default logging handler for new installations is now `PeriodicallyFlushingMemoryHandler`, a buffered logging handler which periodically flushes itself. ([\#10518](matrix-org/synapse#10518))
- Add support for new redaction rules for historical events specified in [MSC2716](matrix-org/matrix-spec-proposals#2716). ([\#10538](matrix-org/synapse#10538))
- Add a setting to disable TLS when sending email. ([\#10546](matrix-org/synapse#10546))
- Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10549](matrix-org/synapse#10549), [\#10560](matrix-org/synapse#10560), [\#10569](matrix-org/synapse#10569), [\#10574](matrix-org/synapse#10574), [\#10575](matrix-org/synapse#10575), [\#10579](matrix-org/synapse#10579), [\#10583](matrix-org/synapse#10583))
- Admin API to delete several media for a specific user. Contributed by @dklimpel. ([\#10558](matrix-org/synapse#10558), [\#10628](matrix-org/synapse#10628))
- Add support for routing `/createRoom` to workers. ([\#10564](matrix-org/synapse#10564))
- Update the Synapse Grafana dashboard. ([\#10570](matrix-org/synapse#10570))
- Add an admin API (`GET /_synapse/admin/username_available`) to check if a username is available (regardless of registration settings). ([\#10578](matrix-org/synapse#10578))
- Allow editing a user's `external_ids` via the "Edit User" admin API. Contributed by @dklimpel. ([\#10598](matrix-org/synapse#10598))
- The Synapse manhole no longer needs coroutines to be wrapped in `defer.ensureDeferred`. ([\#10602](matrix-org/synapse#10602))
- Add option to allow modules to run periodic tasks on all instances, rather than just the one configured to run background tasks. ([\#10638](matrix-org/synapse#10638))

Bugfixes
--------

- Add some clarification to the sample config file. Contributed by @Kentokamoto. ([\#10129](matrix-org/synapse#10129))
- Fix a long-standing bug where protocols which are not implemented by any appservices were incorrectly returned via `GET /_matrix/client/r0/thirdparty/protocols`. ([\#10532](matrix-org/synapse#10532))
- Fix exceptions in logs when failing to get remote room list. ([\#10541](matrix-org/synapse#10541))
- Fix longstanding bug which caused the user's presence "status message" to be reset when the user went offline. Contributed by @dklimpel. ([\#10550](matrix-org/synapse#10550))
- Allow public rooms to be previewed in the spaces summary APIs from [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10580](matrix-org/synapse#10580))
- Fix a bug introduced in v1.37.1 where an error could occur in the asynchronous processing of PDUs when the queue was empty. ([\#10592](matrix-org/synapse#10592))
- Fix errors on /sync when read receipt data is a string. Only affects homeservers with the experimental flag for [MSC2285](matrix-org/matrix-spec-proposals#2285) enabled. Contributed by @SimonBrandner. ([\#10606](matrix-org/synapse#10606))
- Additional validation for the spaces summary API to avoid errors like `ValueError: Stop argument for islice() must be None or an integer`. The missing validation has existed since v1.31.0. ([\#10611](matrix-org/synapse#10611))
- Revert behaviour introduced in v1.38.0 that strips `org.matrix.msc2732.device_unused_fallback_key_types` from `/sync` when its value is empty. This field should instead always be present according to [MSC2732](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2732-olm-fallback-keys.md). ([\#10623](matrix-org/synapse#10623))

Improved Documentation
----------------------

- Add documentation for configuring a forward proxy. ([\#10443](matrix-org/synapse#10443))
- Updated the reverse proxy documentation to highlight the homserver configuration that is needed to make Synapse aware that is is intentionally reverse proxied. ([\#10551](matrix-org/synapse#10551))
- Update CONTRIBUTING.md to fix index links and the instructions for SyTest in docker. ([\#10599](matrix-org/synapse#10599))

Deprecations and Removals
-------------------------

- No longer build `.deb` packages for Ubuntu 20.10 Groovy Gorilla, which has now EOLed. ([\#10588](matrix-org/synapse#10588))
- The `template_dir` configuration settings in the `sso`, `account_validity` and `email` sections of the configuration file are now deprecated in favour of the global `templates.custom_template_directory` setting. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\#10596](matrix-org/synapse#10596))

Internal Changes
----------------

- Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\#10119](matrix-org/synapse#10119))
- Reduce errors in PostgreSQL logs due to concurrent serialization errors. ([\#10504](matrix-org/synapse#10504))
- Include room ID in ignored EDU log messages. Contributed by @ilmari. ([\#10507](matrix-org/synapse#10507))
- Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\#10527](matrix-org/synapse#10527), [\#10530](matrix-org/synapse#10530))
- Fix CI to not break when run against branches rather than pull requests. ([\#10529](matrix-org/synapse#10529))
- Mark all events stemming from the [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` endpoint as historical. ([\#10537](matrix-org/synapse#10537))
- Clean up some of the federation event authentication code for clarity. ([\#10539](matrix-org/synapse#10539), [\#10591](matrix-org/synapse#10591))
- Convert `Transaction` and `Edu` objects to attrs. ([\#10542](matrix-org/synapse#10542))
- Update `/batch_send` endpoint to only return `state_events` created by the `state_events_from_before` passed in. ([\#10552](matrix-org/synapse#10552))
- Update contributing.md to warn against rebasing an open PR. ([\#10563](matrix-org/synapse#10563))
- Remove the unused public rooms replication stream. ([\#10565](matrix-org/synapse#10565))
- Clarify error message when failing to join a restricted room. ([\#10572](matrix-org/synapse#10572))
- Remove references to BuildKite in favour of GitHub Actions. ([\#10573](matrix-org/synapse#10573))
- Move `/batch_send` endpoint defined by [MSC2716](matrix-org/matrix-spec-proposals#2716) to the `/v2_alpha` directory. ([\#10576](matrix-org/synapse#10576))
- Allow multiple custom directories in `read_templates`. ([\#10587](matrix-org/synapse#10587))
- Re-organize the `synapse.federation.transport.server` module to create smaller files. ([\#10590](matrix-org/synapse#10590))
- Flatten the `synapse.rest.client` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10600](matrix-org/synapse#10600))
- Build Debian packages for Debian 12 (Bookworm). ([\#10612](matrix-org/synapse#10612))
- Fix up a couple of links to the database schema documentation. ([\#10620](matrix-org/synapse#10620))
- Fix a broken link to the upgrade notes. ([\#10631](matrix-org/synapse#10631))
Fizzadar pushed a commit to Fizzadar/synapse that referenced this pull request Oct 26, 2021
@tulir
Merge tag 'v1.41.0'
3d29f91 
Synapse 1.41.0 (2021-08-24)
===========================

This release adds support for Debian 12 (Bookworm), but **removes support for Ubuntu 20.10 (Groovy Gorilla)**, which reached End of Life last month.

Note that when using workers the `/_synapse/admin/v1/users/{userId}/media` must now be handled by media workers. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information.

Features
--------

- Enable room capabilities ([MSC3244](matrix-org/matrix-spec-proposals#3244)) by default and set room version 8 as the preferred room version when creating restricted rooms. ([\matrix-org#10571](matrix-org#10571))

Synapse 1.41.0rc1 (2021-08-18)
==============================

Features
--------

- Add `get_userinfo_by_id` method to ModuleApi. ([\matrix-org#9581](matrix-org#9581))
- Initial local support for [MSC3266](matrix-org#10394), Room Summary over the unstable `/rooms/{roomIdOrAlias}/summary` API. ([\matrix-org#10394](matrix-org#10394))
- Experimental support for [MSC3288](matrix-org/matrix-spec-proposals#3288), sending `room_type` to the identity server for 3pid invites over the `/store-invite` API. ([\matrix-org#10435](matrix-org#10435))
- Add support for sending federation requests through a proxy. Contributed by @Bubu and @dklimpel. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\matrix-org#10596](matrix-org#10596)). ([\matrix-org#10475](matrix-org#10475))
- Add support for "marker" events which makes historical events discoverable for servers that already have all of the scrollback history (part of [MSC2716](matrix-org/matrix-spec-proposals#2716)). ([\matrix-org#10498](matrix-org#10498))
- Add a configuration setting for the time a `/sync` response is cached for. ([\matrix-org#10513](matrix-org#10513))
- The default logging handler for new installations is now `PeriodicallyFlushingMemoryHandler`, a buffered logging handler which periodically flushes itself. ([\matrix-org#10518](matrix-org#10518))
- Add support for new redaction rules for historical events specified in [MSC2716](matrix-org/matrix-spec-proposals#2716). ([\matrix-org#10538](matrix-org#10538))
- Add a setting to disable TLS when sending email. ([\matrix-org#10546](matrix-org#10546))
- Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10549](matrix-org#10549), [\matrix-org#10560](matrix-org#10560), [\matrix-org#10569](matrix-org#10569), [\matrix-org#10574](matrix-org#10574), [\matrix-org#10575](matrix-org#10575), [\matrix-org#10579](matrix-org#10579), [\matrix-org#10583](matrix-org#10583))
- Admin API to delete several media for a specific user. Contributed by @dklimpel. ([\matrix-org#10558](matrix-org#10558), [\matrix-org#10628](matrix-org#10628))
- Add support for routing `/createRoom` to workers. ([\matrix-org#10564](matrix-org#10564))
- Update the Synapse Grafana dashboard. ([\matrix-org#10570](matrix-org#10570))
- Add an admin API (`GET /_synapse/admin/username_available`) to check if a username is available (regardless of registration settings). ([\matrix-org#10578](matrix-org#10578))
- Allow editing a user's `external_ids` via the "Edit User" admin API. Contributed by @dklimpel. ([\matrix-org#10598](matrix-org#10598))
- The Synapse manhole no longer needs coroutines to be wrapped in `defer.ensureDeferred`. ([\matrix-org#10602](matrix-org#10602))
- Add option to allow modules to run periodic tasks on all instances, rather than just the one configured to run background tasks. ([\matrix-org#10638](matrix-org#10638))

Bugfixes
--------

- Add some clarification to the sample config file. Contributed by @Kentokamoto. ([\matrix-org#10129](matrix-org#10129))
- Fix a long-standing bug where protocols which are not implemented by any appservices were incorrectly returned via `GET /_matrix/client/r0/thirdparty/protocols`. ([\matrix-org#10532](matrix-org#10532))
- Fix exceptions in logs when failing to get remote room list. ([\matrix-org#10541](matrix-org#10541))
- Fix longstanding bug which caused the user's presence "status message" to be reset when the user went offline. Contributed by @dklimpel. ([\matrix-org#10550](matrix-org#10550))
- Allow public rooms to be previewed in the spaces summary APIs from [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10580](matrix-org#10580))
- Fix a bug introduced in v1.37.1 where an error could occur in the asynchronous processing of PDUs when the queue was empty. ([\matrix-org#10592](matrix-org#10592))
- Fix errors on /sync when read receipt data is a string. Only affects homeservers with the experimental flag for [MSC2285](matrix-org/matrix-spec-proposals#2285) enabled. Contributed by @SimonBrandner. ([\matrix-org#10606](matrix-org#10606))
- Additional validation for the spaces summary API to avoid errors like `ValueError: Stop argument for islice() must be None or an integer`. The missing validation has existed since v1.31.0. ([\matrix-org#10611](matrix-org#10611))
- Revert behaviour introduced in v1.38.0 that strips `org.matrix.msc2732.device_unused_fallback_key_types` from `/sync` when its value is empty. This field should instead always be present according to [MSC2732](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2732-olm-fallback-keys.md). ([\matrix-org#10623](matrix-org#10623))

Improved Documentation
----------------------

- Add documentation for configuring a forward proxy. ([\matrix-org#10443](matrix-org#10443))
- Updated the reverse proxy documentation to highlight the homserver configuration that is needed to make Synapse aware that is is intentionally reverse proxied. ([\matrix-org#10551](matrix-org#10551))
- Update CONTRIBUTING.md to fix index links and the instructions for SyTest in docker. ([\matrix-org#10599](matrix-org#10599))

Deprecations and Removals
-------------------------

- No longer build `.deb` packages for Ubuntu 20.10 Groovy Gorilla, which has now EOLed. ([\matrix-org#10588](matrix-org#10588))
- The `template_dir` configuration settings in the `sso`, `account_validity` and `email` sections of the configuration file are now deprecated in favour of the global `templates.custom_template_directory` setting. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html) for more information. ([\matrix-org#10596](matrix-org#10596))

Internal Changes
----------------

- Improve event caching mechanism to avoid having multiple copies of an event in memory at a time. ([\matrix-org#10119](matrix-org#10119))
- Reduce errors in PostgreSQL logs due to concurrent serialization errors. ([\matrix-org#10504](matrix-org#10504))
- Include room ID in ignored EDU log messages. Contributed by @ilmari. ([\matrix-org#10507](matrix-org#10507))
- Add pagination to the spaces summary based on updates to [MSC2946](matrix-org/matrix-spec-proposals#2946). ([\matrix-org#10527](matrix-org#10527), [\matrix-org#10530](matrix-org#10530))
- Fix CI to not break when run against branches rather than pull requests. ([\matrix-org#10529](matrix-org#10529))
- Mark all events stemming from the [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` endpoint as historical. ([\matrix-org#10537](matrix-org#10537))
- Clean up some of the federation event authentication code for clarity. ([\matrix-org#10539](matrix-org#10539), [\matrix-org#10591](matrix-org#10591))
- Convert `Transaction` and `Edu` objects to attrs. ([\matrix-org#10542](matrix-org#10542))
- Update `/batch_send` endpoint to only return `state_events` created by the `state_events_from_before` passed in. ([\matrix-org#10552](matrix-org#10552))
- Update contributing.md to warn against rebasing an open PR. ([\matrix-org#10563](matrix-org#10563))
- Remove the unused public rooms replication stream. ([\matrix-org#10565](matrix-org#10565))
- Clarify error message when failing to join a restricted room. ([\matrix-org#10572](matrix-org#10572))
- Remove references to BuildKite in favour of GitHub Actions. ([\matrix-org#10573](matrix-org#10573))
- Move `/batch_send` endpoint defined by [MSC2716](matrix-org/matrix-spec-proposals#2716) to the `/v2_alpha` directory. ([\matrix-org#10576](matrix-org#10576))
- Allow multiple custom directories in `read_templates`. ([\matrix-org#10587](matrix-org#10587))
- Re-organize the `synapse.federation.transport.server` module to create smaller files. ([\matrix-org#10590](matrix-org#10590))
- Flatten the `synapse.rest.client` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\matrix-org#10600](matrix-org#10600))
- Build Debian packages for Debian 12 (Bookworm). ([\matrix-org#10612](matrix-org#10612))
- Fix up a couple of links to the database schema documentation. ([\matrix-org#10620](matrix-org#10620))
- Fix a broken link to the upgrade notes. ([\matrix-org#10631](matrix-org#10631))
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@jryans jryans jryans left review comments

@clokep clokep clokep approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@t3chguy @clokep @jryans