Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add custom well-known #13035

Merged
merged 12 commits into from
Jun 16, 2022
Merged

Conversation

Vetchu
Copy link
Contributor

@Vetchu Vetchu commented Jun 13, 2022

Fixes #8739

I think I did correct tests, but please recheck. I'm not sure if this is the way that is desired by the matrix maintainers - I'm open to feedback.

Signed-off-by: Jacek Kusnierz jacek.kusnierz@tum.de

Vetchu added 4 commits June 13, 2022 17:58
Signed-off-by: Jacek Kusnierz <jacek.kusnierz@tum.de>
Signed-off-by: Jacek Kusnierz <jacek.kusnierz@tum.de>
Signed-off-by: Jacek Kusnierz <jacek.kusnierz@tum.de>
@Vetchu Vetchu marked this pull request as ready for review June 13, 2022 17:08
@Vetchu Vetchu requested a review from a team as a code owner June 13, 2022 17:08
@dklimpel
Copy link
Contributor

Can you add some documentation to synapse/docs/usage/configuration/config_documentation.md?

Signed-off-by: Jacek Kusnierz <jacek.kusnierz@tum.de>
Copy link
Contributor

@DMRobertson DMRobertson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. I think my comments can broadly be summarised as "let's make sure users can't misconfigure things">

changelog.d/13035.feature Outdated Show resolved Hide resolved
docs/usage/configuration/config_documentation.md Outdated Show resolved Hide resolved
synapse/config/server.py Outdated Show resolved Hide resolved
docs/usage/configuration/config_documentation.md Outdated Show resolved Hide resolved
tests/rest/test_well_known.py Outdated Show resolved Hide resolved
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
@DMRobertson DMRobertson self-assigned this Jun 14, 2022
Copy link
Contributor

@DMRobertson DMRobertson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting there!

synapse/config/server.py Outdated Show resolved Hide resolved
synapse/config/server.py Outdated Show resolved Hide resolved
synapse/config/server.py Outdated Show resolved Hide resolved
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
@Vetchu
Copy link
Contributor Author

Vetchu commented Jun 15, 2022

One thing that I'm unsure of is, whether extra_well_known_content can be served without any other property in place. Should it work only if public_baseurl is declared or should it be treated independently?

Signed-off-by: Jacek Kusnierz <jacek.kusnierz@tum.de>
@Vetchu
Copy link
Contributor Author

Vetchu commented Jun 15, 2022

I added two test cases to illustrate my point - only public_baseurl and serve_server_wellknown make the server return well_known. Should extra_well_known_content make the server return .well-known as well?

@Vetchu Vetchu requested a review from DMRobertson June 15, 2022 12:14
@DMRobertson
Copy link
Contributor

One thing that I'm unsure of is, whether extra_well_known_content can be served without any other property in place. Should it work only if public_baseurl is declared or should it be treated independently?

Good question. The spec says that the well-known response must contain the base url (as m.homeserver). But the public_baseurl defaults to the server_name if it's not provided:

public_baseurl = config.get("public_baseurl")
if public_baseurl is None:
public_baseurl = f"https://{self.server_name}/"
logger.info("Using default public_baseurl %s", public_baseurl)

Therefore we should be safe to include the custom content as-is.

Comment on lines 62 to 85
@unittest.override_config(
{
"default_identity_server": "https://testis",
}
)
def test_client_well_known_only_default_identity_server(self) -> None:
channel = self.make_request(
"GET", "/.well-known/matrix/client", shorthand=False
)

self.assertEqual(channel.code, HTTPStatus.NOT_FOUND)

@unittest.override_config(
{
"extra_well_known_content": {"custom": False},
}
)
def test_client_well_known_only_extra_well_known(self) -> None:
channel = self.make_request(
"GET", "/.well-known/matrix/client", shorthand=False
)

self.assertEqual(channel.code, HTTPStatus.NOT_FOUND)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised these tests pass as written.

  • override_config makes additions to the default_config; fields that aren't mentioned don't get changed
  • a public_baseurl will be set based on the server_name from default_config (see also - Add custom well-known #13035 (comment))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh. We only actually serve well-known if a public_baseurl is explicitly set:

# For the sake of backwards compatibility with existing installations, this is
# True if public_baseurl is specified explicitly, and otherwise False. (The
# reasoning here is that we have no way of knowing that the default
# public_baseurl is actually correct for existing installations - many things
# will not work correctly, but that's (probably?) better than sending clients
# to a completely broken URL.
self.serve_client_wellknown = False
public_baseurl = config.get("public_baseurl")
if public_baseurl is None:
public_baseurl = f"https://{self.server_name}/"
logger.info("Using default public_baseurl %s", public_baseurl)
else:
self.serve_client_wellknown = True
if public_baseurl[-1] != "/":
public_baseurl += "/"
self.public_baseurl = public_baseurl

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these tests add much compared to test_client_well_known_no_public_baseurl (since that is the condition). Can you remove these please?

@DMRobertson DMRobertson added the X-Awaiting-Changes A contributed PR which needs changes and re-review before it can be merged label Jun 15, 2022
@Vetchu Vetchu requested a review from DMRobertson June 16, 2022 08:21
Copy link
Contributor

@DMRobertson DMRobertson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM! I'm going to ask another Synapse team member to take a quick look to sanity check my thinking.

@DMRobertson
Copy link
Contributor

Many thanks all!

@DMRobertson DMRobertson merged commit 0ef1307 into matrix-org:develop Jun 16, 2022
@rwjack
Copy link

rwjack commented Jun 16, 2022

Okay, now we can remove the .well-known/matrix/client entries from our nginx servers, but what about .well-known/matrix/server?

@DMRobertson
Copy link
Contributor

DMRobertson commented Jun 16, 2022

Okay, now we can remove the .well-known/matrix/client entries from our nginx servers, but what about .well-known/matrix/server?

If you want Synapse to serve the server well-known response, see the serve_server_wellknown config option.

If you want to add additional fields to the well-known server response: that isn't mentioned in the spec (unlike the well-known client) so we'd like to understand the use case---please open a new issue if so.

@rwjack
Copy link

rwjack commented Jun 16, 2022

For client, I use the following data:

m.homeserver
m.identity_server
im.vector.riot.jitsi

As for server, I use the following data:

m.server

Perhaps I didn't understand correctly, but just making sure, can all of the above fields be covered with serve_server_wellknown?

@DMRobertson
Copy link
Contributor

can all of the above fields be covered with serve_server_wellknown?

No.

  • serve_server_wellknown covers m.server (see the docs) only.
  • If a public_baseurl is set in the config, then m.homeserver will be served.
  • If a public_baseurl and default_identity_server are both set in config , then m.identity_server will be served.
  • This PR adds a mechanism which allows you to serve im.vector.riot.jitsi in the client response.

@rwjack
Copy link

rwjack commented Jun 16, 2022

Awesome, so everything can be covered, not in a way I understood though, but you explained it perfectly. Thanks!

@Vetchu Vetchu deleted the feature/custom_well_known branch June 16, 2022 15:30
Fizzadar added a commit to beeper/synapse-legacy-fork that referenced this pull request Aug 23, 2022
Synapse 1.62.0 (2022-07-05)
===========================

No significant changes since 1.62.0rc3.

Authors of spam-checker plugins should consult the [upgrade notes](https://github.com/matrix-org/synapse/blob/release-v1.62/docs/upgrade.md#upgrading-to-v1620) to learn about the enriched signatures for spam checker callbacks, which are supported with this release of Synapse.

Synapse 1.62.0rc3 (2022-07-04)
==============================

Bugfixes
--------

- Update the version of the [ldap3 plugin](https://github.com/matrix-org/matrix-synapse-ldap3/) included in the `matrixdotorg/synapse` DockerHub images and the Debian packages hosted on `packages.matrix.org` to 0.2.1. This fixes [a bug](matrix-org/matrix-synapse-ldap3#163) with usernames containing uppercase characters. ([\matrix-org#13156](matrix-org#13156))
- Fix a bug introduced in Synapse 1.62.0rc1 affecting unread counts for users on small servers. ([\matrix-org#13168](matrix-org#13168))

Synapse 1.62.0rc2 (2022-07-01)
==============================

Bugfixes
--------

- Fix unread counts for users on large servers. Introduced in v1.62.0rc1. ([\matrix-org#13140](matrix-org#13140))
- Fix DB performance when deleting old push notifications. Introduced in v1.62.0rc1. ([\matrix-org#13141](matrix-org#13141))

Synapse 1.62.0rc1 (2022-06-28)
==============================

Features
--------

- Port the spam-checker API callbacks to a new, richer API. This is part of an ongoing change to let spam-checker modules inform users of the reason their event or operation is rejected. ([\matrix-org#12857](matrix-org#12857), [\matrix-org#13047](matrix-org#13047))
- Allow server admins to customise the response of the `/.well-known/matrix/client` endpoint. ([\matrix-org#13035](matrix-org#13035))
- Add metrics measuring the CPU and DB time spent in state resolution. ([\matrix-org#13036](matrix-org#13036))
- Speed up fetching of device list changes in `/sync` and `/keys/changes`. ([\matrix-org#13045](matrix-org#13045), [\matrix-org#13098](matrix-org#13098))
- Improve URL previews for sites which only provide Twitter Card metadata, e.g. LWN.net. ([\matrix-org#13056](matrix-org#13056))

Bugfixes
--------

- Update [MSC3786](matrix-org/matrix-spec-proposals#3786) implementation to check `state_key`. ([\matrix-org#12939](matrix-org#12939))
- Fix a bug introduced in Synapse 1.58 where Synapse would not report full version information when installed from a git checkout. This is a best-effort affair and not guaranteed to be stable. ([\matrix-org#12973](matrix-org#12973))
- Fix a bug introduced in Synapse 1.60 where Synapse would fail to start if the `sqlite3` module was not available. ([\matrix-org#12979](matrix-org#12979))
- Fix a bug where non-standard information was required when requesting the `/hierarchy` API over federation. Introduced
  in Synapse v1.41.0. ([\matrix-org#12991](matrix-org#12991))
- Fix a long-standing bug which meant that rate limiting was not restrictive enough in some cases. ([\matrix-org#13018](matrix-org#13018))
- Fix a bug introduced in Synapse 1.58 where profile requests for a malformed user ID would ccause an internal error. Synapse now returns 400 Bad Request in this situation. ([\matrix-org#13041](matrix-org#13041))
- Fix some inconsistencies in the event authentication code. ([\matrix-org#13087](matrix-org#13087), [\matrix-org#13088](matrix-org#13088))
- Fix a long-standing bug where room directory requests would cause an internal server error if given a malformed room alias. ([\matrix-org#13106](matrix-org#13106))

Improved Documentation
----------------------

- Add documentation for how to configure Synapse with Workers using Docker Compose. Includes example worker config and docker-compose.yaml. Contributed by @Thumbscrew. ([\matrix-org#12737](matrix-org#12737))
- Ensure the [Poetry cheat sheet](https://matrix-org.github.io/synapse/develop/development/dependencies.html) is available in the online documentation. ([\matrix-org#13022](matrix-org#13022))
- Mention removed community/group worker endpoints in upgrade.md. Contributed by @olmari. ([\matrix-org#13023](matrix-org#13023))
- Add instructions for running Complement with `gotestfmt`-formatted output locally. ([\matrix-org#13073](matrix-org#13073))
- Update OpenTracing docs to reference the configuration manual rather than the configuration file. ([\matrix-org#13076](matrix-org#13076))
- Update information on downstream Debian packages. ([\matrix-org#13095](matrix-org#13095))
- Remove documentation for the Delete Group Admin API which no longer exists. ([\matrix-org#13112](matrix-org#13112))

Deprecations and Removals
-------------------------

- Remove the unspecced `DELETE /directory/list/room/{roomId}` endpoint, which hid rooms from the [public room directory](https://spec.matrix.org/v1.3/client-server-api/#listing-rooms). Instead, `PUT` to the same URL with a visibility of `"private"`. ([\matrix-org#13123](matrix-org#13123))

Internal Changes
----------------

- Add tests for cancellation of `GET /rooms/$room_id/members` and `GET /rooms/$room_id/state` requests. ([\matrix-org#12674](matrix-org#12674))
- Report login failures due to unknown third party identifiers in the same way as failures due to invalid passwords. This prevents an attacker from using the error response to determine if the identifier exists. Contributed by Daniel Aloni. ([\matrix-org#12738](matrix-org#12738))
- Merge the Complement testing Docker images into a single, multi-purpose image. ([\matrix-org#12881](matrix-org#12881), [\matrix-org#13075](matrix-org#13075))
- Simplify the database schema for `event_edges`. ([\matrix-org#12893](matrix-org#12893))
- Clean up the test code for client disconnection. ([\matrix-org#12929](matrix-org#12929))
- Remove code generating comments in configuration. ([\matrix-org#12941](matrix-org#12941))
- Add `Cross-Origin-Resource-Policy: cross-origin` header to content repository's thumbnail and download endpoints. ([\matrix-org#12944](matrix-org#12944))
- Replace noop background updates with `DELETE` delta. ([\matrix-org#12954](matrix-org#12954), [\matrix-org#13050](matrix-org#13050))
- Use lower isolation level when inserting read receipts to avoid serialization errors. Contributed by Nick @ Beeper. ([\matrix-org#12957](matrix-org#12957))
- Reduce the amount of state we pull from the DB. ([\matrix-org#12963](matrix-org#12963))
- Enable testing against PostgreSQL databases in Complement CI. ([\matrix-org#12965](matrix-org#12965), [\matrix-org#13034](matrix-org#13034))
- Fix an inaccurate comment. ([\matrix-org#12969](matrix-org#12969))
- Remove the `delete_device` method and always call `delete_devices`. ([\matrix-org#12970](matrix-org#12970))
- Use a GitHub form for issues rather than a hard-to-read, easy-to-ignore template. ([\matrix-org#12982](matrix-org#12982))
- Move [MSC3715](matrix-org/matrix-spec-proposals#3715) behind an experimental config flag. ([\matrix-org#12984](matrix-org#12984))
- Add type hints to tests. ([\matrix-org#12985](matrix-org#12985), [\matrix-org#13099](matrix-org#13099))
- Refactor macaroon tokens generation and move the unsubscribe link in notification emails to `/_synapse/client/unsubscribe`. ([\matrix-org#12986](matrix-org#12986))
- Fix documentation for running complement tests. ([\matrix-org#12990](matrix-org#12990))
- Faster joins: add issue links to the TODO comments in the code. ([\matrix-org#13004](matrix-org#13004))
- Reduce DB usage of `/sync` when a large number of unread messages have recently been sent in a room. ([\matrix-org#13005](matrix-org#13005), [\matrix-org#13096](matrix-org#13096), [\matrix-org#13118](matrix-org#13118))
- Replaced usage of PyJWT with methods from Authlib in `org.matrix.login.jwt`. Contributed by Hannes Lerchl. ([\matrix-org#13011](matrix-org#13011))
- Modernize the `contrib/graph/` scripts. ([\matrix-org#13013](matrix-org#13013))
- Remove redundant `room_version` parameters from event auth functions. ([\matrix-org#13017](matrix-org#13017))
- Decouple `synapse.api.auth_blocking.AuthBlocking` from `synapse.api.auth.Auth`. ([\matrix-org#13021](matrix-org#13021))
- Add type annotations to `synapse.storage.databases.main.devices`. ([\matrix-org#13025](matrix-org#13025))
- Set default `sync_response_cache_duration` to two minutes. ([\matrix-org#13042](matrix-org#13042))
- Rename CI test runs. ([\matrix-org#13046](matrix-org#13046))
- Increase timeout of complement CI test runs. ([\matrix-org#13048](matrix-org#13048))
- Refactor entry points so that they all have a `main` function. ([\matrix-org#13052](matrix-org#13052))
- Refactor the Dockerfile-workers configuration script to use Jinja2 templates in Synapse workers' Supervisord blocks. ([\matrix-org#13054](matrix-org#13054))
- Add headers to individual options in config documentation to allow for linking. ([\matrix-org#13055](matrix-org#13055))
- Make Complement CI logs easier to read. ([\matrix-org#13057](matrix-org#13057), [\matrix-org#13058](matrix-org#13058), [\matrix-org#13069](matrix-org#13069))
- Don't instantiate modules with keyword arguments. ([\matrix-org#13060](matrix-org#13060))
- Fix type checking errors against Twisted trunk. ([\matrix-org#13061](matrix-org#13061))
- Allow MSC3030 `timestamp_to_event` calls from anyone on world-readable rooms. ([\matrix-org#13062](matrix-org#13062))
- Add a CI job to check that schema deltas are in the correct folder. ([\matrix-org#13063](matrix-org#13063))
- Avoid rechecking event auth rules which are independent of room state. ([\matrix-org#13065](matrix-org#13065))
- Reduce the duplication of code that invokes the rate limiter. ([\matrix-org#13070](matrix-org#13070))
- Add a Subject Alternative Name to the certificate generated for Complement tests. ([\matrix-org#13071](matrix-org#13071))
- Add more tests for room upgrades. ([\matrix-org#13074](matrix-org#13074))
- Pin dependencies maintained by matrix.org to [semantic version](https://semver.org/) bounds. ([\matrix-org#13082](matrix-org#13082))
- Correctly report prometheus DB stats for `get_earliest_token_for_stats`. ([\matrix-org#13085](matrix-org#13085))
- Fix a long-standing bug where a finished logging context would be re-started when Synapse failed to persist an event from federation. ([\matrix-org#13089](matrix-org#13089))
- Simplify the alias deletion logic as an application service. ([\matrix-org#13093](matrix-org#13093))
- Add type annotations to `tests.test_server`. ([\matrix-org#13124](matrix-org#13124))
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
X-Awaiting-Changes A contributed PR which needs changes and re-review before it can be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make it possible to customize /.well-known/matrix/client response
6 participants