Remove old `/_matrix/client/*/admin` endpoints #8785

dklimpel · 2020-11-19T21:42:03Z

Fixes: #8177

Remove historical_admin_path_patterns
Add also necessary tests for GET /_matrix/client/r0/admin/whois/{userId}

Pull Request Checklist

Pull request is based on the develop branch
Pull request includes a changelog file. The entry should:
- Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
- Use markdown where necessary, mostly for code blocks.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
Pull request includes a sign off
Code style is correct (run the linters)

Signed-off-by: Dirk Klimpel dirk@klimpel.org

clokep · 2020-11-20T12:36:54Z

Looks like this is failing sytests. I can't tell immediately if it is due to this PR, but it looks like it might be!

dklimpel · 2020-11-20T13:46:17Z

It is because of this PR. Sytest uses old APIs and needs an update.

dklimpel · 2020-11-20T17:17:31Z

@clokep Can you restart sytest test? Or I have to do merge develop to restart tests?

clokep · 2020-11-20T17:53:16Z

@clokep Can you restart sytest test? Or I have to do merge develop to restart tests?

I've kicked them off again.

clokep · 2020-11-23T13:04:11Z

I haven't had a chance to look at this yet, but I wonder if we should add a note to UPGRADE.rst about this?

dklimpel · 2020-11-23T13:10:27Z

ok, I thought that https://github.com/matrix-org/synapse/blob/develop/CHANGES.md#removal-warning was enough

clokep · 2020-11-24T14:06:57Z

ok, I thought that https://github.com/matrix-org/synapse/blob/develop/CHANGES.md#removal-warning was enough

It is likely worth re-iterating for people who might have missed it! Thanks for adding some info!

clokep

Looks great overall! I left a few minor comments, the most important being the endpoints registered for the whois API.

clokep · 2020-11-24T14:08:56Z

synapse/rest/admin/media.py

        +
        # This path kept around for legacy reasons
-        historical_admin_path_patterns("/quarantine_media/(?P<room_id>[^/]+)")
+        admin_patterns("/quarantine_media/(?P<room_id>[^/]+)")


Probably something else to deprecate at some point, but no real rush on that (and out of scope for this pull request).

Yes. It is a task for an other PR.

clokep · 2020-11-24T14:13:33Z

synapse/rest/admin/users.py

+        +
+        # URL for spec reason
+        # https://matrix.org/docs/spec/client_server/r0.6.1#get-matrix-client-r0-admin-whois-userid
+        [re.compile("/_matrix/client/r0/admin" + path_regex)]


I suspect that this should use client_patterns("/admin$", v1=True) to ensure this ends up at all the old paths:

/_matrix/client/api/v1/admin

/_matrix/client/unstable/admin

/_matrix/client/r0/admin

This can be imported: from synapse.rest.client.v2_alpha._base import client_patterns

tests/rest/admin/test_user.py

clokep · 2020-11-24T14:19:23Z

tests/rest/admin/test_user.py

@@ -1768,3 +1768,111 @@ def test_mau_limit(self):
        # though the MAU limit would stop the user doing so.
        puppet_token = self._get_token()
        self.helper.join(room_id, user=self.other_user, tok=puppet_token)
+
+
+class WhoisRestTestCase(unittest.HomeserverTestCase):


Thanks for adding tests here! I wonder if we need to each test to both URLs, but I don't see any harm in including them.

UPGRADE.rst

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

clokep

Looks great! Thanks so much!

@angdraug

Synapse 1.24.0 (2020-12-09) =========================== Due to the two security issues highlighted below, server administrators are encouraged to update Synapse. We are not aware of these vulnerabilities being exploited in the wild. Security advisory ----------------- The following issues are fixed in v1.23.1 and v1.24.0. - There is a denial of service attack ([CVE-2020-26257](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26257)) against the federation APIs in which future events will not be correctly sent to other servers over federation. This affects all servers that participate in open federation. (Fixed in [#8776](matrix-org/synapse#8776)). - Synapse may be affected by OpenSSL [CVE-2020-1971](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1971). Synapse administrators should ensure that they have the latest versions of the cryptography Python package installed. To upgrade Synapse along with the cryptography package: * Administrators using the [`matrix.org` Docker image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu packages from `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages) should ensure that they have version 1.24.0 or 1.23.1 installed: these images include the updated packages. * Administrators who have [installed Synapse from source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source) should upgrade the cryptography package within their virtualenv by running: ```sh <path_to_virtualenv>/bin/pip install 'cryptography>=3.3' ``` * Administrators who have installed Synapse from distribution packages should consult the information from their distributions. Internal Changes ---------------- - Add a maximum version for pysaml2 on Python 3.5. ([\#8898](matrix-org/synapse#8898)) Synapse 1.24.0rc2 (2020-12-04) ============================== Bugfixes -------- - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](matrix-org/synapse#8878)) Internal Changes ---------------- - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](matrix-org/synapse#8875)) Synapse 1.24.0rc1 (2020-12-02) ============================== Features -------- - Add admin API for logging in as a user. ([\#8617](matrix-org/synapse#8617)) - Allow specification of the SAML IdP if the metadata returns multiple IdPs. ([\#8630](matrix-org/synapse#8630)) - Add support for re-trying generation of a localpart for OpenID Connect mapping providers. ([\#8801](matrix-org/synapse#8801), [\#8855](matrix-org/synapse#8855)) - Allow the `Date` header through CORS. Contributed by Nicolas Chamo. ([\#8804](matrix-org/synapse#8804)) - Add a config option, `push.group_by_unread_count`, which controls whether unread message counts in push notifications are defined as "the number of rooms with unread messages" or "total unread messages". ([\#8820](matrix-org/synapse#8820)) - Add `force_purge` option to delete-room admin api. ([\#8843](matrix-org/synapse#8843)) Bugfixes -------- - Fix a bug where appservices may be sent an excessive amount of read receipts and presence. Broke in v1.22.0. ([\#8744](matrix-org/synapse#8744)) - Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body. ([\#8776](matrix-org/synapse#8776)) - Fix a bug where synctl could spawn duplicate copies of a worker. Contributed by Waylon Cude. ([\#8798](matrix-org/synapse#8798)) - Allow per-room profiles to be used for the server notice user. ([\#8799](matrix-org/synapse#8799)) - Fix a bug where logging could break after a call to SIGHUP. ([\#8817](matrix-org/synapse#8817)) - Fix `register_new_matrix_user` failing with "Bad Request" when trailing slash is included in server URL. Contributed by @angdraug. ([\#8823](matrix-org/synapse#8823)) - Fix a minor long-standing bug in login, where we would offer the `password` login type if a custom auth provider supported it, even if password login was disabled. ([\#8835](matrix-org/synapse#8835)) - Fix a long-standing bug which caused Synapse to require unspecified parameters during user-interactive authentication. ([\#8848](matrix-org/synapse#8848)) - Fix a bug introduced in v1.20.0 where the user-agent and IP address reported during user registration for CAS, OpenID Connect, and SAML were of the wrong form. ([\#8784](matrix-org/synapse#8784)) Improved Documentation ---------------------- - Clarify the usecase for a msisdn delegate. Contributed by Adrian Wannenmacher. ([\#8734](matrix-org/synapse#8734)) - Remove extraneous comma from JSON example in User Admin API docs. ([\#8771](matrix-org/synapse#8771)) - Update `turn-howto.md` with troubleshooting notes. ([\#8779](matrix-org/synapse#8779)) - Fix the example on how to set the `Content-Type` header in nginx for the Client Well-Known URI. ([\#8793](matrix-org/synapse#8793)) - Improve the documentation for the admin API to list all media in a room with respect to encrypted events. ([\#8795](matrix-org/synapse#8795)) - Update the formatting of the `push` section of the homeserver config file to better align with the [code style guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format). ([\#8818](matrix-org/synapse#8818)) - Improve documentation how to configure prometheus for workers. ([\#8822](matrix-org/synapse#8822)) - Update example prometheus console. ([\#8824](matrix-org/synapse#8824)) Deprecations and Removals ------------------------- - Remove old `/_matrix/client/*/admin` endpoints which were deprecated since Synapse 1.20.0. ([\#8785](matrix-org/synapse#8785)) - Disable pretty printing JSON responses for curl. Users who want pretty-printed output should use [jq](https://stedolan.github.io/jq/) in combination with curl. Contributed by @tulir. ([\#8833](matrix-org/synapse#8833)) Internal Changes ---------------- - Simplify the way the `HomeServer` object caches its internal attributes. ([\#8565](matrix-org/synapse#8565), [\#8851](matrix-org/synapse#8851)) - Add an example and documentation for clock skew to the SAML2 sample configuration to allow for clock/time difference between the homserver and IdP. Contributed by @localguru. ([\#8731](matrix-org/synapse#8731)) - Generalise `RoomMemberHandler._locally_reject_invite` to apply to more flows than just invite. ([\#8751](matrix-org/synapse#8751)) - Generalise `RoomStore.maybe_store_room_on_invite` to handle other, non-invite membership events. ([\#8754](matrix-org/synapse#8754)) - Refactor test utilities for injecting HTTP requests. ([\#8757](matrix-org/synapse#8757), [\#8758](matrix-org/synapse#8758), [\#8759](matrix-org/synapse#8759), [\#8760](matrix-org/synapse#8760), [\#8761](matrix-org/synapse#8761), [\#8777](matrix-org/synapse#8777)) - Consolidate logic between the OpenID Connect and SAML code. ([\#8765](matrix-org/synapse#8765)) - Use `TYPE_CHECKING` instead of magic `MYPY` variable. ([\#8770](matrix-org/synapse#8770)) - Add a commandline script to sign arbitrary json objects. ([\#8772](matrix-org/synapse#8772)) - Minor log line improvements for the SSO mapping code used to generate Matrix IDs from SSO IDs. ([\#8773](matrix-org/synapse#8773)) - Add additional error checking for OpenID Connect and SAML mapping providers. ([\#8774](matrix-org/synapse#8774), [\#8800](matrix-org/synapse#8800)) - Add type hints to HTTP abstractions. ([\#8806](matrix-org/synapse#8806), [\#8812](matrix-org/synapse#8812)) - Remove unnecessary function arguments and add typing to several membership replication classes. ([\#8809](matrix-org/synapse#8809)) - Optimise the lookup for an invite from another homeserver when trying to reject it. ([\#8815](matrix-org/synapse#8815)) - Add tests for `password_auth_provider`s. ([\#8819](matrix-org/synapse#8819)) - Drop redundant database index on `event_json`. ([\#8845](matrix-org/synapse#8845)) - Simplify `uk.half-shot.msc2778.login.application_service` login handler. ([\#8847](matrix-org/synapse#8847)) - Refactor `password_auth_provider` support code. ([\#8849](matrix-org/synapse#8849)) - Add missing `ordering` to background database updates. ([\#8850](matrix-org/synapse#8850)) - Allow for specifying a room version when creating a room in unit tests via `RestHelper.create_room_as`. ([\#8854](matrix-org/synapse#8854))

dklimpel added 2 commits November 19, 2020 22:36

Remove old /_matrix/client/*/admin endpoints

75e1be6

Add newsfile

24de1f3

clokep added the X-Awaiting-Changes A contributed PR which needs changes and re-review before it can be merged label Nov 20, 2020

dklimpel mentioned this pull request Nov 20, 2020

Update deprecated URLs /_matrix/client/*/admin matrix-org/sytest#984

Merged

improve doc

8a91858

clokep removed the X-Awaiting-Changes A contributed PR which needs changes and re-review before it can be merged label Nov 23, 2020

clokep requested a review from a team November 23, 2020 13:03

Update UPGRADE.rst

26f45f6

clokep suggested changes Nov 24, 2020

View reviewed changes

dklimpel and others added 2 commits November 24, 2020 16:01

Apply suggestions from code review - Part 1

c735e39

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Apply suggestions from code review - Part 2

e83c82f

dklimpel requested a review from clokep November 24, 2020 19:35

clokep approved these changes Nov 25, 2020

View reviewed changes

clokep merged commit 3f0ff53 into matrix-org:develop Nov 25, 2020

dklimpel deleted the remove_old_admin branch November 25, 2020 23:46

richvdh mentioned this pull request Nov 27, 2020

Add plugin support. matrix-org/sytest#941

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove old `/_matrix/client/*/admin` endpoints #8785

Remove old `/_matrix/client/*/admin` endpoints #8785

dklimpel commented Nov 19, 2020

clokep commented Nov 20, 2020

dklimpel commented Nov 20, 2020

dklimpel commented Nov 20, 2020

clokep commented Nov 20, 2020

clokep commented Nov 23, 2020

dklimpel commented Nov 23, 2020

clokep commented Nov 24, 2020

clokep left a comment

clokep Nov 24, 2020

dklimpel Nov 24, 2020

clokep Nov 24, 2020

clokep Nov 24, 2020

clokep left a comment

Remove old /_matrix/client/*/admin endpoints #8785

Remove old /_matrix/client/*/admin endpoints #8785

Conversation

dklimpel commented Nov 19, 2020

Pull Request Checklist

clokep commented Nov 20, 2020

dklimpel commented Nov 20, 2020

dklimpel commented Nov 20, 2020

clokep commented Nov 20, 2020

clokep commented Nov 23, 2020

dklimpel commented Nov 23, 2020

clokep commented Nov 24, 2020

clokep left a comment

Choose a reason for hiding this comment

clokep Nov 24, 2020

Choose a reason for hiding this comment

dklimpel Nov 24, 2020

Choose a reason for hiding this comment

clokep Nov 24, 2020

Choose a reason for hiding this comment

clokep Nov 24, 2020

Choose a reason for hiding this comment

clokep left a comment

Choose a reason for hiding this comment

Remove old `/_matrix/client/*/admin` endpoints #8785

Remove old `/_matrix/client/*/admin` endpoints #8785