Support OIDC backchannel logouts #11414

sandhose · 2021-11-23T17:04:31Z

Based on #11482.

Tested (and works) with Keycloak.

How to test this?

Spin up a Keycloak instance, create a realm
Configure Synapse to login via Keycloak: https://matrix-org.github.io/synapse/latest/openid.html#keycloak
In Keycloak, on the client used by Synapse, add the following backchannel logout URL: {public_url}/_synapse/client/oidc/backchannel_logout
Ensure Backchannel Logout Session Required is enabled on that client
Login via OIDC in any Matrix client
Log the user out of Keycloak by going to {keycloak}/auth/realms/{realms}/account/#/ (which is the self-service console) and click "Sign Out"
The user should be logged out of Synapse

There is a Docker Compose stack here to help with testing: https://github.com/sandhose/synapse-keycloak-devstack

Additionally, there are two edge cases which can be tested (and which are covered by tests):

when the logout happens before the client sends the login token (e.g. happens during the "Continuing will grant app.element.io access to your account." screen)
when the logout happens during the username mapping session, so when the localpart_template is not set and the user chooses a username themselves

Pull Request Checklist

Pull request is based on the develop branch
Pull request includes a changelog file. The entry should:
- Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
- Use markdown where necessary, mostly for code blocks.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
Pull request includes a sign off
Code style is correct
(run the linters)

clokep · 2021-12-02T18:10:28Z

Removing this from the review queue until #11482 gets worked out.

clokep

Looks pretty sensible, I left a bunch of comments but I think most are minor!

clokep · 2021-12-06T18:08:07Z

synapse/config/oidc.py

@@ -562,6 +569,9 @@ class OidcProviderConfig:
    # "openid" scope is used.
    jwks_uri: Optional[str]

+    # Whether Synapse should react to backchannel logouts
+    backchannel_logout_enabled: bool


It makes me slightly sad to add another configuration option here, but I don't think there's much we can do here to avoid it?

The metadata seems to include whether this feature is supported or not. Can we auto-enable it in those cases?

I mean, we could probably completely skip the configuration parameter. The only real benefit right now are warnings when the config enables backchannel logouts, but the OP does not advertise proper support.

Right now the behaviour is to delete the device, which does a "hard" logout. From a client perspective, it means it won't keep E2EE data, so you can't get back the deleted device. Maybe that's not the right behaviour, maybe that should be configurable.

Would it make sense to have instead a config parameter backchannel_logout_behaviour with the following values: logout (the default), soft_logout, nothing, and maybe implement soft_logout/other behaviours later?

Right now the behaviour is to delete the device, which does a "hard" logout. From a client perspective, it means it won't keep E2EE data, so you can't get back the deleted device. Maybe that's not the right behaviour, maybe that should be configurable.

Would it make sense to have instead a config parameter backchannel_logout_behaviour with the following values: logout (the default), soft_logout, nothing, and maybe implement soft_logout/other behaviours later?

I don't think doing a hard logout makes sense, actually. It likely makes more sense to do a soft-logout? I'm not really sure what a backchannel-logout is supposed to represent, but logging out of the IdP causing Synapse to destroy a session, causing a user to lose decryption keys sounds not great.

I haven't really found a good way to soft-logout a user :(
It looks like soft-logouts are only sent when the token expired.

but logging out of the IdP causing Synapse to destroy a session, causing a user to lose decryption keys sounds not great.

I agree, but on the other hand leaving 'dead' devices on the account doesn't seem great either. Would you be fine with leaving this behaviour for now, see how it feels with whatever customer is asking for this change, and iterate on it later if it makes more sense to do a soft-logout?

clokep · 2021-12-06T18:32:50Z

synapse/handlers/oidc.py

+        This extracts the logout_token from the request and try to figure out
+        from whom it is coming from. This works by matching the iss claim with
+        the issuer and the aud claim with the client_id.


I think this is a little more readable?

Suggested change

This extracts the logout_token from the request and try to figure out

from whom it is coming from. This works by matching the iss claim with

the issuer and the aud claim with the client_id.

This extracts the logout_token from the request and tries to figure out

which account it is associated with. This works by matching the iss claim

with the issuer and the aud claim with the client_id.

Ha, I did not properly read your suggestion. It's not "which account it is associated with" but rather "from which IdP it is coming from"

Isn't this matching to a particular user though, not just the IdP?

This method specifically matches the IDP, and then passes down to the OidcProvider#handle_backchannel_logout, which then finds the user

synapse/handlers/oidc.py

sandhose

Thanks for the review @clokep! I replied to most comments and will fix some things now

I'd like your thoughts on the caveat I highlighted in the PR message:

One caveat (not sure how to easily fix this): in the following scenario, the user is still logged in even if the OIDC session ended:

start the OIDC login from an "untrusted" client

when on the "consent" screen served by Synapse (Continuing will grant http://XXXX/ access to your account.), do not click continue yet

instead, log out from Keycloak

click on "continue" on the consent screen: the user is successfully logged in, even if the session ended in Keycloak

sandhose · 2021-12-07T08:29:08Z

synapse/config/oidc.py

@@ -562,6 +569,9 @@ class OidcProviderConfig:
    # "openid" scope is used.
    jwks_uri: Optional[str]

+    # Whether Synapse should react to backchannel logouts
+    backchannel_logout_enabled: bool


I mean, we could probably completely skip the configuration parameter. The only real benefit right now are warnings when the config enables backchannel logouts, but the OP does not advertise proper support.

Right now the behaviour is to delete the device, which does a "hard" logout. From a client perspective, it means it won't keep E2EE data, so you can't get back the deleted device. Maybe that's not the right behaviour, maybe that should be configurable.

Would it make sense to have instead a config parameter backchannel_logout_behaviour with the following values: logout (the default), soft_logout, nothing, and maybe implement soft_logout/other behaviours later?

synapse/handlers/oidc.py

synapse/python_dependencies.py

clokep · 2021-12-09T14:43:31Z

One caveat (not sure how to easily fix this): in the following scenario, the user is still logged in even if the OIDC session ended:

start the OIDC login from an "untrusted" client

when on the "consent" screen served by Synapse (Continuing will grant http://XXXX/ access to your account.), do not click continue yet

instead, log out from Keycloak

click on "continue" on the consent screen: the user is successfully logged in, even if the session ended in Keycloak

This seems like it is probably OK? Shouldn't keycloak be rejecting this if the session ended?

msitruk · 2022-06-23T10:47:52Z

Hello any news ?
Is there any other way to resolv this lack of working logout mecanism for oidc ... waiting this PR ?

reivilibre · 2022-08-05T09:28:33Z

What's the thinking on the state of this PR? Is it still wanted; does it need more review; is there something to be done to it?

sandhose

I'm getting back to the PR. I rebased on top of develop and applied a few fixes.

Still on my todo list is:

add tests
add documentation
figure out what to do with the potential login/logout "race condition"
merge some of the JWT handling code of the ID token and the logout token

synapse/handlers/oidc.py

sandhose · 2022-09-16T16:23:19Z

synapse/config/oidc.py

@@ -562,6 +569,9 @@ class OidcProviderConfig:
    # "openid" scope is used.
    jwks_uri: Optional[str]

+    # Whether Synapse should react to backchannel logouts
+    backchannel_logout_enabled: bool


I haven't really found a good way to soft-logout a user :(
It looks like soft-logouts are only sent when the token expired.

but logging out of the IdP causing Synapse to destroy a session, causing a user to lose decryption keys sounds not great.

I agree, but on the other hand leaving 'dead' devices on the account doesn't seem great either. Would you be fine with leaving this behaviour for now, see how it feels with whatever customer is asking for this change, and iterate on it later if it makes more sense to do a soft-logout?

sandhose · 2022-09-16T16:26:18Z

synapse/handlers/oidc.py

+        This extracts the logout_token from the request and try to figure out
+        from whom it is coming from. This works by matching the iss claim with
+        the issuer and the aud claim with the client_id.


This method specifically matches the IDP, and then passes down to the OidcProvider#handle_backchannel_logout, which then finds the user

Signed-off-by: Quentin Gliech <quenting@element.io>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Signed-off-by: Quentin Gliech <quenting@element.io>

Mainly adds a warning if the user_mapping_provider is not using the `sub` claim as subject Signed-off-by: Quentin Gliech <quenting@element.io>

sandhose · 2022-10-26T12:11:50Z

It should be ready to review! Now that #13844 is merged, I was able to revoke login tokens in-flight (there is a test for that), and now that #13910 is merged, I was able to write proper unit tests for everything.

It just lacks documentation, which I can add later today, or in a future PR. At least it is ready to review code-wise

There was an issue before Twisted 20.3.0, where request.args would not be filled if the `Content-Length` header was missing. I fixed some unit tests by adding this header in `make_request`.

sandhose · 2022-10-26T15:32:55Z

I spent way too much time figuring out why the tests were failing with the olddeps. Turns out, there was a bug with Twisted<20.3.0 where request.args was not filled on url encoded form bodies if the Content-Length was missing.

It might be this patch, but I'm not 100% sure: twisted/twisted#9678

I don't think it's worth fixing on the handler side, since most HTTP clients fill this header, unless you're streaming the body, which will probably not be the case for OIDC Backchannel logout notifications.

I fixed the tests by filling the Content-Length header in tests.server.make_request, see 55076e1

synapse/rest/synapse/client/oidc/backchannel_logout_resource.py

synapse/config/oidc.py

docs/usage/configuration/config_documentation.md

synapse/handlers/oidc.py

clokep · 2022-10-28T17:14:24Z

synapse/handlers/oidc.py

@@ -1112,6 +1456,7 @@ def _get_secret(self) -> bytes:
        logger.info(
            "Generating new JWT for %s: %s %s", self._oauth_issuer, header, payload
        )
+        jwt = JsonWebToken(header["alg"])


What's this change for? Is it ensuring JsonWebToken doesn't explode?

we were importing authlib.jose.jwt before and it's not the case anymore. I think I did this because it was not available in all versions of authlib?

🤦 I failed to see where jwt was being used...

It looks like jwt has existed to be imported since at least 0.15?

I don't have a strong opinion about this though.

synapse/handlers/oidc.py

docs/usage/configuration/config_documentation.md

tests/rest/client/test_auth.py

clokep · 2022-10-28T17:36:11Z

@sandhose Overall this looks reasonable. I'm sure some of the questions I asked were answered previously during our last round of reviews, but I'm afraid those have been lost to time. 😢 Sorry about that!

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

…t tokens

clokep

Looks good if tests pass! 🎉

@asymmetric

Synapse 1.71.0 (2022-11-08) =========================== Please note that, as announced in the release notes for Synapse 1.69.0, legacy Prometheus metric names are now disabled by default. They will be removed altogether in Synapse 1.73.0. If not already done, server administrators should update their dashboards and alerting rules to avoid using the deprecated metric names. See the [upgrade notes](https://matrix-org.github.io/synapse/v1.71/upgrade.html#upgrading-to-v1710) for more details. **Note:** in line with our [deprecation policy](https://matrix-org.github.io/synapse/latest/deprecation_policy.html) for platform dependencies, this will be the last release to support PostgreSQL 10, which reaches upstream end-of-life on November 10th, 2022. Future releases of Synapse will require PostgreSQL 11+. Features -------- - Support back-channel logouts from OpenID Connect providers. ([\#11414](matrix-org/synapse#11414)) - Allow use of Postgres and SQLlite full-text search operators in search queries. ([\#11635](matrix-org/synapse#11635), [\#14310](matrix-org/synapse#14310), [\#14311](matrix-org/synapse#14311)) - Implement [MSC3664](matrix-org/matrix-spec-proposals#3664), Pushrules for relations. Contributed by Nico. ([\#11804](matrix-org/synapse#11804)) - Improve aesthetics of HTML templates. Note that these changes do not retroactively apply to templates which have been [customised](https://matrix-org.github.io/synapse/latest/templates.html#templates) by server admins. ([\#13652](matrix-org/synapse#13652)) - Enable write-ahead logging for SQLite installations. Contributed by [@asymmetric](https://github.com/asymmetric). ([\#13897](matrix-org/synapse#13897)) - Show erasure status when [listing users](https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#query-user-account) in the Admin API. ([\#14205](matrix-org/synapse#14205)) - Provide a specific error code when a `/sync` request provides a filter which doesn't represent a JSON object. ([\#14262](matrix-org/synapse#14262))

@asymmetric

Synapse 1.71.0 (2022-11-08) =========================== Please note that, as announced in the release notes for Synapse 1.69.0, legacy Prometheus metric names are now disabled by default. They will be removed altogether in Synapse 1.73.0. If not already done, server administrators should update their dashboards and alerting rules to avoid using the deprecated metric names. See the [upgrade notes](https://matrix-org.github.io/synapse/v1.71/upgrade.html#upgrading-to-v1710) for more details. **Note:** in line with our [deprecation policy](https://matrix-org.github.io/synapse/latest/deprecation_policy.html) for platform dependencies, this will be the last release to support PostgreSQL 10, which reaches upstream end-of-life on November 10th, 2022. Future releases of Synapse will require PostgreSQL 11+. No significant changes since 1.71.0rc2. Synapse 1.71.0rc2 (2022-11-04) ============================== Improved Documentation ---------------------- - Document the changes to monthly active user metrics due to deprecation of legacy Prometheus metric names. ([\matrix-org#14358](matrix-org#14358), [\matrix-org#14360](matrix-org#14360)) Deprecations and Removals ------------------------- - Disable legacy Prometheus metric names by default. They can still be re-enabled for now, but they will be removed altogether in Synapse 1.73.0. ([\matrix-org#14353](matrix-org#14353)) Internal Changes ---------------- - Run unit tests against Python 3.11. ([\matrix-org#13812](matrix-org#13812)) Synapse 1.71.0rc1 (2022-11-01) ============================== Features -------- - Support back-channel logouts from OpenID Connect providers. ([\matrix-org#11414](matrix-org#11414)) - Allow use of Postgres and SQLlite full-text search operators in search queries. ([\matrix-org#11635](matrix-org#11635), [\matrix-org#14310](matrix-org#14310), [\matrix-org#14311](matrix-org#14311)) - Implement [MSC3664](matrix-org/matrix-spec-proposals#3664), Pushrules for relations. Contributed by Nico. ([\matrix-org#11804](matrix-org#11804)) - Improve aesthetics of HTML templates. Note that these changes do not retroactively apply to templates which have been [customised](https://matrix-org.github.io/synapse/latest/templates.html#templates) by server admins. ([\matrix-org#13652](matrix-org#13652)) - Enable write-ahead logging for SQLite installations. Contributed by [@asymmetric](https://github.com/asymmetric). ([\matrix-org#13897](matrix-org#13897)) - Show erasure status when [listing users](https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#query-user-account) in the Admin API. ([\matrix-org#14205](matrix-org#14205)) - Provide a specific error code when a `/sync` request provides a filter which doesn't represent a JSON object. ([\matrix-org#14262](matrix-org#14262)) Bugfixes -------- - Fix a long-standing bug where the `update_synapse_database` script could not be run with multiple databases. Contributed by @thefinn93 @ Beeper. ([\matrix-org#13422](matrix-org#13422)) - Fix a bug which prevented setting an avatar on homeservers which have an explicit port in their `server_name` and have `max_avatar_size` and/or `allowed_avatar_mimetypes` configuration. Contributed by @ashfame. ([\matrix-org#13927](matrix-org#13927)) - Check appservice user interest against the local users instead of all users in the room to align with [MSC3905](matrix-org/matrix-spec-proposals#3905). ([\matrix-org#13958](matrix-org#13958)) - Fix a long-standing bug where Synapse would accidentally include extra information in the response to [`PUT /_matrix/federation/v2/invite/{roomId}/{eventId}`](https://spec.matrix.org/v1.4/server-server-api/#put_matrixfederationv2inviteroomideventid). ([\matrix-org#14064](matrix-org#14064)) - Fix a bug introduced in Synapse 1.64.0 where presence updates could be missing from `/sync` responses. ([\matrix-org#14243](matrix-org#14243)) - Fix a bug introduced in Synapse 1.60.0 which caused an error to be logged when Synapse received a SIGHUP signal if debug logging was enabled. ([\matrix-org#14258](matrix-org#14258)) - Prevent history insertion ([MSC2716](matrix-org/matrix-spec-proposals#2716)) during an partial join ([MSC3706](matrix-org/matrix-spec-proposals#3706)). ([\matrix-org#14291](matrix-org#14291)) - Fix a bug introduced in Synapse 1.34.0 where device names would be returned via a federation user key query request when `allow_device_name_lookup_over_federation` was set to `false`. ([\matrix-org#14304](matrix-org#14304)) - Fix a bug introduced in Synapse 0.34.0 where logs could include error spam when background processes are measured as taking a negative amount of time. ([\matrix-org#14323](matrix-org#14323)) - Fix a bug introduced in Synapse 1.70.0 where clients were unable to PUT new [dehydrated devices](matrix-org/matrix-spec-proposals#2697). ([\matrix-org#14336](matrix-org#14336)) Improved Documentation ---------------------- - Explain how to disable the use of [`trusted_key_servers`](https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#trusted_key_servers). ([\matrix-org#13999](matrix-org#13999)) - Add workers settings to [configuration manual](https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#individual-worker-configuration). ([\matrix-org#14086](matrix-org#14086)) - Correct the name of the config option [`encryption_enabled_by_default_for_room_type`](https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#encryption_enabled_by_default_for_room_type). ([\matrix-org#14110](matrix-org#14110)) - Update docstrings of `SynapseError` and `FederationError` to bettter describe what they are used for and the effects of using them are. ([\matrix-org#14191](matrix-org#14191)) Internal Changes ---------------- - Remove unused `@lru_cache` decorator. ([\matrix-org#13595](matrix-org#13595)) - Save login tokens in database and prevent login token reuse. ([\matrix-org#13844](matrix-org#13844)) - Refactor OIDC tests to better mimic an actual OIDC provider. ([\matrix-org#13910](matrix-org#13910)) - Fix type annotation causing import time error in the Complement forking launcher. ([\matrix-org#14084](matrix-org#14084)) - Refactor [MSC3030](matrix-org/matrix-spec-proposals#3030) `/timestamp_to_event` endpoint to loop over federation destinations with standard pattern and error handling. ([\matrix-org#14096](matrix-org#14096)) - Add initial power level event to batch of bulk persisted events when creating a new room. ([\matrix-org#14228](matrix-org#14228)) - Refactor `/key/` endpoints to use `RestServlet` classes. ([\matrix-org#14229](matrix-org#14229)) - Switch to using the `matrix-org/backend-meta` version of `triage-incoming` for new issues in CI. ([\matrix-org#14230](matrix-org#14230)) - Build wheels on macos 11, not 10.15. ([\matrix-org#14249](matrix-org#14249)) - Add debugging to help diagnose lost device list updates. ([\matrix-org#14268](matrix-org#14268)) - Add Rust cache to CI for `trial` runs. ([\matrix-org#14287](matrix-org#14287)) - Improve type hinting of `RawHeaders`. ([\matrix-org#14303](matrix-org#14303)) - Use Poetry 1.2.0 in the Twisted Trunk CI job. ([\matrix-org#14305](matrix-org#14305)) <details> <summary>Dependency updates</summary> Runtime: - Bump anyhow from 1.0.65 to 1.0.66. ([\matrix-org#14278](matrix-org#14278)) - Bump jinja2 from 3.0.3 to 3.1.2. ([\matrix-org#14271](matrix-org#14271)) - Bump prometheus-client from 0.14.0 to 0.15.0. ([\matrix-org#14274](matrix-org#14274)) - Bump psycopg2 from 2.9.4 to 2.9.5. ([\matrix-org#14331](matrix-org#14331)) - Bump pysaml2 from 7.1.2 to 7.2.1. ([\matrix-org#14270](matrix-org#14270)) - Bump sentry-sdk from 1.5.11 to 1.10.1. ([\matrix-org#14330](matrix-org#14330)) - Bump serde from 1.0.145 to 1.0.147. ([\matrix-org#14277](matrix-org#14277)) - Bump serde_json from 1.0.86 to 1.0.87. ([\matrix-org#14279](matrix-org#14279)) Tooling and CI: - Bump black from 22.3.0 to 22.10.0. ([\matrix-org#14328](matrix-org#14328)) - Bump flake8-bugbear from 21.3.2 to 22.9.23. ([\matrix-org#14042](matrix-org#14042)) - Bump peaceiris/actions-gh-pages from 3.8.0 to 3.9.0. ([\matrix-org#14276](matrix-org#14276)) - Bump peaceiris/actions-mdbook from 1.1.14 to 1.2.0. ([\matrix-org#14275](matrix-org#14275)) - Bump setuptools-rust from 1.5.1 to 1.5.2. ([\matrix-org#14273](matrix-org#14273)) - Bump twine from 3.8.0 to 4.0.1. ([\matrix-org#14332](matrix-org#14332)) - Bump types-opentracing from 2.4.7 to 2.4.10. ([\matrix-org#14133](matrix-org#14133)) - Bump types-requests from 2.28.11 to 2.28.11.2. ([\matrix-org#14272](matrix-org#14272)) </details>

sandhose marked this pull request as ready for review December 1, 2021 17:16

sandhose requested a review from a team as a code owner December 1, 2021 17:16

clokep self-assigned this Dec 1, 2021

clokep removed the request for review from a team December 2, 2021 18:10

clokep self-requested a review December 6, 2021 17:54

clokep reviewed Dec 6, 2021

View reviewed changes

sandhose commented Dec 7, 2021

View reviewed changes

clokep reviewed Dec 7, 2021

View reviewed changes

synapse/python_dependencies.py Outdated Show resolved Hide resolved

sandhose commented Sep 16, 2022

View reviewed changes

sandhose mentioned this pull request Sep 19, 2022

Save the login tokens in database instead of being macaroons #13841

Closed

sandhose mentioned this pull request Sep 26, 2022

Refactor OIDC tests to better mimic an actual OIDC provider #13910

Merged

sandhose and others added 7 commits October 26, 2022 13:53

Support back-channel logouts from OIDC providers

49fe26e

Signed-off-by: Quentin Gliech <quenting@element.io>

Apply suggestions from code review

a0bb53d

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Validate exp and nbf claims in logout tokens if present

a5d74fa

Signed-off-by: Quentin Gliech <quenting@element.io>

Docstrings on issuer/client_id properties

51f3cd8

Signed-off-by: Quentin Gliech <quenting@element.io>

Misc changes

1fd3658

Mainly adds a warning if the user_mapping_provider is not using the `sub` claim as subject Signed-off-by: Quentin Gliech <quenting@element.io>

Check the sub claim when receiving a logout token

17580ae

Revoke in-flight logins

42df126

Fix unit tests on old version of Twisted

55076e1

There was an issue before Twisted 20.3.0, where request.args would not be filled if the `Content-Length` header was missing. I fixed some unit tests by adding this header in `make_request`.

sandhose requested a review from clokep October 26, 2022 15:39

Docs

eb7537a