Authenticating with QR code always times out when using multiple workers

[Marx1st]

Trying to authenticate Element X (iOS, Version 1.9.3) via QR code with our Matrix installation. Element Web displays the code, Element X scans it successfully and tries to set up the connection. After some time it will fail and show an error (unexpected error).

To Reproduce
My vars.yml file looks like this:

matrix_domain: uni-wuppertal.de

matrix_server_fqn_matrix: matrixsrv.uni-wuppertal.de

matrix_client_element_enabled: false

matrix_homeserver_implementation: synapse

matrix_synapse_workers_enabled: true

matrix_synapse_workers_preset: buw

matrix_synapse_user_directory_search_all_users: true

matrix_synapse_user_directory_prefer_local_users: true

matrix_synapse_url_preview_enabled: true

matrix_url_preview_accept_language: ['en-US;q=0.8', 'en;q=0.8', 'de-DE;q=0.9', 'de;q=0.9', '*;q=0.5']

matrix_homeserver_generic_secret_key: 'REDACTED'

matrix_playbook_reverse_proxy_type: playbook-managed-traefik

matrix_static_files_container_labels_well_known_matrix_endpoint_traefik_rule: "(Host(`{{ matrix_static_files_container_labels_well_known_matrix_endpoint_traefik_hostname }}`) || Host(`www.uni-wuppertal.de`)) && PathPrefix(`{{ matrix_static_files_container_labels_well_known_matrix_endpoint_traefik_path_prefix }}`)"

traefik_config_certificatesResolvers_acme_email: 'REDACTED'

traefik_dashboard_enabled: true
traefik_dashboard_hostname: "{{ matrix_server_fqn_matrix }}"
traefik_dashboard_basicauth_enabled: true
traefik_dashboard_basicauth_user: traefik-admin
traefik_dashboard_basicauth_password: 'REDACTED'

postgres_connection_password: 'REDACTED'

postgres_backup_enabled: true
postgres_backup_data_path: "/matrix/synapse/storage/media-store/db-backup/"

prometheus_enabled: true
grafana_enabled: true
grafana_anonymous_access: false
grafana_default_admin_user: "statsadmin"
grafana_default_admin_password: "REDACTED"

matrix_synapse_admin_enabled: true

matrix_authentication_service_enabled: true
matrix_authentication_service_config_secrets_encryption: 'REDACTED'
matrix_authentication_service_syn2mas_process_extra_arguments:
  - "--upstreamProviderMapping oidc-keycloak:01JBGVEAQG1AZCM4SE0NZ36Q0B"

matrix_authentication_service_config_upstream_oauth2_providers:
  - id: 01JBGVEAQG1AZCM4SE0NZ36Q0B
    issuer: https://sphub-rp.uni-wuppertal.de/realms/zim-public
    human_name: ZIM Login
    client_id: synapse
    client_secret: 'REDACTED'
    token_endpoint_auth_method: client_secret_basic
    scope: "openid email profile"
    claims_imports:
      localpart:
        action: require
        template: "{% raw %}{{ user.preferred_username }}{% endraw %}"
      displayname:
        action: suggest
        template: "{% raw %}{{ user.name }}{% endraw %}"
      email:
        action: suggest
        template: "{% raw %}{{ user.email }}{% endraw %}"
        set_email_verification: always

Our worker setup:

    room_workers_count: 2
    sync_workers_count: 2
    client_reader_workers_count: 2
    federation_reader_workers_count: 2
    generic_workers_count: 0
    pusher_workers_count: 2
    federation_sender_workers_count: 2
    media_repository_workers_count: 2
    appservice_workers_count: 2
    user_dir_workers_count: 2
    background_workers_count: 2
    stream_writer_events_stream_workers_count: 2
    stream_writer_typing_stream_workers_count: 1
    stream_writer_to_device_stream_workers_count: 1
    stream_writer_account_data_stream_workers_count: 1
    stream_writer_receipts_stream_workers_count: 1
    stream_writer_presence_stream_workers_count: 1

Expected behavior
Scan the QR code and getting logged in automatically to our Matrix instance.

Matrix Server:

• OS: Ubuntu 22.04
• Architecture amd64

Client:

• Device: iPhone 14
• OS: iOS 18

[Marx1st]

I've tried to find out which worker is the trouble maker when it has more then one instance, but even with this worker setup, authenticating via QR code just times out:

    room_workers_count: 1
    sync_workers_count: 1
    client_reader_workers_count: 1
    federation_reader_workers_count: 1
    generic_workers_count: 0
    pusher_workers_count: 1
    federation_sender_workers_count: 1
    media_repository_workers_count: 1
    appservice_workers_count: 1
    user_dir_workers_count: 1
    background_workers_count: 1
    stream_writer_events_stream_workers_count: 1
    stream_writer_typing_stream_workers_count: 1
    stream_writer_to_device_stream_workers_count: 1
    stream_writer_account_data_stream_workers_count: 1
    stream_writer_receipts_stream_workers_count: 1
    stream_writer_presence_stream_workers_count: 1

Only after setting:

matrix_synapse_workers_enabled: false
and thus disabling workers completely, are QR codes working.

[spantaleev]

It seems like we need to explicitly enable MSC4108 (QR code login) support for Synapse for QR code login to work.

In 509542c, I've added code which does it automatically when Matrix Authentication Service is enabled. Please give it a try and let me know if it solves the issue!

[Marx1st]

Unfortunately it still doesn't work.

Does it do more then adding

  experimental_features:
    msc4108_enabled: true

to the homeserver.yaml?

Because we already have:

matrix_synapse_configuration_extension_yaml: |
  experimental_features:
    msc4108_enabled: true

[spantaleev]

That's all it does, indeed.

I suppose there's something more to getting QR codes working then.

[Marx1st]

We had that option, already, because without it Element wouldn't show a QR code, at all ;)