Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add workers settings to configuration manual #14086

Merged
merged 29 commits into from
Oct 27, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
3c0abc6
Add workers settings to configuration manual
dklimpel Oct 6, 2022
167b28e
newsfile
dklimpel Oct 6, 2022
41c7992
Merge branch 'develop' into worker_doc
dklimpel Oct 7, 2022
895fa0d
Update `pusher_instances`
dklimpel Oct 8, 2022
2e6fcbf
Merge remote-tracking branch 'synapse/develop' into worker_doc
dklimpel Oct 13, 2022
2c0652d
Move from `config_documentation.md` to `workers.md`
dklimpel Oct 13, 2022
b6d9285
Merge remote-tracking branch 'synapse/develop' into worker_doc
dklimpel Oct 23, 2022
3b144ef
Move docs back to `config_documentation.md`
dklimpel Oct 24, 2022
1ac4cf6
newsfile and small update
dklimpel Oct 24, 2022
628a448
Apply suggestions from code review
dklimpel Oct 26, 2022
327c81c
update url to python logger
dklimpel Oct 26, 2022
d12e006
update headlines
dklimpel Oct 26, 2022
c106747
update links after headline change
dklimpel Oct 26, 2022
955dab6
remove link from `daemon process`
dklimpel Oct 26, 2022
18d692a
Apply suggestions from code review
dklimpel Oct 26, 2022
1d677f3
Apply suggestions from code review
dklimpel Oct 26, 2022
2c91c5a
update `event persister` doc
dklimpel Oct 26, 2022
b0a20d1
extend example for `federation_sender_instances` and `pusher_instances`
dklimpel Oct 26, 2022
17ec5d3
more infos about stream writers
dklimpel Oct 26, 2022
e0fc9e3
Merge branch 'develop' into worker_doc
dklimpel Oct 26, 2022
76f8b5f
add link to DAG
dklimpel Oct 27, 2022
a3ad96a
small rewording
dklimpel Oct 27, 2022
6c9cea2
Apply suggestions from code review
dklimpel Oct 27, 2022
0f9b29f
Apply suggestions from code review
dklimpel Oct 27, 2022
5045ca0
update `pusher_instances`
dklimpel Oct 27, 2022
a943440
update `worker_listeners`
dklimpel Oct 27, 2022
a214ee8
update `stream_writers`
dklimpel Oct 27, 2022
c0434bd
Update `worker_name`
dklimpel Oct 27, 2022
8394c3e
Fix dmr typo in `stream_writers`!!
Oct 27, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/14086.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add workers settings to [configuration manual](https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#workers).
143 changes: 130 additions & 13 deletions docs/usage/configuration/config_documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -1338,7 +1338,8 @@ Config options related to logging.
---
### `log_config`

This option specifies a yaml python logging config file as described [here](https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema).
This option specifies a yaml python logging config file as described
[here](https://docs.python.org/3.11/library/logging.config.html#configuration-dictionary-schema).

Example configuration:
```yaml
Expand Down Expand Up @@ -3673,11 +3674,137 @@ opentracing:
## Workers ##
Configuration options related to workers.
dklimpel marked this conversation as resolved.
Show resolved Hide resolved

Please also note the [further documentation for workers](../../workers.md).

---
### `worker_app`

The type of worker. The currently available worker applications are listed
in [worker documentation](../../workers.md#available-worker-applications).

The most common worker is the `synapse.app.generic_worker`.

Example configuration:
```yaml
worker_app: synapse.app.generic_worker
```
---
### `worker_name`

A unique name for the worker. The worker needs a name to be addressed in
further parameters and identification in log files.

Example configuration:
```yaml
worker_name: generic_worker1
```
---
### `worker_replication_host`

The HTTP replication endpoint that it should talk to on the main Synapse process.
The main Synapse process defines this with a `replication` resource in
[`listeners` option](#listeners).

Example configuration:
```yaml
worker_replication_host: 127.0.0.1
```
---
### `worker_replication_http_port`

The HTTP replication port that it should talk to on the main Synapse process.
The main Synapse process defines this with a `replication` resource in
[`listeners` option](#listeners).

Example configuration:
```yaml
worker_replication_http_port: 9093
```
---
### `worker_replication_secret`

A shared secret used by the replication APIs to authenticate HTTP requests
from workers.

By default this is unused and traffic is not authenticated.
dklimpel marked this conversation as resolved.
Show resolved Hide resolved

Example configuration:
```yaml
worker_replication_secret: "secret_secret"
```
---
### `worker_listeners`

A worker can handle HTTP requests. If handling HTTP requests, a `worker_listeners`
option with an http listener, in the same way as the [`listeners` option](#listeners)
in the shared config.

Example configuration:
```yaml
worker_listeners:
- type: http
port: 8083
resources:
- names: [client, federation]
```
---
### `worker_daemonize`

Specifies whether the worker should be daemonize. If [systemd](../../systemd-with-workers/README.md)
is used, this must not configured. Systemd manages daemonization itself. Defaults to `false`.

Example configuration:
```yaml
worker_daemonize: true
```
---
### `worker_pid_file`

When running Synapse worker as a daemon, the file to store the pid in. Defaults to none.
This is the same way as the [`pid_file` option](#pid_file) in the shared config.

Example configuration:
```yaml
worker_pid_file: DATADIR/generic_worker1.pid
```
---
### `worker_log_config`

This option specifies a yaml python logging config file as described
[here](https://docs.python.org/3.11/library/logging.config.html#configuration-dictionary-schema).
This is the same way as the [`log_config` option](#log_config) in the shared config.

Example configuration:
```yaml
worker_log_config: /etc/matrix-synapse/generic-worker-log.yaml
```
---
### `start_pushers`

Controls sending of push notifications on the main process. Set to `false`
if using a [pusher worker](../../workers.md#synapseapppusher). Defaults to `true`.

Example configuration:
```yaml
start_pushers: false
```
---
### `pusher_instances`

It is possible to run multiple pusher workers, in which case the
work is balanced across them. Use this setting to list the pushers by [`worker_name`](#worker_name).

Example configuration:
```yaml
pusher_instances:
- pusher_worker1
```
---
### `send_federation`

Controls sending of outbound federation transactions on the main process.
Set to false if using a federation sender worker. Defaults to true.
Set to false if using a [federation sender worker](../../workers.md#synapseappfederation_sender).
DMRobertson marked this conversation as resolved.
Show resolved Hide resolved
Defaults to `true`.

Example configuration:
```yaml
Expand Down Expand Up @@ -3736,17 +3863,6 @@ Example configuration:
run_background_tasks_on: worker1
```
---
### `worker_replication_secret`
dklimpel marked this conversation as resolved.
Show resolved Hide resolved

A shared secret used by the replication APIs to authenticate HTTP requests
from workers.

By default this is unused and traffic is not authenticated.

Example configuration:
```yaml
worker_replication_secret: "secret_secret"
```
### `redis`

Configuration for Redis when using workers. This *must* be enabled when
Expand All @@ -3765,6 +3881,7 @@ redis:
port: 6379
password: <secret_password>
```
---
## Background Updates ##
Configuration settings related to background updates.

Expand Down
14 changes: 8 additions & 6 deletions docs/workers.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,13 +123,15 @@ public internet; replication traffic is:
### Worker configuration

In the config file for each worker, you must specify:
* The type of worker (`worker_app`). The currently available worker applications are listed below.
* A unique name for the worker (`worker_name`).
* The type of worker ([`worker_app`](usage/configuration/config_documentation.md#worker_app)).
The currently available worker applications are listed below.
* A unique name for the worker ([`worker_name`](usage/configuration/config_documentation.md#worker_name)).
* The HTTP replication endpoint that it should talk to on the main synapse process
(`worker_replication_host` and `worker_replication_http_port`)
* If handling HTTP requests, a `worker_listeners` option with an `http`
listener, in the same way as the [`listeners`](usage/configuration/config_documentation.md#listeners)
option in the shared config.
([`worker_replication_host`](usage/configuration/config_documentation.md#worker_replication_host) and
[`worker_replication_http_port`](usage/configuration/config_documentation.md#worker_replication_http_port))
* If handling HTTP requests, a [`worker_listeners`](usage/configuration/config_documentation.md#worker_listeners)
option with an `http` listener, in the same way as the
[`listeners`](usage/configuration/config_documentation.md#listeners) option in the shared config.
* If handling the `^/_matrix/client/v3/keys/upload` endpoint, the HTTP URI for
the main process (`worker_main_http_uri`).

Expand Down