Docker image for theportalwiki.com's Matrix.org server.

WTF is this?

What is this repo?

A pwiki-synapse docker image, which runs a Synapse server. Synapse is a Matrix.org server implementation.
Soon: Another thing to run a web client?
A guide to set this all up.

How do I use it?

DNS setup

Make sure to set up an SRV record pointing to port 443 on the host that will be running the Synapse server.

$ dig -t srv _matrix._tcp.theportalwiki.com
_matrix._tcp.theportalwiki.com. 3600    IN      SRV     10 0 443 theportalwiki.com.

Because we will reverse-proxy the Synapse server, please do not use port 443 for SYNAPSE_PORT below. SYNAPSE_PORT is the host-local port to which the Synapse server will be bound.

Host setup

Checkout the repo

$ git clone https://github.com/ThePortalWiki/pwiki-matrix
$ cd pwiki-matrix

Make sure to back these up.

Host user

You should create a new user on the host with a unique UID/GID. Reusing the port number of the Synapse server can be helpful here. The rest of this README assumes that the UID/GID are both 8449.

$ PWIKI_SYNAPSE_UID=8449
$ PWIKI_SYNAPSE_GID=8449
$ useradd --uid="$PWIKI_SYNAPSE_UID" --gid="$PWIKI_SYNAPSE_GID" -s /bin/false -d / -M pwiki-synapse

TLS volume

The Synapse server requires a valid TLS key and certificate in order to work. These are exposed to the container inside a /tls volume, which is expected to contain at least the following files:

/tls
├── privkey.pem    # TLS private key.
└── fullchain.pem  # Complete TLS certificate chain.

The rest of this README assumes that this lives on the host in the /etc/pwiki-synapse/tls directory, and it is group-readable by the host group with GID 9001. This GID will be used such that the non-root user inside the Docker container can share the same GID so that it is able to read the TLS files from the /tls volume. It is differentiated from SYNAPSE_GID so that it remains possible to have certs for a single domain owned by a shared group on the host, rather than having to maintain separate copies. These filenames within this directory match those generated by LetsEncrypt, although in practice LetsEncrypt does not make its directories available to a certain group. You will need to add your own automation to copy them out of there and/or chgrp them.

$ PWIKI_SYNAPSE_TLS=/etc/pwiki-synapse/tls
$ stat $PWIKI_SYNAPSE_TLS
  File: /etc/pwiki-synapse/tls
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: 800h/2048d      Inode: 690709      Links: 2
Access: (0750/drwxr-x---)  Uid: (    0/    root)   Gid: ( 9001/     tls)
Access: 2016-12-30 17:59:09.120460354 -0500
Modify: 2016-12-30 17:59:09.120460354 -0500
Change: 2016-12-30 17:59:22.193321816 -0500
 Birth: -
$ PWIKI_TLS_GID="$(stat -c '%g' "$(readlink -e "$PWIKI_SYNAPSE_TLS")")"

Media volume

Synapse requires persistent filesystem storage to store files uploaded by users. Pick some directory on the host to store them. The rest of this README assumes that this lives inside /var/lib/pwiki-synapse/media. It is mounted as /synapse-media in the Docker container. It needs to be writable by the synapse user.

Might want to set up backup for that directory as well, although it is not as critical.

$ PWIKI_SYNAPSE_MEDIA=/var/lib/pwiki-synapse/media
$ chown -R "$PWIKI_SYNAPSE_UID:$PWIKI_SYNAPSE_GID" "$PWIKI_SYNAPSE_MEDIA"

Build the Docker images

TURN server image

Synapse requires a TURN server for establishing VoIP connections. This runs in a separate container running a coturn server. Synapse communicates to coturn using a shared secret key in order to set up credentials that users can then use on the TURN server.

Like the Synapse server, it supports TLS, so it also requires access to the TLS cert and key file.

$ COTURN_UID=3478
$ COTURN_GID=3478
$ COTURN_DOMAIN=theportalwiki.com
$ COTURN_PORT=3478
$ docker build                                 \
    --build-arg="COTURN_UID=$COTURN_UID"       \
    --build-arg="COTURN_GID=$COTURN_GID"       \
    --build-arg="TLS_GID=$PWIKI_TLS_GID"       \
    --build-arg="COTURN_DOMAIN=$COTURN_DOMAIN" \
    --build-arg="COTURN_PORT=$COTURN_PORT"     \
    --tag=pwiki-synapse-coturn                 \
    images/pwiki-synapse-coturn

Build arguments

COTURN_UID: The UID to create the internal user. Optional, default 3478.
COTURN_UID: The GID to create the internal user. Optional, default 3478.
TLS_GID: The group ID of the TLS mount. Required.
COTURN_DOMAIN: The external domain name to advertise. Required.
COTURN_PORT: The port number for TCP and UDP TURN requests. Optional. Default 3478.
REBUILD: You can set --build-arg=REBUILD=$(date) to force a rebuild and update all packages within.

Synapse image

$ SYNAPSE_DOMAIN=theportalwiki.com
$ SYNAPSE_PORT=8449
$ docker build                                   \
    --build-arg="SYNAPSE_UID=$PWIKI_SYNAPSE_UID" \
    --build-arg="SYNAPSE_GID=$PWIKI_SYNAPSE_GID" \
    --build-arg="TLS_GID=$PWIKI_TLS_GID"         \
    --build-arg="SYNAPSE_DOMAIN=$SYNAPSE_DOMAIN" \
    --build-arg="SYNAPSE_PORT=$SYNAPSE_PORT"     \
    --build-arg="COTURN_DOMAIN=$COTURN_DOMAIN"   \
    --build-arg="COTURN_PORT=$COTURN_PORT"       \
    --tag=pwiki-synapse                          \
    images/pwiki-synapse

Build arguments

TLS_GID: The group ID of the TLS mount. Required.
SYNAPSE_UID: The UID to create the internal user. Optional, default 8449. Should match $PWIKI_SYNAPSE_UID.
SYNAPSE_GID: The GID to create the internal user. Optional, default 8449. Should match $PWIKI_SYNAPSE_GID.
SYNAPSE_DOMAIN: The domain name for the Synapse server. You can change this to reuse the image for non-pwiki purposes.
SYNAPSE_PORT: The host-local port number that will be forwarded to the Synapse server. 8448 by default. It is only used in order to point the nginx reserve proxy to it; it does not need to match whatever is set as Matrix SRV record for SYNAPSE_DOMAIN. It is only useful to be able to modify this port if you have more than one Synapse server running on the same host.
COTURN_DOMAIN: The external domain name to advertise for TURN. Optional. Defaults to $SYNAPSE_DOMAIN.
COTURN_PORT: The port number for TCP and UDP TURN requests. Optional. Default 3478.
REBUILD: You can set --build-arg=REBUILD=$(date) to force a rebuild and update all packages within.

PostgreSQL image

$ docker build                 \
    --tag=pwiki-synapse-pgsql  \
    images/pwiki-synapse-pgsql

Generate Synapse secrets volume

You need to generate a bunch of secrets for Synapse to work. The rest of this README assumes that you will be storing them in /etc/pwiki-synapse/secrets. These will be mounted into the Synapse container as a volume under /secrets. They should only be readable by root, both inside and outside the container.

Part of the secrets generation involves generating an ed25519 signing key using a special format, so we use a script bundled within the pwiki-synapse image to generate it.

This only needs to be done once but may be safely re-ran as needed if new types of secrets are added. Existing secrets will not be overwritten.

$ PWIKI_SYNAPSE_SECRETS=/etc/pwiki-synapse/secrets
$ docker run --rm                              \
    --name=pwiki-synapse-secrets               \
    --volume="$PWIKI_SYNAPSE_SECRETS:/secrets" \
    pwiki-synapse generate-secrets
$ tree "$PWIKI_SYNAPSE_SECRETS"
/etc/pwiki-synapse/secrets
├── macaroon.key
├── pepper.key
├── postgresql_superuser.password
├── postgresql_synapse.password
├── registration.key
├── signing.key
├── signing_key_id.pub
├── tls.dh
└── turn_shared_secret.key

Back up all of these files immediately.

Run the PostgreSQL container

$ docker run --detach                              \
    --name=pwiki-synapse-pgsql                     \
    --volume="$PWIKI_SYNAPSE_SECRETS:/secrets"     \
    --log-driver=journald                          \
    pwiki-synapse-pgsql

Run the coturn container

$ docker run --detach                              \
    --name=pwiki-synapse-coturn                    \
    --volume="$PWIKI_SYNAPSE_SECRETS:/secrets"     \
    --volume="$PWIKI_SYNAPSE_TLS:/tls"             \
    --publish="$COTURN_PORT:$COTURN_PORT/tcp"      \
    --publish="$COTURN_PORT:$COTURN_PORT/udp"      \
    --log-driver=journald                          \
    pwiki-synapse-coturn

Run the Synapse container

$ docker run --detach                              \
    --name=pwiki-synapse                           \
    --link=pwiki-synapse-pgsql:postgres            \
    --volume="$PWIKI_SYNAPSE_SECRETS:/secrets"     \
    --volume="$PWIKI_SYNAPSE_TLS:/tls"             \
    --volume="$PWIKI_SYNAPSE_MEDIA:/synapse-media" \
    --publish="127.0.0.1:$SYNAPSE_PORT:8448"       \
    --log-driver=journald                          \
    pwiki-synapse

This binds to the port $SYNAPSE_PORT on the host's lo interface. Note that the Docker-side port should always be 8448.

Your Matrix server should now be running. Logs can be read with

$ journalctl CONTAINER_NAME=pwiki-synapse

Set up nginx reverse proxying

The above command doesn't expose the server to the outside world. Instead, you should reserve-proxy it behind nginx. This doesn't actually have many advantages but it does mean that your server can be reached on the default TLS port (443) rather than 8448.

$ ./resources/generate-nginx-config.sh             \
    --domain="$SYNAPSE_DOMAIN"                     \
    --tls-key="$PWIKI_SYNAPSE_TLS/privkey.pem"     \
    --tls-cert="$PWIKI_SYNAPSE_TLS/fullchain.pem"  \
    --dh-params="$PWIKI_SYNAPSE_SECRETS/tls.dh"    \
    --synapse-port="$SYNAPSE_PORT"                 \
    | tee "/etc/nginx/conf/synapse-$SYNAPSE_DOMAIN.conf"

The script will generate a config file that looks roughly like this:

server {
    server_name "theportalwiki.com";
    listen 443 default_server ssl;
    listen [::]:443 default_server ssl;
    ... More TLS stuff...
    # include "your_custom_file_here.conf";
    location /_matrix {
        ... Proxy stuff...
    }
}

You may optionally pass an extra --include=your_custom_file_here.conf argument, which will add an include "your_custom_file_here.conf"; line in the middle of the generated server block. This lets you reuse the server block for non-Matrix purposes.

Note: This uses the default_server directive, which means that this server block will be selected for all requests sent to port 443. This is necessary because Synapse does not currently support SNI on the federation port. This has the side-effect that you cannot run two Synapse servers on the same host with reverse-proxying on port 443, and that any other HTTPS website running on this host with default_server will need to be modified to remove the default_server directive. These sites will no longer work without SNI.

Make sure that the user nginx runs as can read all the TLS files given as argument, then reload nginx.

$ systemctl reload nginx

You can verify that everything works by visiting using the Matrix.org federationtester tool:

$ curl "https://matrix.org/federationtester/api/report?server_name=$SYNAPSE_DOMAIN"

Registering users

You can register users manually using the built-in registration script bundled inside the pwiki-synapse image. Note that this requires the Synapse server to already be running.

$ docker run --rm -it                           \
    --volume="$PWIKI_SYNAPSE_SECRETS:/secrets"  \
    pwiki-synapse register                      \
    --username=root --password=hunter2          \
    --admin=true

The arguments are a subset of Synapse's own register_new_matrix_user script, as the other arguments are taken care of for you.

Arguments

-u or --user=foo: Username for the new user.
-p or --password=foo: Raw password for the new user. If unspecified, will be prompted interactively.
-a or --admin=true|false: Whether the user is an admin on the server.

Save all these variables for future use

By this point you have a lot of IMPORTANT_UPPERCASE_VARIABLES in your shell. Here's how to save them for the next time to run some docker run commands or the like.

$ ( set -o posix ; set ) | ./resources/save-important-variables.sh > /etc/pwiki-synapse/sourceme.sh

You can then restore the variables in another shell session with:

$ source /etc/pwiki-synapse/sourceme.sh

More to come...

TODO

Trust other domains for identity purposes (e.g. perot.me, lagg.me, colinjstevens.com etc.)
Allow new channel creation by users
Set up other container that shows a fancy web client thing
Allow guest registration (requires ReCaptcha)? Or at least guest viewing
Set up PostgreSQL backups
Add script to generate systemd units for building and running the containers with the correct arguments

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

WTF is this?

What is this repo?

How do I use it?

DNS setup

Host setup

Checkout the repo

Host user

TLS volume

Media volume

Build the Docker images

TURN server image

Build arguments

Synapse image

Build arguments

PostgreSQL image

Generate Synapse secrets volume

Run the PostgreSQL container

Run the coturn container

Run the Synapse container

Set up nginx reverse proxying

Registering users

Arguments

Save all these variables for future use

More to come...

TODO

Files

README.md

Latest commit

History

README.md

File metadata and controls

WTF is this?

What is this repo?

How do I use it?

DNS setup

Host setup

Checkout the repo

Host user

TLS volume

Media volume

Build the Docker images

TURN server image

Build arguments

Synapse image

Build arguments

PostgreSQL image

Generate Synapse secrets volume

Run the PostgreSQL container

Run the coturn container

Run the Synapse container

Set up nginx reverse proxying

Registering users

Arguments

Save all these variables for future use

More to come...

TODO