Update Ansible playbooks to make for an easy one-button install of mailserver stack on new VPS

[ksylvan]

Classification

• Feature/Enhancement
• Documentation

Reproducibility

• Not applicable

Docker information

This is a task to update the https://github.com/ksylvan/docker-mail-server repository, which was based on whatever version of Ansible was running back 6 or 7 years ago and the excellent hardware/mailserver stack that is the precursor to the current mailserver2 repository.

Assigning this to myself. 😄

[ksylvan]

@AndrewSav @sknight80 @SaraSmiseth I'm going to create a new repository for the Ansible playbooks.

As I wrote on discord, I don't like the name docker-mail-server, since it doesn't really describe the whole concept of "your full featured domain as code" for people who aren't necessarily familiar with Docker and Ansible.

Any ideas for a new name for the repo?

[AndrewSav]

@ksylvan while I applaud your effort, my opinion that making something as complicated as our setup easy is futile. Do not get me wrong if it works, I'm all for it. I'm very experienced with ansible, and I know it is not nice to work with or troubleshoot and many things can break. With outlook as such I feel I'm not qualified to suggest a better name. Ping me if/when you need testing, I might be able to help with that.

[ksylvan]

@AndrewSav I appreciate your point of view.

I did it once, about six years ago, and it made deploying 4 domains with similar configurations very easy, so I'll want to do it again with our updated mailserver2 repo.

I'll hit you up when I'm ready for testing.

[AndrewSav]

@ksylvan This is the "script" I have been using to set up the mailserver. I never published it because I feel it is specific to my own use case. Feel free to ignore, but just posting here in case it's useful. This is from a few years back, so some things can be broken now, too.

Setting up mailserver

DNS records

Make sure that A records for these applications point to the same IP address of the installation performed:

• mail (main host name)
• webmail (rainloop web ui)
• postfixadmin
• spam (rspamd)
• traefik (if traefik dashboard is exposed)

Create an MX record (prefer. 10) for each domain being set up point it to mail of the main domain. See this for more details and for TXT records for SPF and DMARK. Note, that DKIM keys will be set up later, when generated.

Docker

Install docker and compose:

sudo apt-get install -o Dpkg::Options::="--force-confold" -y apt-transport-https ca-certificates curl git
DOCKER_COMPOSE_VERSION=`git ls-remote https://github.com/docker/compose | grep refs/tags | grep -oE "[0-9]+\.[0-9][0-9]+\.[0-9]+$" | sort --version-sort | tail -n 1`
curl -fsSL get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo apt-get install jq -y
sudo curl -L https://github.com/docker/compose/releases/download/$DOCKER_COMPOSE_VERSION/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Swap

Enable swap:

sudo swapon --show
sudo fallocate -l 1G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo nano /etc/fstab
#/swapfile swap swap defaults 0 0
sudo swapon --show
sudo free -h
sudo reboot

Mailserver

Go to the mailserver directory, and put .env file with custom settings there. Take it from KeePass (mailserver.env).

docker network create http_network
mkdir -p /mnt/docker/acme
touch /mnt/docker/acme/acme.json
chmod 600 /mnt/docker/acme/acme.json
docker-compose up -d

Run docker-compose logs | grep -i generated and grab MariaDB generated root password. Store in KeePass. Note that since it takes a bit of time for Traefik to obtain the new certificates, mailserver cannot detect them on startup. Once the certs are created (which you can check by going to the webmail url in the browser) run:

docker-compose up -d --force-recreate

Examine logs docker-compose logs -f for errors.

Rainloop (admin)

• Perform initial configuration, repeat smtp/imap/sieve steps for all domains.
• Confgiure rainloop storage in MariaDB store Rainloop db password in KeePass.
• Configure Rainloop change password plugin for postfixadmin

Configure integrations. For google select:

• Authorization
• Google Drive Integration (Compose view)
• Google Viewer Integration (Preview for Microsoft Word, Excel and PowerPoint files)

Get Client Id, Client Secret, and API Key from KeePass.

Postfixadmin

Follow Postfixadmin initial configuration. Record setup password, admin email and password in last pass. Create all domains in postfixadmin.
In each domain create a mailbox and a catch-all alias. Record credentials in KeePass.

Rainloop (user)

Go to the webmail URL in the browser and login as one of the users you just created in postfixadmin. Use Add account from the dropdown menu in top right corner to add the rest of accounts created in postfix. Use Settings -> Social to set up google authentication. If you have mail backup, restore to /mnt/docker/mail/vhosts/. After that you will need to re-check checks in the Settings -> Folder List, and change and save filters in Settings -> Filters.

DKIM

Setup TXT records for DKIM. See this for more details, look at /mnt/docker/mail/dkim/ for keys.

Test

• Check you can login to spam.
• Check you can send and receive emails.
• Check, that Sieve filters are still working.
• Check email with https://www.mail-tester.com/.

docker-compose.yml:

# IPv4 only
# docker network create http_network

# IPv4/IPv6 network
# docker network create http_network --ipv6 --subnet "fd00:0000:0000:0000::/64"
# Refer to https://github.com/hardware/mailserver/#ipv6-support for more information.

networks:
  http_network:
    external: true
  mail_network:
    external: false

services:
 
  traefik:
    image: traefik:${TRAEFIK_DOCKER_TAG}
    container_name: "traefik"
    restart: ${RESTART_MODE}
    networks:
      - http_network
    ports:
      # This allows incoming connection on 80 to be forwarder to port 80 of traefik
      - "80:80"
      # This allows incoming connection on 443 to be forwarder to port 443 of traefik
      - "443:443"
      # As above. Browse to port 8080 http to see trafik dashboard
      # - "8080:8080"
    volumes:
      # static config
      - "./traefik.toml:/traefik.toml"
      # dynamic config
      - "./file.toml:/file.toml"
      # let's encrypt data
      - "${VOLUMES_ROOT_PATH}/acme:/acme"
      # This is required for the docker provider of traefik to work (read container labels, etc)
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=http_network"
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.rule=Host(`traefik.${MAILSERVER_DOMAIN}`)"
      - "traefik.http.routers.traefik.service=api@internal"
      - "traefik.http.routers.traefik.middlewares=auth"
      - "traefik.http.middlewares.auth.basicauth.users=${TRAEFIK_AUTH}"
      - "traefik.http.routers.traefik.tls=true"
      - "traefik.http.routers.traefik.tls.certresolver=letsencrypt"
      - "traefik.http.routers.traefik.tls.domains[0].main=traefik.${MAILSERVER_DOMAIN}"
      - "traefik.http.routers.traefik.tls.options=default"

  mailserver:
    image: mailserver2/mailserver:${MAILSERVER_DOCKER_TAG}
    #image: sarasmiseth/mailserver:testing
    container_name: mailserver
    restart: ${RESTART_MODE}
    domainname: ${MAILSERVER_DOMAIN}                    # Mail server A/MX/FQDN & reverse PTR = mail.domain.tld.
    hostname: ${MAILSERVER_HOSTNAME}
    # extra_hosts:                          - Required for external database (on other server or for local database on host)
    #  - "mariadb:xx.xx.xx.xx"              - Replace with IP address of MariaDB server
    #  - "redis:xx.xx.xx.xx"                - Replace with IP address of Redis server
    ports:
      - "25:25"       # SMTP                - Required
    # - "110:110"     # POP3       STARTTLS - Optional - For webmails/desktop clients
      - "143:143"     # IMAP       STARTTLS - Optional - For webmails/desktop clients
    # - "465:465"     # SMTPS      SSL/TLS  - Optional - Enabled for compatibility reason, otherwise disabled
      - "587:587"     # Submission STARTTLS - Optional - For webmails/desktop clients
      - "993:993"     # IMAPS      SSL/TLS  - Optional - For webmails/desktop clients
    # - "995:995"     # POP3S      SSL/TLS  - Optional - For webmails/desktop clients
      - "4190:4190"   # SIEVE      STARTTLS - Optional - Recommended for mail filtering
    # - "11334:11334" # HTTP                - Optional - Rspamd WebUI
    environment:
      - DBPASS=${DATABASE_USER_PASSWORD}       # MariaDB database password (required)
      - RSPAMD_PASSWORD=${RSPAMD_PASSWORD}     # Rspamd WebUI password (required)
      - ADD_DOMAINS=${ADD_DOMAINS}
      - WHITELIST_SPAM_ADDRESSES=test@example.com,another@domain.tld
      - DEBUG_MODE=rspamd
    # - ADD_DOMAINS=aa.tld, www.bb.tld...      # Add additional domains separated by commas (needed for dkim keys etc.)
    # - DEBUG_MODE=true                        # Enable Postfix, Dovecot, Rspamd and Unbound verbose logging
    # - ENABLE_POP3=true                       # Enable POP3 protocol
    # - ENABLE_FETCHMAIL=true                  # Enable fetchmail forwarding
    # - DISABLE_RATELIMITING=false             # Enable ratelimiting policy
    # - DISABLE_CLAMAV=true                    # Disable virus scanning
    # - DISABLE_SIGNING=true                   # Disable DKIM/ARC signing
    # - DISABLE_GREYLISTING=true               # Disable greylisting policy
    #
    # Full list : https://github.com/hardware/mailserver#environment-variables
    #
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=http_network"
      - "traefik.http.routers.spam.entrypoints=websecure"
      - "traefik.http.routers.spam.rule=Host(`spam.${MAILSERVER_DOMAIN}`)"
      - "traefik.http.routers.spam.service=spam"
      - "traefik.http.routers.spam.tls=true"
      - "traefik.http.routers.spam.tls.certresolver=letsencrypt"
      - "traefik.http.routers.spam.tls.domains[0].main=${MAILSERVER_HOSTNAME}.${MAILSERVER_DOMAIN}"
      - "traefik.http.routers.spam.tls.domains[0].sans=spam.${MAILSERVER_DOMAIN}"
      - "traefik.http.routers.spam.tls.options=default"
      - "traefik.http.services.spam.loadbalancer.server.port=11334"
      - "traefik.http.services.spam.loadbalancer.server.scheme=http"
    volumes:
      - ${VOLUMES_ROOT_PATH}/mail:/var/mail
      - ${VOLUMES_ROOT_PATH}/acme:/etc/letsencrypt/acme
    depends_on:
      - mariadb
      - redis
    networks:
      - mail_network
      - http_network

  # Administration interface
  # https://github.com/hardware/postfixadmin
  # http://postfixadmin.sourceforge.net/
  # Configuration : https://github.com/hardware/mailserver/wiki/Postfixadmin-initial-configuration
  postfixadmin:
    image: mailserver2/postfixadmin:${POSTFIXADMIN_DOCKER_TAG}
    restart: ${RESTART_MODE}
    domainname: ${MAILSERVER_DOMAIN}
    hostname: ${MAILSERVER_HOSTNAME}
    container_name: postfixadmin
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=http_network"
      - "traefik.http.routers.postfixadmin.entrypoints=websecure"
      - "traefik.http.routers.postfixadmin.rule=Host(`postfixadmin.${MAILSERVER_DOMAIN}`)"
      - "traefik.http.routers.postfixadmin.service=postfixadmin"
      - "traefik.http.routers.postfixadmin.tls=true"
      - "traefik.http.routers.postfixadmin.tls.certresolver=letsencrypt"
      - "traefik.http.routers.postfixadmin.tls.domains[0].main=postfixadmin.${MAILSERVER_DOMAIN}"
      - "traefik.http.routers.postfixadmin.tls.options=default"
      - "traefik.http.services.postfixadmin.loadbalancer.server.port=8888"
      - "traefik.http.services.postfixadmin.loadbalancer.server.scheme=http"
    environment:
      - DBPASS=${DATABASE_USER_PASSWORD}
    depends_on:
      - mailserver
      - mariadb
    networks:
      - mail_network
      - http_network

  # Webmail (Optional)
  # https://github.com/hardware/rainloop
  # https://www.rainloop.net/
  # Configuration : https://github.com/hardware/mailserver/wiki/Rainloop-initial-configuration
  rainloop:
    image: mailserver2/rainloop:${RAINLOOP_DOCKER_TAG}
    container_name: rainloop
    restart: ${RESTART_MODE}
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=http_network"
      - "traefik.http.routers.rainloop.entrypoints=websecure"
      - "traefik.http.routers.rainloop.rule=Host(`webmail.${MAILSERVER_DOMAIN}`)"
      - "traefik.http.routers.rainloop.service=rainloop"
      - "traefik.http.routers.rainloop.tls=true"
      - "traefik.http.routers.rainloop.tls.certresolver=letsencrypt"
      - "traefik.http.routers.rainloop.tls.domains[0].main=webmail.${MAILSERVER_DOMAIN}"
      - "traefik.http.routers.rainloop.tls.options=default"
      - "traefik.http.services.rainloop.loadbalancer.server.port=8888"
      - "traefik.http.services.rainloop.loadbalancer.server.scheme=http"
    volumes:
      - ${VOLUMES_ROOT_PATH}/rainloop:/rainloop/data
    #environment:
    #  LOG_TO_STDOUT: "true"
    depends_on:
      - mailserver
      - mariadb
    networks:
      - mail_network
      - http_network

  # Authoritative DNS server (Optional)
  # https://github.com/hardware/nsd-dnssec
  # https://www.nlnetlabs.nl/projects/nsd/
  # Configuration : https://github.com/hardware/mailserver/wiki/NSD-initial-configuration
  # nsd:
  #   image: hardware/nsd-dnssec
  #   restart: ${RESTART_MODE}
  #   ports:
  #     - "53:53"
  #     - "53:53/udp"
  #   volumes:
  #     - ${VOLUMES_ROOT_PATH}/nsd/conf:/etc/nsd
  #     - ${VOLUMES_ROOT_PATH}/nsd/zones:/zones
  #     - ${VOLUMES_ROOT_PATH}/nsd/db:/var/db/nsd

  # Database
  # https://github.com/docker-library/mariadb
  # https://mariadb.org/
  mariadb:
    image: mariadb:10.5
    container_name: mariadb
    restart: ${RESTART_MODE}
    # Info : These variables are ignored when the volume already exists (if databases was created before).
    environment:
      - MYSQL_RANDOM_ROOT_PASSWORD=yes
      - MYSQL_DATABASE=postfix
      - MYSQL_USER=postfix
      - MYSQL_PASSWORD=${DATABASE_USER_PASSWORD}
    volumes:
      - ${VOLUMES_ROOT_PATH}/mysql/db:/var/lib/mysql
    networks:
      - mail_network
    #command: --skip-grant-tables
    

  # Cache Database
  # https://github.com/docker-library/redis.
  # https://redis.io/
  redis:
    image: redis:6.0-alpine
    container_name: redis
    restart: ${RESTART_MODE}
    command: redis-server --appendonly yes
    sysctls:
      - net.core.somaxconn=1024
    volumes:
      - ${VOLUMES_ROOT_PATH}/redis/db/:/data
    networks:
      - mail_network

.env:

# This file is used to define environment variables to be used
# for variable substitution in your docker compose file.
# https://docs.docker.com/compose/env-file/

# Your domain name (eg. domain.tld)
MAILSERVER_DOMAIN=CHANGEME

# MariaDB/PostgreSQL database password
DATABASE_USER_PASSWORD=CHANGEME

# Rspamd WebUI and controller password
RSPAMD_PASSWORD=CHANGEME

# Traefik dashboard password
TRAEFIK_AUTH=CHANGEME

# Your mailserver hostname (eg. mail for mail.domain.tld)
MAILSERVER_HOSTNAME=CHANGEME

# Mailserver version
MAILSERVER_DOCKER_TAG=CHANGEME

# Docker volumes parent folder
VOLUMES_ROOT_PATH=/mnt/docker

# Docker containers restart mode
# https://docs.docker.com/compose/compose-file/#restart
RESTART_MODE=unless-stopped

POSTFIXADMIN_DOCKER_TAG=CHANGEME

RAINLOOP_DOCKER_TAG=CHANGEME

TRAEFIK_DOCKER_TAG=CHANGEME

ADD_DOMAINS=CHANGEME,CHANGEME

file.toml:

[tls.options.default]
minVersion = "VersionTLS12"
cipherSuites = [
  "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305",
  "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
  "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
  "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
  "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
  "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA"
]

traefik.toml:

## static configuration

[entryPoints.websecure]
address = ":443"

[entryPoints.web]
address = ":80"
[entryPoints.web.http.redirections.entryPoint]
to = "websecure"

[providers.docker]
exposedByDefault=false

[providers.file]
filename = "/file.toml"


[log]
#level = "DEBUG"

#[accessLog]

[api]
#insecure = true

[certificatesResolvers.letsencrypt.acme]
email = "CHANGEME"
#caserver= "https://acme-staging-v02.api.letsencrypt.org/directory"
storage = "/acme/acme.json"
[certificatesResolvers.letsencrypt.acme.tlsChallenge]

[ksylvan]

Good stuff, @AndrewSav. Thank you.

Did you look at my original scripts in https://github.com/ksylvan/docker-mail-server ?

[AndrewSav]

@ksylvan well I clicked on the link and saw the repo and the readme, but I did not go over any files as I was not sure at what state of completion they were.

[ksylvan]

@AndrewSav Those scripts are complete and I used them to deploy a few domains.

I want to recreate that basic idea now using Mailer2 with improvements.

[AndrewSav]

@ksylvan pretty advanced stuff, selinux, fail2ban, both RedHat and Debian support, looks like you had put in a lot of work back then.

[ksylvan]

@AndrewSav Yes, and all of it is working on four existing domains even up to now, and I recently updated all of them to use mailserver2; so I want to improve on the scripts/deployment. One of my clients back then was very impressed that their domain was up and working within a couple of hours of us buying their domain name.