Option to deploy in docker swarm

[rafagsiqueira]

Dear contributors,

I have been working on deploying indico to a docker swarm for the last couple of weeks and since it finally worked I thought I should share the configuration parameters with you:

For the docker swarm a few extra objects are needed, since it is not possible to use local mounts and also to persist data when a node is lost.

First we create an overlay network for communication of all the services from multiple nodes:
docker network create -d overlay indico

Then a few configuration files are "uploaded" to the docker swarm:

docker config create indico-conf indico.conf
docker config create indiconginx-conf nginx.conf
docker config create indicodb-conf create-extensions.sh

This is the content of create-extensions.sh:

#!/bin/bash
set -e

psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
    CREATE EXTENSION unaccent;
    CREATE EXTENSION pg_trgm;
EOSQL

This is the content of nginx.conf. A few changes are necessary to avoid nginx from crashing when it can't reach indico-static or indico-web:

server {
  listen 80;
  listen [::]:80;

  access_log            /dev/stdout combined;
  error_log             /dev/stdout info;

  location /.xsf/indico/ {
    internal;
    alias /opt/indico/;
  }

  location ~ ^/static/assets/(core|(?:plugin|theme)-[^/]+)/(.*)$ {
    alias /opt/indico/static/assets/$1/$2;
    access_log off;
  }

  location ~ ^/(ihelp|css|images|js|(static/fonts))(/.*)$ {
    resolver 127.0.0.11; 
    set $static indico-static;
    proxy_pass  http://$static:8080;
  }

  location /robots.txt {
    alias /opt/indico/static/htdocs/robots.txt;
    access_log off;
  }

  location /static/custom {
    alias /opt/indico/custom;
    access_log off;
  }

  location / {
    resolver 127.0.0.11;
    root /var/empty/nginx;
    set $web indico-web;
    proxy_pass http://$web:59999;
    proxy_set_header  Host $host:$server_port;
    proxy_set_header  X-Real-IP $http_cf_connecting_ip;
    proxy_set_header  X-Forwarded-For $http_cf_connecting_ip;
    proxy_set_header  X-Forwarded-Proto $http_x_forwarded_proto;
    client_max_body_size 1G;
  }

}

Then I created volumes that can be mounted as needed on the nodes when running each container. This example is for a Docker Swarm deployed on AWS. But volumes can be created using other type of volume plugins:

docker volume create \
  -d "cloudstor:aws" \
  --opt ebstype=gp2 \
  --opt size=1 \
  --opt backing=relocatable \
  indicodb-vol
docker volume create \
  -d "cloudstor:aws" \
  --opt ebstype=gp2 \
  --opt size=1 \
  --opt backing=relocatable \
  indicostatic-vol
docker volume create \
  -d "cloudstor:aws" \
  --opt ebstype=gp2 \
  --opt size=1 \
  --opt backing=relocatable \
  indicocustom-vol

Then I tagged one of the nodes to make sure the containers that use indicocustom-vol and indicostatic-vol, both run on the same node:
docker node update --label-add has_volumes=indico

Finally the docker-cloud.yml. Note that I had to build my own version of indico-static, since it wasn't available in your docker hub repo. But I replaced the value in this config considering that you might want to also push the indico-static image:

version: "3.4"
services:
  indico-web: &indico-web
    image: getindico/indico:latest
    command: /opt/indico/run_indico.sh
    environment:
      - SERVICE_HOSTNAME=<your hostname>
      - SERVICE_PORT=80
      - SERVICE_PROTOCOL=http
      - PGHOST=indico-postgres
      - PGUSER=indico
      - PGPASSWORD=<hidden>
      - PGDATABASE=indico
      - PGPORT=5432
      - INDICO_DEFAULT_TIMEZONE=US/Eastern
      - INDICO_DEFAULT_LOCALE=en_GB
      - REDIS_CACHE_URL=redis://indico-redis:6379/1
      - CELERY_BROKER=redis://indico-redis:6379/0 
      - USE_EXTERNAL_DB=y
      - C_FORCE_ROOT=true
      - INDICO_AUTH_PROVIDERS={}
      - INDICO_IDENTITY_PROVIDERS={}
      - INDICO_LOCAL_IDENTITIES=yes
      - SECRET_KEY=<your secret key>
    volumes:
      - 'indicocustom-vol:/opt/indico/custom'
      - 'indicostatic-vol:/opt/indico/static'
    networks:
      - indico
    deploy:          
      placement:                                                           
        constraints:                                                       
          - node.labels.has_volumes == indico
    configs:
      - source: indico-conf
        target: /opt/indico/etc/indico.conf
    tmpfs:
      - /opt/indico/tmp
  indico-static: 
    image: getindico/indico-static:latest
    networks:
      - indico
  indico-celery:
    <<: *indico-web
    command: /opt/indico/run_celery.sh
    ports: []
    volumes: []
    networks:
      - indico
  indico-redis:
    image: redis
    networks:
      - indico
  indico-postgres:
    image: postgres
    environment:
      - POSTGRES_USER=indico
      - POSTGRES_PASSWORD=<hidden>
      - POSTGRES_DB=indico
      - PGDATA=/var/lib/postgresql/data/pgdata
    volumes:
      - 'indicodb-vol:/var/lib/postgresql/data'
    configs:
      - source: indicodb-conf
        target: /docker-entrypoint-initdb.d/create-extensions.sh
    networks:
      - indico
    deploy:
      placement:
        constraints:
          - node.labels.has_volumes == indico
  indico-nginx:
    image: nginx:latest
    networks:
      - traefik-net
      - indico
    deploy:
      labels:
        - "traefik.port=80"
        - "traefik.frontend.rule=Host:<your hostname>"
        - "traefik.docker.network=traefik-net"
      placement:
        constraints:
          - node.labels.has_volumes == indico
    environment:
      - SERVICE_HOSTNAME=<your hostname>
      - SERVICE_PROTOCOL=http
    configs:
      - source: indiconginx-conf
        target: /etc/nginx/conf.d/default.conf 
    volumes:
      - 'indicocustom-vol:/opt/indico/custom'
      - 'indicostatic-vol:/opt/indico/static'
volumes:
  indicocustom-vol:
    external: true
  indicostatic-vol:
    external: true
  indicodb-vol:
    external: true
configs:
  indico-conf:
    external: true
  indiconginx-conf:
    external: true
  indicodb-conf:
    external: true
networks:
  traefik-net:
    external: true
  indico:
    external: true

Also please note that I left the traefik parameters. Traefik is the load balancer I use for our docker swarm cluster. Hope you can appreciate the effort. And again thank you for your help!

[pferreir]

Thanks a lot for the detailed report, it's much appreciated! My question is: would it be feasible to add the extra attributes to the existing docker-compose-yml? As far as I understand, you can use it with docker-swarm too.

[tamasgal]

Is this still the "state of the art" deployment to a swarm or are there any updates? I need to migrate a very old instance (2.2.4.) with hundreds of users into our swarm.

[ThiefMaster]

No idea to be honest, I don't know anyone (besides the user who opened this issue) that's deploying Indico on Docker Swarm.