Skip to content

Upgrade the prod server

Arnaud Morvan edited this page Dec 7, 2016 · 12 revisions
# Connect to the composition manager server
ssh compose0.exoscale.infra.camptocamp.org

# a bunch of envvars with the list of all the servers
source /opt/c2corg-docker/server-list.env

# one directory for each production server, self-descriptive names:
ls /opt/c2corg-docker/composition/production/

# choose one
cd /opt/c2corg-docker/composition/production/api0/

Notice the hidden ./.env in this directory.
https://docs.docker.com/compose/environment-variables/#/the-env-file
This envvar file teaches docker-compose to connect to a remote docker
instance. Each per-server directory has it's own .env file.

You can now edit the docker-compose.yml file. In this example, you'll want
to change the image version to a more recent one which travis made
available on dockerhub (https://hub.docker.com/r/c2corg/v6_api/tags/).

# download the new image
docker-compose pull

# shutdown the running container(s)
docker-compose down

# re-create it with the new image version (you'll notice 2 monitoring
# containers get stopped/started too)
docker-compose up -d

# make sure everything is working properly
docker-compose ps
docker-compose logs -f

# don't forget to save your changes
git add . && git commit

BTW, please don't clone this repository elsewhere. It's full of credentials, and it's backuped on the c2c.org backup server.

Now repeat the docker-compose dance for the api1 and api2 subdirectories. Congrats, you've done a rolling upgrade of the service with zero downtime !

You can check haproxy's status here: http://www.camptocamp.org:8008/stats (assuming you have modified your /etc/hosts). You should notice one backend going offline if you refresh the page between docker-compose "down" and "up".

Also, I setup 2 basic kibana dashboards (more to come) you'll probably want to keep track of the first few days:

https://c2corgv6.logs.camptocamp.net/kibana/#/dashboard/elasticsearch/Python%20stacktraces https://c2corgv6.logs.camptocamp.net/kibana/#/dashboard/elasticsearch/HTTP%2050x