Reduce the downtime required when performing deployments

[osg74]

The current deployment process can be improved so that we have minimal downtime of the live server by implementing the following improvements:

• the system packages, virtualenv and app code can all be deployed before taking the live site down
• the only mandatory downtime is for the database rebuild since we need to prevent any database writes so that we don't lose data while the deployment is in progress
• we can use versioned symlinks to switch over to the new release with minimal downtime

[osg74]

Reassigned to @carlio -- could you check that the new deployment process meets the above requirements where applicable please?

[carlio]

The deployment process is as follows:

In a new folder:

1. Create a new virtualenv
2. Check out the required commit hash
3. Install all dependencies
4. Shut down RSR gunicorn process(es)
5. Run DB migrations if necessary
6. Change "current" symlink to new version
7. Start RSR gunicorn process

The downtime between builds should be no more than a second or so if there are no migrations to run. Additionally, Nginx continues to run, meaning we could serve a custom 502.html if we wanted to.

See https://github.com/akvo/akvo-provisioning/tree/develop/puppet/modules/rsr/templates

[osg74]

Perfect, thanks.