Continuous integration deploy

Continuous integration deploy

The following describes the steps for setting up a continuous integration deploy on an Amazon EC2 Ubuntu Server 18.04 LTS. It can be readily adapted to other environments.

The deploy upgrades docker and docker-compose, fetches the latest docker-compose.yml, pulls the latest images, and restarts the containers.

When creating your server, make sure all necessary firewall ports are open. As of now the necessary ports are:

• 15672 (RabbitMQ Admin)

All usernames and passwords can be found in docker-compose.yml and example.secrets.env.

1. As root, install docker and docker-compose:

    apt-get update
    curl -sSL https://get.docker.com/ | sh
   

2. Manually add your user to the docker group in /etc/group or if using AWS, add the ubuntu user: usermod -aG docker ubuntu

3. Still as root user:

   apt-get install python3-pip
   pip3 install -U docker-compose
   

4. Log out and log back in. Verify that docker is working with docker ps.

5. Follow these (or similar) instructions for setting up email.

6. Add the deploy script as /opt/sfm/run_sfm.sh or if on AWS as /home/ubuntu/run_sfm.sh:

    #!/bin/bash 
   
    function error_exit
    {
    	echo "SFM CI deploy: $1" 1>&2
    	exit 1
    }
   
    export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
   
    #Upgrade docker and docker-compose
    curl -sSL https://get.docker.com/ > get_docker.sh
    if [ "$?" != "0" ]; then
    	error_exit "Downloading docker install failed"
    fi
   
    sh get_docker.sh
    if [ "$?" != "0" ]; then
    	error_exit "Docker install failed"
    fi
   
    pip3 install -U docker-compose
    if [ "$?" != "0" ]; then
            error_exit "Upgrading docker-compose failed"
    fi
   
    #Get files
    curl -L https://github.com/gwu-libraries/sfm-docker/raw/master/example.docker-compose.yml > docker-compose.yml
    if [ "$?" != "0" ]; then
            error_exit "Fetching docker-compose.yml failed"
    fi
    curl -L https://github.com/gwu-libraries/sfm-docker/raw/master/example.env > .env
    if [ "$?" != "0" ]; then
            error_exit "Fetching .env failed"
    fi
    curl -L https://github.com/gwu-libraries/sfm-docker/raw/master/smoketests.docker-compose.yml > smoketests.docker-compose.yml
    if [ "$?" != "0" ]; then
            error_exit "Fetching smoketests.docker-compose.yml failed"
    fi
   
    #Kill and delete existing
    docker kill $(docker ps -q)
    docker rm $(docker ps -a -q)
   
    #Remove dangling images
    docker rmi $(docker images -q -f dangling=true)
   
    #Pull containers
    docker-compose -f docker-compose.yml -f smoketests.docker-compose.yml pull
    if [ "$?" != "0" ]; then
            error_exit "Pulling containers failed"
    fi
   
    #Start containers
    docker-compose up -d
    if [ "$?" != "0" ]; then
            error_exit "Bringing up containers failed"
    fi
   
    sleep 60
   
    #Run smoke tests
    docker-compose -f docker-compose.yml -f smoketests.docker-compose.yml run --rm smoketests /bin/bash -c "appdeps.py --port-wait mq:5672 --port-wait ui:8080 && python -m unittest discover"
    if [ "$?" != "0" ]; then
            error_exit "Smoke test failed"
    fi
   
    echo "SFM CI deploy: Succeeded"
   

and then make executable:

    chmod +x /opt/sfm/run_sfm.sh

5. Setup cron to run script at 1am every night and email the results:

    sudo crontab -e
   

and add line (with appropriate email address):

    0 1 * * * /opt/sfm/run_sfm.sh > /opt/sfm/log.txt 2>&1; cat /opt/sfm/log.txt | mail -s "`tail -1 /opt/sfm/log.txt`" sfmadmin@email.gwu.edu