A slide show version of this presentation is available at this link: CICD Learning Lab Slides
- Environment Prep
- Stage 1 - Continuous Integration
- Stage 2 - Continuous Delivery
- Install the Application
- Stage 3 - Continuous Deployment
- Stage 4 - Monitoring and Notify Phase
- Bonus - CICD In Action!
- Clean-Up
Here are links to each part of the lab. They do build on each other so be sure to go in order.
This lab is intended to be an introduction to setting up a very basic CI/CD (Continuous Integration/Continuous Delivery) Pipeline. There are many different technologies and methods that can be used for CI/CD, in this lab we will use:
- Cisco's Mantl Container Stack - Mantl.io which includes:
- Docker Container Engine - Docker
- Mesos Scheduler - Mesos/Marathon
- Development Language - Python and Flask
- Source Control - github.com
- Container Registry - hub.docker.com
- CICD Server - drone.io
- Notifications - CiscoSpark.com
To run through this lab, you will need to have accounts (all free) created with the following services, as well as a set of tools properly setup on your laptop or workstation.
-
CiscoSpark.com Developer Token
- You will also use your Spark login to retrieve the Spark API token.
- Go to developer.ciscospark.com, where you will find your token by logging into the developer portal, clicking on your image in the upper right corner, and clicking Copy
- Note if you are using 2FA for your GitHub account, let the admin know, as that will change how you do authentication later in the lab exerices.
To successfully complete this lab, you must have a supported working environment (i.e. laptop/workstation). There are two options for meeting this requirement:
- Natively use your own workstation by installing and configuring the below list of software and utilities.
- Run the lab within the provided Docker container to avoid the need to install additional software on your workstation.
-
For this option, you will need to have the ability to run a Docker container on your workstation. Options for this include native Docker installation, boot2docker, or docker-machine options.
-
Though Windows Operation Systems are becoming more common with developers, and many development tools and utilities are being made available natively for Windows users, the exercises in this lab are NOT supported on Windows Natively. If you are working from a Windows machine, please use the provided Docker Container option.
If you will be completing the lab using your native workstation, you will need to ensure the following components are installed and functioning correctly.
- git
- command line tools required
- GUI clients optional
- Installation Instructions: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
- docker
- for platforms with native docker available (ie Linux/Mac) having the docker daemon installed and running can be used
- docker-machine is also fully acceptable for this lab
- Installation Instructions: https://docs.docker.com/engine/installation/
- Python 2.7
- Installation Instructions: https://www.python.org/downloads/
- If you already have Python 3.x installed, you may still need to install 2.7.
- Installation Instructions: https://www.python.org/downloads/
- drone command line tools
- drone is the CICD tool used for this lab, and the command line tools are used to properly secure the secret information (i.e. userids, passwords) used for all the services utilized
- the tools require docker to be installed and working
- Installation Instructions: http://readme.drone.io/0.4/devs/cli/
- Basic Installation Steps
- Linux
curl http://downloads.drone.io/drone-cli/drone_linux_amd64.tar.gz | tar zxsudo install -t /usr/local/bin drone
- Mac
curl http://downloads.drone.io/drone-cli/drone_darwin_amd64.tar.gz | tar zxsudo cp drone /usr/local/bin
This lab uses Drone version 0.4. Drone 0.5 is the latest release which introduced new concepts. Check back later when this lab is updated to reflect 0.5
If your workstation is Windows based, or you prefer to NOT use your native workstation, you can run the lab exercises from within a Docker container.
To use this method, you will need to be able to run a Docker container on a host machine somewhere.
From your Docker host, run the following command to pull down and enter an interactive shell on the provided development container.
# It may take some time to complete this command while the full container is downloaded
docker run -it --name cicdlab hpreston/devbox:cicdlab
[root@cf95a414877e coding]#
This will put you at a prompt similar to the above. The container is a Linux based working environment with the following utilities and software installed and ready to use.
- nano
- Provided as a text editor to use for executing the lab steps
- git
- docker
-
the container has the docker tools installed, but the
docker runcommand above will NOT enable you to run additional containers from inside -
running containers is not a required step in the lab, but if you would like to do so, you can use the following command instead
docker run -it --name cicdlab -v /var/run/docker.sock:/var/run/docker.sock hpreston/devbox:cicdlab -
this command will link the docker daemon on the host machine into the container
-
- drone cli tools
If you exit out of the container before completing the lab and want to continue from where you left off, do not execute a docker run command again. This will create a new clean container that lacks any of your work. Instead follow the below to start the original container.
# Verify that you have a container in a stopped state
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cf95a414877e hpreston/devbox:cicdlab "/bin/bash" 2 minutes ago Exited (0) 10 seconds ago cicdlab
# Restart your stopped container
docker start -i cicdlab
[root@cf95a414877e coding]#
Your instructor will provide you an TXT file with this content needed to complete the lab.
# CICD Learning Lab Infrastructure Details
# Lab Guide
https://github.com/imapex-training/cicd_learning_lab/blob/master/README.md
# Build Server
drone server address:
# Target Cloud Infrastructure
Mantl Control Server Address:
Mantl Username:
Mantl Password:
Mantl Application Domain:
# Spark Room Info
Spark Room Name:
Spark RoomId:
For this lab you will be leveraging a Lab Mantl Instance and Drone Build Server. Your lab admin will provide the following information. Make a note of these details as you will need them periodically during the following lab exercises.
Before beginning this step, be sure to be at a command line prompt from your prepared working environment. This will either be your local machine, or within the provided container.
# Start a clean instance of the container
docker run -it --name cicdlab hpreston/devbox:cicdlab
[root@cf95a414877e coding]# exit
# If you need to restart an exited container
# Verify that you have a container in a stopped state
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cf95a414877e hpreston/devbox:cicdlab "/bin/bash" 2 minutes ago Exited (0) 10 seconds ago cicdlab
# Restart your stopped container
docker start -i cicdlab
[root@cf95a414877e coding]#
With all the pre-reqs completed, you are ready to start the lab. We'll start by setting up our new application code repo, container repository, and continuous integration server configuration.
-
Log into GitHub and visit the demo app repo imapex-training/cicd_demoapp
-
Click Fork to create a copy of the repo in your account
- Make a local clone of YOUR repo on your laptop. Do NOT clone the
imapex-training/cicd_demoapprepo. In this case, we will create a directory called "coding". If you choose to deviate from that directory name, just remember what directory you created for your local repo, as this will be used multiple times through the lab.
Cloning
# if you don't have a local directory where you keep projects, create one
mkdir ~/coding
# enter this new directory (or where ever you keep your code)
cd ~/coding
# clone your fork of the demo app
# replace USERNAME with your GitHub user name
git clone https://github.com/USERNAME/cicd_demoapp
Here are some basic details about the demoapp we are building.
- Web Application written in Python using Flask Framework
- It's a "Hello World" Application
- Deployed via a Docker Container
- Includes test cases for full coverage
- Includes Installation Scripts
Checkout the source repo and code: imapex-training/cicd_demoapp
-
Log into hub.docker.com with your account
-
Click the blue Create Repository button in the upper right corner
-
Name the repo cicd_demoapp and provide a short description.
-
Make sure the lab administrator has enabled your GitHub account on the lab server.
-
Navigate to the drone server address provided by the lab administrator, and click Login.
-
Drone uses GitHub for authentication, so you will either be prompted to log into your account, and then authorize drone, or simply authorize drone if you're already logged into GitHub.
-
Once you've logged in, click the Available Repositories tab in the upper right corner.
-
Find the cicd_demoapp repo in the list, and click to activate it. Drone will then setup a WebHook in GitHub to be notified of relevant events, such as as code pushes, automatically.
-
On the Active Repositories tab, you should now see a new entry for the cicd_demoapp. If you click on it, there should be no builds reported yet.
environment:
SPARK_TOKEN: <FROM YOUR DEVELOPER.CISCOSPARK.COM ACCOUNT>
SPARK_ROOM: <ROOMID PROVIDED BY THE LAB ADMIN>
DOCKER_USERNAME: <YOUR HUB.DOCKER.COM USERNAME>
DOCKER_PASSWORD: <YOUR HUB.DOCKER.COM PASSWORD>
DOCKER_EMAIL: <YOUR HUB.DOCKER.COM EMAIL ADDRESS>
MANTL_USERNAME: <MANTL USER PROVIDED BY LAB ADMIN>
MANTL_PASSWORD: <MANTL PASSWORD PROVIDED BY LAB ADMIN>
MANTL_CONTROL: <MANTL SERVER ADDRESS PROVIDED BY LAB ADMIN>
In order for Drone to be able to do the hard work of testing your application, building a container and publishing it to your registry, notifying your team of status, and deploying the updated code to production, it requires credentials for several systems to act on your behalf. These details are often referred to as secrets in application development. As a general rule, you want to protect these details through encryption to maintain their security. Commiting your passwords to a code repo is never a good idea, but it is especially bad when using a public repo like github.com.
Drone provides a method to create an encrypted file with needed secrets that can be safely included in a code repo. This is why we installed the drone command line utilities as part of the pre-reqs.
In this step you will be entering several commands in a terminal window. These need to be run from your local repo directory. If you followed the directions when cloning the repo locally, this command will place you in the correct directory
cd ~/coding/cicd_demoapp
-
The drone utilities on your laptop need to know the address and access information for the drone server you are using. We use session environment variables for this. You will replace the variable's value with the information the lab admin gives you. The follow code can be copied and pasted directly into a terminal window if you'd like to do that, but you can also just type in the line that doesn't start with the hash mark.
# Configure the drone server address, # Use the address provided by the lab administrator export DRONE_SERVER=http://DRONE_SERVER
-
Find your personal Drone token by viewing your profile in the drone console. Click the down arrow next to your picture in the upper right corner and navigate to Profile. Click the button to Show Token and copy the displayed value.
-
Copy/Paste or execute this command to store the value in your terminal session.
# Configure your token export DRONE_TOKEN=<your token>
-
Test the command line tools by listing the repositories configured. You should see your cicd_demoapp listed like below. If you don't see something similar, please let the lab admin know.
drone repo ls <yourusername>/cicd_demoapp
-
Next we will create the clear text version of our secrets file. This file is only used to build the encrypted version and should NEVER be added to or commited to your repo. A sample template for the secrets file was included in the demo app. We will copy this file and edit it to put in the values that we will use for the lab exercises.
cp drone_secrets_sample.yml drone_secrets.yml
-
Edit the copied file in whatever IDE or editor you prefer. You'll need to provide the details on each line of the file. This is a YML format, so be sure to maintain proper spacing, including a single space after the colon in each line.
environment: SPARK_TOKEN: <FROM YOUR DEVELOPER.CISCOSPARK.COM ACCOUNT> SPARK_ROOM: <EITHER ONE OF YOUR OWN ROOMS, OR A ROOMID PROVIDED BY THE LAB ADMIN> DOCKER_USERNAME: <YOUR HUB.DOCKER.COM USERNAME> DOCKER_PASSWORD: <YOUR HUB.DOCKER.COM PASSWORD> DOCKER_EMAIL: <YOUR HUB.DOCKER.COM EMAIL ADDRESS> MANTL_USERNAME: <MANTL USER PROVIDED BY LAB ADMIN> MANTL_PASSWORD: <MANTL PASSWORD PROVIDED BY LAB ADMIN> MANTL_CONTROL: <MANTL SERVER ADDRESS PROVIDED BY LAB ADMIN>
- Save your secrets file, but do NOT add or commit it to your repo.
- Pay particular attention that you do not mistakenly ADD your completed secrets file to git. If you are leveraging an IDE or GUI to execute the git add/commit/push steps, it is very easy to send sensitive files to GitHub
-
Run this command to create the encrypted .drone.sec file.
# Replace USERNAME with your GitHub username drone secure --repo USERNAME/cicd_demoapp --in drone_secrets.yml
- Add this file to git, commit and push it to GitHub. When you
git pushyou may be prompted for your GitHub credentials, provide them. The code below will commit and push ONLY the encrypted file, not the clear text file.- Note if you are using a graphical IDE and choose to use that for your commit/push actions, it's very easy to accidentally save and commit all changed files in a directory, therefore accidentally pushing your clear text secrets YML file. Be conscious of what you commit and push.
# add the file to the git repo
git add .drone.sec
# commit the change
git commit -m "Added .drone.sec file"
# push changes to GitHub
git push
-
Return to the drone server web interface and look at your repo status. You should see a build has kicked off.
In each of the following steps, you will be repeating the steps to secure your secrets. If you close your terminal window, you will need to re-export the DRONE_SERVER and DRONE_TOKEN values
Before beginning this step, be sure to be at a command line prompt from your prepared working environment. This will either be your local machine, or within the provided container.
# Start a clean instance of the container
docker run -it --name cicdlab hpreston/devbox:cicdlab
[root@cf95a414877e coding]# exit
# If you need to restart an exited container
# Verify that you have a container in a stopped state
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cf95a414877e hpreston/devbox:cicdlab "/bin/bash" 2 minutes ago Exited (0) 10 seconds ago cicdlab
# Restart your stopped container
docker start -i cicdlab
[root@cf95a414877e coding]#
The first step of the CICD process is to react to new code commits and test the changes against the larger application needs. In this lab, we will configure drone to send an alert to a Spark room each time a commit has been made.
In the root of the code repository is a file named .drone.yml that provides the instructions to Drone on what actions to take upon each run. We will be updating this file at each stage of the lab to move from Continuous Integration -> Delivery -> Deployment.
In this step you will be entering several commands in a terminal window. These need to be run from your local repo directory. If you followed the directions when cloning the repo locally, this command will place you in the correct directory
cd ~/coding/cicd_demoapp
- Open the .drone.yml file in your editor. The file should begin looking like the code block below. The build directive at the top represents where the integration phase of the process will take place.
In the commands: list under run_tests: are the different steps needed to test a successful code change.
build:
build_starting:
image: python:2
commands:
- echo "Beginning new build"
- Now we will add Testing to the build phase. Update
.drone.ymlto match the updated content below. In the commands: list under run_tests: are the different steps needed to test a successful code change.- NOTE You can simply copy and paste the contents from below directly into your file.
build:
build_starting:
image: python:2
commands:
- echo "Beginning new build"
run_tests:
image: python:2-alpine
commands:
- pip install -r requirements.txt
- python testing.py
-
As part of the security of drone, every change to the .drone.yml file requires the secrets file to be recreated. Since we've updated this file, we need to re-secure our secrets file.
# Replace USERNAME with your GitHub username drone secure --repo USERNAME/cicd_demoapp --in drone_secrets.yml
-
If the command above gives the error
Error: you must provide the Drone server address., you likely closed the terminal window after the Environment Prep step. Run these commands to reset the environment variables.export DRONE_SERVER=http://DRONE_SERVER export DRONE_TOKEN=<your token>
-
Now commit and push the changes to the drone configuraiton and secrets file to GitHub.
- Remember if you are using an IDE, be careful not to commit & push the changes to the file drone_secrets.yml
# add the file to the git repo git add .drone.sec git add .drone.yml # commit the change git commit -m "Updated Build Phase to run UnitTests" # push changes to GitHub git push
-
Now check the Drone web interface.
-
Click on the build and scroll through the log displayed. You can use this log to monitor the process in each stage as we add additional steps to the build process. The Drone log window also has an arrow at the bottom right that you can click and it will automatically tail the end of the log file as it progresses.
- If the build reports a Failure, check the log to see what might have gone wrong. Common reasons include forgetting to re-create the secrets file, forgetting to commit the secrets file after recreating, or mis-entered credentials in the plain text secrets file.
Okay, so drone said it did something... but you may be wondering what actually happened. This image and walkthrough shows the steps that are occuring along the way.
- You committed and pushed code to GitHub.com
- GitHub sent a WebHook to the Drone server notifying it of the committed code.
- Drone checks the .drone.yml file and executes the commands in the build phase. During this phase, Drone has two steps:
- build_starting
- Fetch a container from hub.docker.com to begin the build. This container is identified in the
image: python:2line of the drone config file. Drone will run the commands listed in this section
- Fetch a container from hub.docker.com to begin the build. This container is identified in the
- run_tests
- Fetch a container from hub.docker.com to do the testing. This container is identified in the
image: python:2-alpineline of the drone config file. Drone will run the tests described by the commands listed in this section
- Fetch a container from hub.docker.com to do the testing. This container is identified in the
- build_starting
Before beginning this step, be sure to be at a command line prompt from your prepared working environment. This will either be your local machine, or within the provided container.
# Start a clean instance of the container
docker run -it --name cicdlab hpreston/devbox:cicdlab
[root@cf95a414877e coding]# exit
# If you need to restart an exited container
# Verify that you have a container in a stopped state
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cf95a414877e hpreston/devbox:cicdlab "/bin/bash" 2 minutes ago Exited (0) 10 seconds ago cicdlab
# Restart your stopped container
docker start -i cicdlab
[root@cf95a414877e coding]#
In this step, we will take our successfully tested application, build a Docker Container, and publish it to a docker registry where it can be used as an artifact for our application.
In the root of the code repository is a file .drone.yml that provides the instructions to drone on what actions to take upon each run. We will be updating this file at each stage of the lab to move from Continuous Integration -> Delivery -> Deployment.
In this step you will be entering several commands in a terminal window. These need to be run from your local repo directory. If you followed the directions when cloning the repo locally, this command will place you in the correct directory
cd ~/coding/cicd_demoapp
- Open the .drone.yml file in your editor. We will be adding the next phase, publish to the existing configuration. Update your drone file to look like this.
- Note You can simply copy and paste the contents from below directly into your file. The notation of $$VARIABLE references details stored encrypted within the .drone.sec file. Do NOT replace them with clear text credentials.
build:
build_starting:
image: python:2
commands:
- echo "Beginning new build"
run_tests:
image: python:2-alpine
commands:
- pip install -r requirements.txt
- python testing.py
publish:
docker:
repo: $$DOCKER_USERNAME/cicd_demoapp
tag: latest
username: $$DOCKER_USERNAME
password: $$DOCKER_PASSWORD
email: $$DOCKER_EMAIL
-
As part of the security of drone, every change to the .drone.yml file requires the secrets file to be recreated. Since we've updated this file, we need to re-secure our secrets file.
# Replace USERNAME with your GitHub username drone secure --repo USERNAME/cicd_demoapp --in drone_secrets.yml
-
If the command above gives the error
Error: you must provide the Drone server address., you likely closed the terminal window after the Environment Prep step. Run these commands to reset the environment variables.export DRONE_SERVER=http://DRONE_SERVER export DRONE_TOKEN=<your token>
-
Now commit and push the changes to the drone configuraiton and secrets file to GitHub.
- Remember if you are using an IDE, be careful not to commit & push the changes to the file drone_secrets.yml
# add the file to the git repo git add .drone.sec git add .drone.yml # commit the change git commit -m "Added Publish Phase to build and push container" # push changes to GitHub git push
-
Now check the Drone web interface. A new build should have kicked off.
- If the build reports a Failure, check the log to see what might have gone wrong. Common reasons include forgetting to re-create the secrets file, forgetting to commit the secrets file after recreating, or mis-entered credentials in the plain text secrets file.
-
Once the build completes, log into hub.docker.com, and open your repo. Click the Tags tab. You should see a new tag of latest displayed. This is the container that was build and pushed up by drone.
Okay, so drone said it did something... but you may be wondering what actually happened. This image and walkthrough shows the steps that are occuring along the way.
-
You committed and pushed code to GitHub.com
-
GitHub sent a WebHook to the Drone server notifying it of the committed code.
-
Drone checks the .drone.yml file and executes the commands in the build phase. During this phase, Drone has two steps:
- build_starting
- Fetch a container from hub.docker.com to begin the build. This container is identified in the
image: python:2line of the drone config file. Drone will run the commands listed in this section
- Fetch a container from hub.docker.com to begin the build. This container is identified in the
- run_tests
- Fetch a container from hub.docker.com to do the testing. This container is identified in the
image: python:2-alpineline of the drone config file. Drone will run the tests described by the commands listed in this section
- Fetch a container from hub.docker.com to do the testing. This container is identified in the
- build_starting
-
Drone checks the .drone.yml file and executes the commands in the publish phase. During this phase, Drone will:
- Build a Docker Container using the Dockerfile definition included in the Git repo
- Push the container up to hub.docker.com using the credentials contained in the secrets file
Before beginning this step, be sure to be at a command line prompt from your prepared working environment. This will either be your local machine, or within the provided container.
# Start a clean instance of the container
docker run -it --name cicdlab hpreston/devbox:cicdlab
[root@cf95a414877e coding]# exit
# If you need to restart an exited container
# Verify that you have a container in a stopped state
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cf95a414877e hpreston/devbox:cicdlab "/bin/bash" 2 minutes ago Exited (0) 10 seconds ago cicdlab
# Restart your stopped container
docker start -i cicdlab
[root@cf95a414877e coding]#
Now that we have used CICD to automate the creation of a Docker container for our application, we can now deploy our application. This step could certainly be automated as well, but in this lab we will manually install our application to illustrate the process and rely on automation to keep it up to date.
- Included in your repository are three files that are used to define, install, and uninstall the application.
- sample-demoapp.json is a template for the Marathon Application definition that will be used. A custom version will be created at installation that references YOUR Docker container.
- app_install.sh is a bash script that installs the demo application. This has three steps.
- Collect environment details: Lab Mantl Address, Lab Username, Lab Password, Your Docker Username, and the Lab Application Domain
- Create an application definition for your deployment
- Install your application using the REST API for Marathon.
- app_uninstall.sh is a bash script that uninstalls the demoapp. This has two steps.
- Collect environment details: Lab Mantl Address, Lab Username, Lab Password, and Your Docker Username
- Destroy your application using the REST API for Marathon.
- You could manually install the application using the Marathon GUI, however the GUI lacks the ability to configure certain parameters we need for the lab. Besides... deploying through APIs is so much cooler!
In this step you will be entering several commands in a terminal window. These need to be run from your local repo directory. If you followed the directions when cloning the repo locally, this command will place you in the correct directory
cd ~/coding/cicd_demoapp
From the root of your code repository...
- Execute the installation script. Remember to check the information the lab administrator has provided for the correct responses to the script prompts. Be sure to pay attention to the last few lines of the script output, as these will give you the URLs for your application (you'll need this to test the app) and the control interface for Marathon.
./app_install.sh
# You will be prompted to enter the information needed before the applicaiton deploys.
# The process will look like this
Please provide the following details on your lab environment.
What is the address of your Mantl Control Server?
eg: control.mantl.internet.com
control.mantl.domain.com
What is the username for your Mantl account?
admin
What is the password for your Mantl account?
**HIDDEN**
What is the your Docker Username?
hpreston
What is the Lab Application Domain?
mantl.domain.com
***************************************************
Installing the demoapp as class/hpreston
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1508 0 997 100 511 1643 842 --:--:-- --:--:-- --:--:-- 1642
{
"acceptedResourceRoles": null,
"args": null,
"backoffFactor": 1.15,
"backoffSeconds": 1,
"cmd": null,
"constraints": [],
"container": {
"docker": {
"forcePullImage": true,
"image": "hpreston/cicd_demoapp:latest",
"network": "BRIDGE",
"parameters": [],
"portMappings": [
{
"containerPort": 5000,
"hostPort": 0,
"protocol": "tcp",
"servicePort": 0
}
],
"privileged": false
},
"type": "DOCKER",
"volumes": []
},
"cpus": 0.1,
"dependencies": [],
"deployments": [
{
"id": "7e659013-3b2e-4446-954e-2405c3ebc865"
}
],
"disk": 0,
"env": {},
"executor": "",
"healthChecks": [
{
"gracePeriodSeconds": 300,
"ignoreHttp1xx": false,
"intervalSeconds": 60,
"maxConsecutiveFailures": 3,
"portIndex": 0,
"protocol": "TCP",
"timeoutSeconds": 20
}
],
"id": "/class/hpreston",
"instances": 1,
"labels": {},
"maxLaunchDelaySeconds": 3600,
"mem": 16,
"ports": [
0
],
"requirePorts": false,
"storeUrls": [],
"tasks": [],
"tasksHealthy": 0,
"tasksRunning": 0,
"tasksStaged": 0,
"tasksUnhealthy": 0,
"upgradeStrategy": {
"maximumOverCapacity": 1,
"minimumHealthCapacity": 1
},
"uris": [],
"user": null,
"version": "2016-07-01T12:45:10.685Z"
}
***************************************************
Installed
Wait 2-3 minutes for the service to deploy.
Then you can visit your application at:
http://class-hpreston.mantl.domain.com/hello/world
You can also watch the progress from the GUI at:
https://control.mantl.domain.com/marathon
-
The application is now being deployed, and you can watch the progress from the lab console address (provided by the lab admin as well as at the end of the script output).
-
Once the application is fully deployed (shouldn't take more than 3 minutes), and shows healthy status in the console, you can visit your applicaiton at the URL that was provided at the end of installation script.
Okay, so building on the process from the previous step, this diagram shows what we've added.
- You committed and pushed code to GitHub.com
- GitHub sent a WebHook to the Drone server notifying it of the committed code.
- Drone checks the .drone.yml file and executes the commands in the build phase. During this phase, Drone will:
- Fetch a container from hub.docker.com. This container is identified in the
image: python:2line of the drone config file. Drone will run the commands and tests described in this phase from the fetched container - Send a notification message to the subscribed Spark room stating that the build has begun.
- Drone checks the .drone.yml file and executes the commands in the publish phase. During this phase, Drone will:
-
Build a Docker Container using the Dockerfile definition included in the Git repo
-
Push the container up to hub.docker.com using the credentials contained in the secrets file
-
You manually Install the application on Mantl
Before beginning this step, be sure to be at a command line prompt from your prepared working environment. This will either be your local machine, or within the provided container.
# Start a clean instance of the container
docker run -it --name cicdlab hpreston/devbox:cicdlab
[root@cf95a414877e coding]# exit
# If you need to restart an exited container
# Verify that you have a container in a stopped state
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cf95a414877e hpreston/devbox:cicdlab "/bin/bash" 2 minutes ago Exited (0) 10 seconds ago cicdlab
# Restart your stopped container
docker start -i cicdlab
[root@cf95a414877e coding]#
In this step, we will automatically deploy our changes by issuing a restart command to Marathon that will pull down the new docker container.
In the root of the code repository is a file .drone.yml that provides the instructions to drone on what actions to take upon each run. We will be updating this file at each stage of the lab to move from Continuous Integration -> Delivery -> Deployment.
In this step you will be entering several commands in a terminal window. These need to be run from your local repo directory. If you followed the directions when cloning the repo locally, this command will place you in the correct directory
cd ~/coding/cicd_demoapp
- Open the .drone.yml file in your editor. We will be adding the next phase, deploy to the existing configuration. Update your drone file to look like this.
- Note You can simply copy and paste the contents from below directly into your file. The notation of $$VARIABLE references details stored encrypted within the .drone.sec file. Do NOT replace them with clear text credentials.
build:
build_starting:
image: python:2
commands:
- echo "Beginning new build"
run_tests:
image: python:2-alpine
commands:
- pip install -r requirements.txt
- python testing.py
publish:
docker:
repo: $$DOCKER_USERNAME/cicd_demoapp
tag: latest
username: $$DOCKER_USERNAME
password: $$DOCKER_PASSWORD
email: $$DOCKER_EMAIL
deploy:
webhook:
image: plugins/drone-webhook
skip_verify: true
method: POST
auth:
username: $$MANTL_USERNAME
password: $$MANTL_PASSWORD
urls:
- https://$$MANTL_CONTROL/marathon/v2/apps/class/$$DOCKER_USERNAME/restart?force=true
-
As part of the security of drone, every change to the .drone.yml file requires the secrets file to be recreated. Since we've updated this file, we need to re-secure our secrets file.
# Replace USERNAME with your GitHub username drone secure --repo USERNAME/cicd_demoapp --in drone_secrets.yml
-
If the command above gives the error
Error: you must provide the Drone server address., you likely closed the terminal window after the Environment Prep step. Run these commands to reset the environment variables.export DRONE_SERVER=http://DRONE_SERVER export DRONE_TOKEN=<your token>
-
Now commit and push the changes to the drone configuraiton and secrets file to GitHub.
- Remember if you are using an IDE, be careful not to commit & push the changes to the file drone_secrets.yml
# add the file to the git repo git add .drone.sec git add .drone.yml # commit the change git commit -m "Added Deploy phase to restart application" # push changes to GitHub git push
-
Now check the Drone web interface. A new build should have kicked off. Also, watch for the Spark message to come through in the client.
- If the build reports a Failure, check the log to see what might have gone wrong. Common reasons include forgetting to re-create the secrets file, forgetting to commit the secrets file after recreating, or mis-entered credentials in the plain text secrets file.
-
While the build is running, open Marathon from the Lab Mantl installation, and watch your application. When Drone gets to the deploy phase, you should see a new task get created as the application restarts.
Okay, so drone said it did something... but you may be wondering what actually happened. This image and walkthrough shows the steps that are occuring along the way.
-
You committed and pushed code to GitHub.com
-
GitHub sent a WebHook to the Drone server notifying it of the committed code.
-
Drone checks the .drone.yml file and executes the commands in the build phase. During this phase, Drone has two steps:
- build_starting
- Fetch a container from hub.docker.com to begin the build. This container is identified in the
image: python:2line of the drone config file. Drone will run the commands listed in this section
- Fetch a container from hub.docker.com to begin the build. This container is identified in the
- run_tests
- Fetch a container from hub.docker.com to do the testing. This container is identified in the
image: python:2-alpineline of the drone config file. Drone will run the tests described by the commands listed in this section
- Fetch a container from hub.docker.com to do the testing. This container is identified in the
- build_starting
-
Drone checks the .drone.yml file and executes the commands in the publish phase. During this phase, Drone will:
- Build a Docker Container using the Dockerfile definition included in the Git repo
- Push the container up to hub.docker.com using the credentials contained in the secrets file
- Drone checks the .drone.yml file and executes the commands in the deploy phase. In this phase, the following actions will take place:
- Drone sends a WebHook command to Marathon to cause an application restart
- Marathon pulls the new container from hub.docker.com containing the code changes
Before beginning this step, be sure to be at a command line prompt from your prepared working environment. This will either be your local machine, or within the provided container.
# Start a clean instance of the container
docker run -it --name cicdlab hpreston/devbox:cicdlab
[root@cf95a414877e coding]# exit
# If you need to restart an exited container
# Verify that you have a container in a stopped state
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cf95a414877e hpreston/devbox:cicdlab "/bin/bash" 2 minutes ago Exited (0) 10 seconds ago cicdlab
# Restart your stopped container
docker start -i cicdlab
[root@cf95a414877e coding]#
In this step, we will configure the job to send a notification to the Spark room indicating whether the build was successful or failed.
In the root of the code repository is a file .drone.yml that provides the instructions to drone on what actions to take upon each run. We will be updating this file at each stage of the lab to move from Continuous Integration -> Delivery -> Deployment.
In this step you will be entering several commands in a terminal window. These need to be run from your local repo directory. If you followed the directions when cloning the repo locally, this command will place you in the correct directory
cd ~/coding/cicd_demoapp
- Open the .drone.yml file in your editor. We will be adding the next phase, notify to the existing configuration. Update your drone file to look like this.
- Note You can simply copy and paste the contents from below directly into your file. The notation of $$VARIABLE references details stored encrypted within the .drone.sec file. Do NOT replace them with clear text credentials.
build:
build_starting:
image: python:2
commands:
- echo "Beginning new build"
run_tests:
image: python:2-alpine
commands:
- pip install -r requirements.txt
- python testing.py
publish:
docker:
repo: $$DOCKER_USERNAME/cicd_demoapp
tag: latest
username: $$DOCKER_USERNAME
password: $$DOCKER_PASSWORD
email: $$DOCKER_EMAIL
deploy:
webhook:
image: plugins/drone-webhook
skip_verify: true
method: POST
auth:
username: $$MANTL_USERNAME
password: $$MANTL_PASSWORD
urls:
- https://$$MANTL_CONTROL/marathon/v2/apps/class/$$DOCKER_USERNAME/restart?force=true
notify:
spark:
image: hpreston/drone-spark
auth_token: $$SPARK_TOKEN
roomId: $$SPARK_ROOM
-
As part of the security of drone, every change to the .drone.yml file requires the secrets file to be recreated. Since we've updated this file, we need to re-secure our secrets file.
# Replace USERNAME with your GitHub username drone secure --repo USERNAME/cicd_demoapp --in drone_secrets.yml
-
If the command above gives the error
Error: you must provide the Drone server address., you likely closed the terminal window after the Environment Prep step. Run these commands to reset the environment variables.export DRONE_SERVER=http://DRONE_SERVER export DRONE_TOKEN=<your token>
-
Now commit and push the changes to the drone configuraiton and secrets file to GitHub.
- Remember if you are using an IDE, be careful not to commit & push the changes to the file drone_secrets.yml
# add the file to the git repo git add .drone.sec git add .drone.yml # commit the change git commit -m "Added Notify Phase" # push changes to GitHub git push
-
Now check the Drone web interface. A new build should have kicked off. Also, watch for the Spark message to come through in the client.
- If the build reports a Failure, check the log to see what might have gone wrong. Common reasons include forgetting to re-create the secrets file, forgetting to commit the secrets file after recreating, or mis-entered credentials in the plain text secrets file.
-
At the end of the build check the Spark Room. You should see a status notification regarding the state of the build.
Okay, so drone said it did something and we got a Spark message... but you may be wondering what actually happened. This image and walkthrough shows the steps that are occuring along the way.
-
You committed and pushed code to GitHub.com
-
GitHub sent a WebHook to the Drone server notifying it of the committed code.
-
Drone checks the .drone.yml file and executes the commands in the build phase. During this phase, Drone has two steps:
- build_starting
- Fetch a container from hub.docker.com to begin the build. This container is identified in the
image: python:2line of the drone config file. Drone will run the commands listed in this section
- Fetch a container from hub.docker.com to begin the build. This container is identified in the
- run_tests
- Fetch a container from hub.docker.com to do the testing. This container is identified in the
image: python:2-alpineline of the drone config file. Drone will run the tests described by the commands listed in this section
- Fetch a container from hub.docker.com to do the testing. This container is identified in the
- build_starting
-
Drone checks the .drone.yml file and executes the commands in the publish phase. During this phase, Drone will:
- Build a Docker Container using the Dockerfile definition included in the Git repo
- Push the container up to hub.docker.com using the credentials contained in the secrets file
- Drone checks the .drone.yml file and executes the commands in the deploy phase. In this phase, the following actions will take place:
- Drone sends a WebHook command to Marathon to cause an application restart
- Marathon pulls the new container from hub.docker.com containing the code changes
- Drone checks the .drone.yml file and executes the commands in the notify phase. During notification, Drone will check the staus of the build and then send notifications to the Spark room.
- If the build was successful, send a Success notification
- If the build failed, send a Failure notificiation and blame someone.
Before beginning this step, be sure to be at a command line prompt from your prepared working environment. This will either be your local machine, or within the provided container.
# Start a clean instance of the container
docker run -it --name cicdlab hpreston/devbox:cicdlab
[root@cf95a414877e coding]# exit
# If you need to restart an exited container
# Verify that you have a container in a stopped state
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cf95a414877e hpreston/devbox:cicdlab "/bin/bash" 2 minutes ago Exited (0) 10 seconds ago cicdlab
# Restart your stopped container
docker start -i cicdlab
[root@cf95a414877e coding]#
Now that we have our full CICD pipeline up and running, let's make some application changes and watch them deploy.
The default demo app has a single route hello/world. We will be adding an additional route to the application, but let's verify that it doesn't exist already.
- Try to navigate to
hello/universeat your application. This would be available at a URL such ashttp://class-USERNAME.mantl.domain.com/hello/universe, but replace USERNAME with your Docker Username, and mantl.domain.com with the Lab Application Domain provided by your instructor.
-
You should get back a page not found error.
In this step you will be entering several commands in a terminal window. These need to be run from your local repo directory. If you followed the directions when cloning the repo locally, this command will place you in the correct directory
cd ~/coding/cicd_demoapp
- In your editor or IDE, open
testing.pyand add the new tests for /hello/universe. You can simply copy and paste the below into your editor.
import demoapp
import unittest
class FlaskTestCase(unittest.TestCase):
def setUp(self):
demoapp.app.config['TESTING'] = True
self.app = demoapp.app.test_client()
def test_correct_http_response(self):
resp = self.app.get('/hello/world')
self.assertEquals(resp.status_code, 200)
def test_correct_content(self):
resp = self.app.get('/hello/world')
self.assertEquals(resp.data, '"Hello World!"\n')
def test_universe_correct_http_response(self):
resp = self.app.get('/hello/universe')
self.assertEquals(resp.status_code, 200)
def test_universe_correct_content(self):
resp = self.app.get('/hello/universe')
self.assertEquals(resp.data, '"Hello Universe!"\n')
def tearDown(self):
pass
if __name__ == '__main__':
unittest.main()
- Since we are NOT changing the actual build process stored in
.drone.yml, we don't need to recreate the secrets file. Once the build process is complete, as long as new tests or steps aren't added these files can be left alone. Developers can focus on their code and changes, not the build process.
-
Commit and push our updated test to begin the CICD process.
# add the file to the git repo git add testing.py # commit the change git commit -m "Added new Tests for hello/universe" # push changes to GitHub git push
-
Check the drone server to check the status of the build. You can also check in Spark for Status updates.
- The build failed because we built the test BEFORE the feature. That's test driven development, and the failure is exactly what we want to see. When drone fails in the testing, the new container isn't built and published.
- In your editor or IDE, open
demoapp.pyand add the new class and resource for HelloUniverse. You can simply copy and paste the below into your editor.
from flask import Flask, request
from flask_restful import Resource, Api, reqparse
app = Flask(__name__)
app = Flask(__name__)
api = Api(app)
class HelloWorld(Resource):
def get(self):
text = "Hello World!"
return text
api.add_resource(HelloWorld, '/hello/world')
class HelloUniverse(Resource):
def get(self):
text = "Hello Universe!"
return text
api.add_resource(HelloUniverse, '/hello/universe')
if __name__ == '__main__':
# Run Flask
app.run(debug=True, host='0.0.0.0', port=int("5000"))
- Since we are NOT changing the actual build process stored in
.drone.yml, we don't need to recreate the secrets file. Once the build process is complete, as long as new tests or steps aren't added these files can be left alone. Developers can focus on their code and changes, not the build process.
-
Commit and push our updated application to begin the CICD process.
# add the file to the git repo git add demoapp.py # commit the change git commit -m "Added hello/universe to the application" # push changes to GitHub git push
-
Check the drone server to verify the build has begun. You can also monitor the Spark room for the completed message.
-
Once drone completes the build, check Marathon and watch as the application restarts.
-
Wait for Marathon to show the application as healthy.
-
Refresh the web page with the
hello/universepage Not Found. You should now have your new message available.
- Feel free to experiment with more changes to the application. Each commit will result in the application restarting with the updated code.
Okay, so let's review the steps in the full pipeline.
-
You committed and pushed code to GitHub.com
-
GitHub sent a WebHook to the Drone server notifying it of the committed code.
-
Drone checks the .drone.yml file and executes the commands in the build phase. During this phase, Drone has two steps:
- build_starting
- Fetch a container from hub.docker.com to begin the build. This container is identified in the
image: python:2line of the drone config file. Drone will run the commands listed in this section
- Fetch a container from hub.docker.com to begin the build. This container is identified in the
- run_tests
- Fetch a container from hub.docker.com to do the testing. This container is identified in the
image: python:2-alpineline of the drone config file. Drone will run the tests described by the commands listed in this section
- Fetch a container from hub.docker.com to do the testing. This container is identified in the
- build_starting
-
Drone checks the .drone.yml file and executes the commands in the publish phase. During this phase, Drone will:
- Build a Docker Container using the Dockerfile definition included in the Git repo
- Push the container up to hub.docker.com using the credentials contained in the secrets file
- Drone checks the .drone.yml file and executes the commands in the deploy phase. In this phase, the following actions will take place:
- Drone sends a WebHook command to Marathon to cause an application restart
- Marathon pulls the new container from hub.docker.com containing the code changes
- Drone checks the .drone.yml file and executes the commands in the notify phase. During notification, Drone will check the staus of the build and then send notifications to the Spark room.
- If the build was successful, send a Success notification
- If the build failed, send a Failure notificiation and blame someone.
In this lab we built and put to use a full CICD pipeline for a very simple application. Full production application pipelines are more involved with a greater amount of testing to ensure only good code is deployed, but the basic steps and process are the same.
In this step you will be entering several commands in a terminal window. These need to be run from your local repo directory. If you followed the directions when cloning the repo locally, this command will place you in the correct directory
cd ~/coding/cicd_demoapp
From the root of your code repository...
- Execute the uninstallation script.
$ ./app_uninstall.sh
# You will be prompted to enter the information needed before the applicaiton deploys.
# The process will look like this
Please provide the following details on your lab environment.
What is the address of your Mantl Control Server?
eg: control.mantl.internet.com
control.mantl.domain.com
What is the username for your Mantl account?
admin
What is the password for your Mantl account?
What is the your Docker Username?
hpreston
Uninstalling the demoapp at class/hpreston
{"version":"2016-07-01T13:26:43.421Z","deploymentId":"731e0df3-056c-4340-ae03-f5155a5e3b50"}
- The application has now been destroyed.