A (magic) shell script to deploy Git repositories
Read the annotated source
Repository: AlphaHydrae/deploy
Shamelessly inspired by: visionmedia/deploy
- Usage
- Installation
- Requirements
- Configuration file
- Hooks
- Configuration file
- Command line options & environment variables
- Command line options format
- Default command
- Sub-commands
- Cleaning up old releases
deploy is a shell script to deploy your Git projects through SSH.
Add a deploy.conf
file in your project's directory.
Here's an example for a Node.js project:
# deploy.conf
[production]
repo https://github.com/me/my-app
host my.server.com
user deploy
path /var/www/app
# describe how to deploy your app
env NODE_ENV=production
deploy npm install --production
post-deploy npm start
deploy is language-agnostic. For a Rails project, you could replace the last 3 lines with:
env RAILS_ENV=production
deploy bundle install --without development test
deploy rake assets:precompile
post-deploy rails server
Now run deploy!
deploy production setup
deploy production rev master
It will:
- Connect to
my.server.com
as thedeploy
user through SSH - The
setup
command will set up a deployment structure and clone your repository - The second
rev
command will checkout the latest version of yourmaster
branch and run the deployment hooks you defined (deploy
andpost-deploy
in the configuration file)
Read on to learn what to write in the configuration file or how to use each sub-command.
With npm:
npm install -g bash-deploy
With curl:
PREFIX=/usr/local/bin \
FROM=https://raw.githubusercontent.com \
&& curl -sSLo $PREFIX/deploy \
$FROM/AlphaHydrae/deploy/master/bin/deploy \
&& chmod +x $PREFIX/deploy
With wget:
PREFIX=/usr/local/bin \
FROM=https://raw.githubusercontent.com \
&& wget -qO $PREFIX/deploy \
$FROM/AlphaHydrae/deploy/master/bin/deploy \
&& chmod +x $PREFIX/deploy
Or download it yourself.
deploy is a one-file bash script which requires the following commands: cat
, cut
, date
, git
, grep
, ls
, mkdir
, readlink
, sed
, ssh
, tail
, tar
and wc
.
Most bash shells have all of those out of the box except perhaps Git.
It also optionally requires the chmod
, cp
and mktemp
commands to update itself.
deploy reads its main configuration from a deploy.conf
file in the current directory.
It can also be customized with environment variables and command line options.
The configuration file is basically a series of sections containing key/value pairs:
# deploy.conf
[staging]
host staging.example.com
user test
# how to deploy
post-deploy ./run-test.sh
[production]
host 192.168.1.42
user root
# how to deploy
env NODE_ENV=production
deploy npm install --production
deploy npm run build
post-deploy pm2 start pm2.json
Each named section, delimited by [name]
, represents an environment (i.e. a host machine) to deploy to,
in this example the staging and production environments.
Lines beginning with #
are comments and are ignored.
Other lines are key/value pairs. A key is a sequence of characters containing no whitespace, followed by at least one tab or space.
For example, in the line deploy npm run build
, the key is deploy
and its value is npm run build
.
Some keys like repo
, host
, port
or user
are simple configuration properties that have one value per environment.
If multiple values are found, the last one is used.
These keys can be overriden through an environment variable of the same name in uppercase and prefixed by DEPLOY_
.
For example, the environment variable to override the repo
key is DEPLOY_REPO
.
Other keys like setup
, deploy
or post-deploy
are multiple-value keys used for hooks.
They can be present multiple time in the same environment and all their values will be used in order.
When deploying your project to multiple environments, there will probably be some configuration properties that are identical for multiple environments. To avoid repetition, an environment can inherit from one or multiple other environments:
# deploy.conf
[common]
user dev
deploy do-stuff
deploy do-more-stuff
[secure]
user sekret
deploy do-sensitive-stuff
[production]
inherits common
inherits secure
user root
deploy just-do-it
In this example, the production
environment inherits from common
and secure
(in that order).
The value of a single-value key like user
will be the last value found in the environment inheritance tree.
In this case, it will be root
for the production
environment (the values inherited from the common
and secure
environments are overwritten).
A multiple-value key like deploy
will include all values found in the entire inheritance tree.
In this case, it will have 4 values for the production
environments, taken in order from the common
, secure
and production
sections:
do-stuff
do-more-stuff
do-sensitive-stuff
just-do-it
For convenience in simple use cases, you can also add a default environment by including a nameless section in your configuration file:
# deploy.conf
[]
user dev
deploy do-stuff
deploy do-more-stuff
[development]
deploy do-it-somehow
[production]
user root
deploy do-it-seriously
All environments that have no inherits
key automatically inherit from the default environment.
Hooks are user-defined commands that can be run during a deployment phase.
There are currently two phases defined: setup and deploy.
The setup phase happens when you run the setup
command,
while the deploy phase happens when you run the rev
command.
There are various hooks for each phase: some that run before, some during and some after deploy does its thing. These are the currently available hooks:
pre-setup
post-setup
pre-deploy
deploy
post-deploy
Hooks are multiple-value keys that are optional and can be used as many times as you want. They will all be run in order.
Each hook is run in a specific working directory and has access to various environment variables.
$DEPLOY_PATH
is always exported and indicates the deployment directory
(which is not necessarily the same as the hook's working directory).
Additional user-defined variables may also be made available (see environment variables).
Here's an example of how you could use hooks to cache your node_modules
directory after every deployment
to shorten future installation times:
# restore cache (if present)
deploy tar -xzf $DEPLOY_PATH/cache.gz -C . || exit 0
# install new dependencies
deploy npm install --production
# update cache
post-deploy tar -czf $DEPLOY_PATH/cache.gz node_modules
See the setup
and rev
commands to learn exactly when and where hooks are executed.
-
repo <url>
(or the$DEPLOY_REPO
variable) defines the Git URL from which your repository will be cloned at setup time. -
path <dir>
(or the$DEPLOY_PATH
variable) defines the directory into which your project will be deployed on the host. -
keep <n>
(or the$DEPLOY_KEEP
variable) defines how many releases to keep after deploying. Older releases will be deleted. It must be eitherall
(the default), or an integer greater than zero.If it's a number, it indicates the number of releases that should be kept after successful deployment. For example, with
keep
set to 3, if there are 5 old releases deployed before running the script, 3 will be deleted (so that 2 old releases and the new one being deployed, 3 in total, remain).It's also possible to manually trigger cleanup with the cleanup command.
Various SSH options can be specified through the configuration file or environment variables:
-
host <address>
(or the$DEPLOY_HOST
variable) is mandatory for all environments. It indicates which host or IP address to connect to. -
user <name>
(or the$DEPLOY_USER
variable) specifies the user to connect as. By default, you connect with the same name as your local user. -
identity <file>
(or the$DEPLOY_IDENTITY
variable) specifies a file from which the private key for public key authentication is read. -
forward-agent yes
(or the$DEPLOY_FORWARD_AGENT
variable) enables agent forwarding. -
port <number>
(or the$DEPLOY_PORT
variable) specifies the host port to connect to. -
tty yes
(or the$DEPLOY_TTY
variable) forces pseudo-terminal allocation.
Commands executed on the host through SSH will be logged to the console.
For some commands, the output may be retrieved and displayed in a more user-friendly manner.
You can define environment variables that will be exported on the host when executing hooks.
The $DEPLOY_PATH
variable is always exported and indicates the deployment directory
configured by the user with the path
config key or the local $DEPLOY_PATH
variable.
-
env <NAME>=<VALUE>...
defines one or multiple environment variables to export on the host when running hooks.# deploy.conf env FOO=BAR env BAZ=QUX CORGE=GRAULT
-
forward-env <NAME>...
defines the name(s) of one or multiple local environment variables to export on the host when running hooks.# deploy.conf forward-env FOO forward-env BAR BAZ QUX
If you have a .env
file in your local project directory, it will automatically be sourced.
This can be handy to create local variables that you can forward to the host.
# .env
export FOO=BAR
export YEAR=$(date "+%Y")
(Note that if you use the -C, --chdir
command line option or the $DEPLOY_CHDIR
variable,
the .env
file is read after the working directory is changed, so setting $DEPLOY_CHDIR
in the file has no effect.)
If you have an environment file named after the environment, it will also be sourced.
For example, when deploying in the "production" environment, a .env.production
file
would be sourced if present in the local project directory.
If both .env
and .env.production
are present, they are sourced in that order.
The properties in the configuration file can also be specified through command line options or environment variables.
For example, the repo
property in the configuration file can also be specified:
- With the
-r, --repo <url>
command line option. - With the
$DEPLOY_REPO
environment variable.
Their precedence is as follows:
- The command line option (e.g.
-r, --repo <url>
) always takes precedence if specified. - Next, the environment variable (e.g.
$DEPLOY_REPO
) takes precedence over the configuration file. - Otherwise, the value in the configuration file (e.g.
repo
) is used.
These options are valid for most sub-commands:
-
-C, --chdir <dir>
(or the$DEPLOY_CHDIR
variable) changes deploy's local working directory before loading the configuration file. -
--color always|never|auto
(or the$DEPLOY_COLOR
variable) enables/disables colors in the output of the script.This defaults to
auto
, which only enables colors if the current terminal is interactive. -
-c, --config <file>
(or the$DEPLOY_CONFIG
variable) allows you to set a custom path for the configuration file (defaults to./deploy.conf
). -
--help
prints usage information and exits. -
-v, -V, --version
prints the current version and exits. -
-y, --yes
(or the$DEPLOY_YES
variable) will automatically accept all confirmation prompts (only valid for therev
,cleanup
andupdate
sub-commands).Use with caution: old releases may be deleted with the
rev
orcleanup
sub-commands if akeep
option is configured.
These options configure where and how the project is deployed on the remote host:
-
-r, --repo <url>
(or the$DEPLOY_REPO
variable) sets the Git URL to fetch the project's source code from when deploying. -
-P, --path <dir>
sets the remote path to deploy the project to on the host (this is the path to the directory managed by deploy, not to the current release). -
-k, --keep <n>
(or the$DEPLOY_KEEP
variable) changes the number of old releases that are kept after successful deployment (see thekeep
option). -
-f, --force
(or the$DEPLOY_FORCE
variable) forces deployment to proceed even when you have uncommitted changes.
These options are valid for all commands which connect to the remote host through SSH
(setup
, rev
, console
, exec
, list
and cleanup
):
-
-A, --forward-agent
(or the$DEPLOY_FORWARD_AGENT
variable) enables forwarding of the authentication agent connection.Use with caution. Users with the ability to bypass file permissions on the remote host (for the agent's UNIX-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent.
-
-H, --host <address>
(or the$DEPLOY_HOST
variable) sets the host to connect to. -
-i, --identity <file>
(or the$DEPLOY_IDENTITY
variable) selects a file from which the identity (private key) for public key authentication is read.The default is
~/.ssh/id_dsa
,~/.ssh/id_ecdsa
,~/.ssh/id_ed25519
and~/.ssh/id_rsa
. -
-p, --port <n>
(or the$DEPLOY_PORT
variable) sets the port to connect to. -
-t, --tty
(or the$DEPLOY_TTY
variable) forces pseudo-terminal allocation.This can be used to execute arbitrary screen-based programs on a remote machine, which can be very useful, e.g. when implementing menu services.
-
-u, --user <name>
(or the$DEPLOY_USER
variable) sets the remote user to connect as.
These options can be used by deploy to update itself with the update
sub-command:
-
--update-path <file>
(or the$DEPLOY_UPDATE_PATH
variable) sets the path to save the updated script when using the update command.This overrides the
--update-prefix
or$DEPLOY_UPDATE_PREFIX
options. -
--update-prefix <dir>
(or the$DEPLOY_UPDATE_PREFIX
variable) sets the base directory when using the update command.The updated script will be saved in
bin/deploy
relative to this path. (Note that setting--update-path
or$DEPLOY_UPDATE_PATH
overrides this option.)
Note that all of deploy's command line options can be used anywhere in a command. The following 3 commands are equivalent:
deploy -u deploy production --keep 3 --yes master
deploy -u deploy --keep 3 --yes production master
deploy production master -u deploy --keep 3 --yes
Some options have a short form which may be specified with one hyphen,
e.g. -h
to display usage or -f
to force deployment with uncommitted local changes.
If a short option takes a value, it may be provided either as the next argument
(e.g. -k 5
) or with an equal sign (e.g. -k=5
).
All options have a long form which may be specified with two hyphens,
e.g. --help
or --force
.
If a long option takes a value, it may be provided either as the next argument
(e.g. --keep 5
) or with an equal sign (e.g. --keep=5
).
The following errors will cause the script to be interrupted and print an error:
-
Badly formatted or unknown options (e.g.
---foo
,-bar
,--baz
). -
Options that take no value specified with a value (e.g.
--help=3
). -
Options that take a value specified without their value.
-
Duplicate options (e.g. both
-k 5
and--keep 3
).
By default, deploy looks in the current directory to find its configuration file
and optional environment files. This can be customized by setting the -c, --chdir <dir>
command line option or the $DEPLOY_CHDIR
variable.
If this option is set, deploy will first move to that directory before sourcing the environment files and reading the configuration file.
deploy is usually used this way:
deploy prod setup
deploy prod list
deploy prod rev master
However, since the most often used sub-command is rev
,
it is also the default command. The following 2 commands are equivalent:
deploy prod master
deploy prod rev master
These are the commands you will use to set up and deploy your projects.
Note that most (but not all) of deploy's sub-commands require an environment to be specified before the actual command name, for example:
deploy setup # error! no environment
deploy production setup # all good
Perform first-time setup tasks on the host before deployment. You should only have to run this once before deploying the first time.
deploy production setup
deploy will create the following structure for you in the deployment directory defined by the path
config key (or the $DEPLOY_PATH
variable):
$DEPLOY_PATH/releases
will contain each deployment's files$DEPLOY_PATH/repo
will be a bare clone of your Git repository$DEPLOY_PATH/tmp
will be used to store temporary files during deployment
This phase has 3 hooks that are all executed in the deployment directory. Note that this directory must already exist on the host and be writable by the user you connect as.
This is what deploy will do during setup:
-
Run user-defined
pre-setup
hooks (if any). -
Create the
releases
and atmp
directories if they don't exist already. -
Clone the repository into the
repo
directory if it isn't already there. -
Run user-defined
post-setup
hooks (if any).
Deploy a new release from the latest changes in the Git repository.
You must provide a Git revision, i.e. a commit, branch or tag to deploy,
either as the [rev]
argument, with the rev
config key or the $DEPLOY_REV
variable.
If your Git repository is private, make sure that the deployment user has access to it.
For each deployment, a new release directory will be created in releases
in the deployment directory.
The name of a release directory is the current date and time in the YYYY-MM-DD-HH-MM-SS
format.
(You can list deployed releases with the list command.)
Note that deploy will refuse to deploy unless all your local changes are committed and pushed.
You can override this behavior with the -f|--force
option (or the $DEPLOY_FORCE
variable).
This is what deploy will do during deployment:
-
If a
keep
option is configured, all previously deployed releases are listed, with those to be deleted in red, and those to be kept and the current release in green, and confirmation is asked that they may be deleted if the deployment is successful. -
User-defined pre-deploy hooks (if any) are run in the directory of the previous release. You may define pre-deploy hooks to perform any task you might want to do before the actual deployment (e.g. you might want to stop the currently running version or put it into maintenance mode).
-
The latest changes are fetched from the repository.
-
The new release directory is created and the source is extracted at the specified revision into it.
-
User-defined deploy hooks (if any) are run in the new release directory. You should define deploy hooks to build your application or install its dependencies at this stage.
-
A symlink of the new release directory is created as
current
in the deployment directory. -
User-defined post-deploy hooks (if any) are run in the current release directory (which is now the same as the new release directory that was just created).
You should define post-deploy hooks to execute or start your application at this stage.
-
Old releases are deleted according to the
keep
option (if configured and any were found).
Print values from your deploy.conf
configuration file.
-
If called with no environment and no argument, it prints the whole configuration file:
$> deploy config [] user dev [production] user root port 222
-
If called with the environment and a key, it prints the last value of that key for that environment (only the last value is printed even for a multiple-value key):
$> deploy production config user root
Exits with status 1 if no value is found for the key.
Print all values of a config key in the current environment and its inherited environments (all values are printed even for a single-value key):
$> deploy production config-all user
dev
root
Exits with status 1 if no value is found for the key.
Print all values of a config section "as is" (with no inheritance). If no environment is specified, the default config section is printed.
Exits with status 1 if the config section is not found (including the default one).
Launch an interactive ssh session on the host.
-
If an absolute path is specified, the session starts there.
deploy production console /var/www
-
If no path is specified, the session starts in the deployment directory.
deploy production console
-
If a relative path is specified, it starts there relative to the deployment directory.
deploy production console current
Deletes old releases based on the keep
option.
The number of releases to keep is determined from the keep
configuration file option,
the $DEPLOY_KEEP
variable or the -k|--keep
command line option.
The releases to delete will be listed and confirmation will be asked before they are actually deleted.
Execute the specified command on the host.
deploy production exec ps -ef
List the releases that have been deployed on the host.
$> deploy production list
2016-12-24-17-45-23
2017-01-01-02-03-43
2017-04-01-00-00-00
Updates deploy to the latest version by downloading it from the Git repository and installing it at /usr/local/bin/deploy
(by default).
In addition to the basic requirements, this sub-command also requires chmod
, cp
and mktemp
to be available in the shell.
The installation path must be a writable file or not exist. If the installation path does not already exist, its parent must be a writable directory.
To perform the update, deploy will download its Git repository into a temporary directory that will be cleaned up when the update is done (or fails).
The correct revision of the script is then copied to the installation path and made executable.
deploy creates a new release directory for each deployment. By default, it keeps these directories forever, leaving you responsible for cleaning them up.
If given a keep
option, it can do it for you automatically after each deployment.
deploy will list the contents of the releases
directory on the host.
If no old release is found, deployment will proceed normally.
If the number of old releases is equal to or greater than the keep
number,
deploy will print the list of releases that will be deleted in red.
Old releases that will be kept will be shown in green.
During deployment, the current release being deployed is also indicated in green
(as it counts towards the keep
number).
If there are any releases to delete, deploy will ask for confirmation
(unless the -y, --yes
command line option or the $DEPLOY_YES
variable is specified).
Responding "no" interrupts the deployment and prints a help message explaining
how to configure the keep
number.
The actual cleanup of old releases is always performed after successful deployment.
A single rm
command is executed on the host through SSH,
deleting all appropriate old release directories in one go.