Skip to content

Setup playbook #599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2,411 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2411 commits
Select commit Hold shift + click to select a range
3717b69
Added sift science public key.
Aug 25, 2016
d340103
Sorted!
Aug 25, 2016
ac9fd11
Merge pull request #540 from CodeNow/SAN-4860-add-sift-science
anandkumarpatel Aug 25, 2016
0b566e6
Added instances:cleanup task to Khronos.
Aug 26, 2016
557252e
remove log_level error from ENV's
Aug 29, 2016
320a60b
Merge pull request #542 from CodeNow/SAN-4866-cleanup-old-instances
Myztiq Aug 29, 2016
2212ecc
Add deploy songs for big-poppa and cream
thejsj Sep 1, 2016
aa9b4db
Merge pull request #544 from CodeNow/add-deploy-songs
Myztiq Sep 1, 2016
dd74688
Add NPM token and BIG_POPPA_HOST
thejsj Sep 2, 2016
a12d9e0
Merge pull request #545 from CodeNow/add-big-poppa-to-pheidi
thejsj Sep 6, 2016
e4b9bd6
Increased header size to avoid 502 gateway errors from certain user a…
Sep 7, 2016
768fe73
Merge pull request #546 from CodeNow/update-navi-proxy-to-handle-larg…
tosih Sep 7, 2016
871000e
0.1.1
Nathan219 Sep 7, 2016
c33bb6c
Merge remote-tracking branch 'origin/master'
Nathan219 Sep 7, 2016
505ecc6
add NewRelic vars for bigPoppa
Nathan219 Sep 12, 2016
0e54fea
revert this 0.1.1 that I accidentally did
Nathan219 Sep 12, 2016
3fd8efa
Merge pull request #547 from CodeNow/add-new-relic-to-big-poppa
anandkumarpatel Sep 12, 2016
21ce5c6
update khronos queues
podviaznikov Sep 13, 2016
0319319
fix
podviaznikov Sep 13, 2016
8ee8686
rename some tasks
podviaznikov Sep 13, 2016
0724758
Merge pull request #549 from CodeNow/rename-khronos-queues
podviaznikov Sep 13, 2016
5c0f4c9
old tags shall not pass
Sep 15, 2016
32f638e
Merge pull request #550 from CodeNow/halt-bad-deployes
und1sk0 Sep 15, 2016
9a7a9c5
Add pg client to big-poppa
thejsj Sep 20, 2016
c95508a
Adding version to psql client
thejsj Sep 20, 2016
eed2e74
Add version
thejsj Sep 20, 2016
044581c
Merge pull request #551 from CodeNow/SAN-5087-add-postgres-client-to-…
thejsj Sep 20, 2016
59f5e96
add new token by runnable-khronos user
podviaznikov Sep 23, 2016
87aeb94
Added Runnable-Test intercom API and APP key.
Sep 23, 2016
abd6d40
Namespace intercom app and id for Khronos.
Sep 24, 2016
e754e4d
Added base configuration for metabase on delta.
Sep 26, 2016
9646647
Put variables in correct file.
Sep 26, 2016
2c452c7
Fix roles to container_kill_start
Sep 26, 2016
ba7ba3f
Remove extra new-line.
Sep 26, 2016
df36f81
Removed all new-lines. Only used for the dockerfile_envs string.
Sep 26, 2016
dde39a4
Change port to remove clashes.
Sep 26, 2016
720b7df
Change pg_port.
Sep 26, 2016
eb9b96f
Lock down version of metabase.
Sep 26, 2016
39dc0d8
Merge pull request #553 from CodeNow/add-test-intercom-keys-to-gamma-…
tosih Sep 26, 2016
4db19b2
Merge pull request #554 from CodeNow/add-metabase-deployment
tosih Sep 26, 2016
1d5ec35
add arithmancy hosts
Sep 27, 2016
496be30
Merge pull request #555 from CodeNow/take-arithmancy
anandkumarpatel Sep 27, 2016
040ffdb
Update variables
podviaznikov Sep 29, 2016
1a055c8
Merge pull request #552 from CodeNow/new-token
podviaznikov Sep 29, 2016
e2a54b6
add gamma vars
Sep 30, 2016
d76cd7a
Merge pull request #556 from CodeNow/add-posgre-to-arithmancy
anandkumarpatel Sep 30, 2016
c4e4b2f
Forgot to add this on the earlier PR.
Sep 30, 2016
f9e9db8
Merge pull request #557 from CodeNow/add-metabase-to-delta-children
anandkumarpatel Sep 30, 2016
2d2a862
add postgres for arithmancy to delta
Oct 5, 2016
02d330a
Merge pull request #558 from CodeNow/arithmancy-delta-postgres
anandkumarpatel Oct 5, 2016
46283d1
created brand new token for the canaries
Nathan219 Oct 7, 2016
5695019
Merge pull request #559 from CodeNow/fix-canary-token
anandkumarpatel Oct 7, 2016
efc87c6
update metabase home
Oct 7, 2016
6ef8997
Merge pull request #560 from CodeNow/give-metabase-a-new-home
anandkumarpatel Oct 7, 2016
9bcfe07
Update README.md
podviaznikov Oct 10, 2016
972c483
Add stripe keys to BP for migrations
thejsj Oct 12, 2016
8f5d7fd
Merge pull request #561 from CodeNow/add-stripe-token-to-big-poppa
thejsj Oct 17, 2016
056028c
new api worker
Oct 17, 2016
4d55e9b
Merge pull request #562 from CodeNow/new_gamma_api_worker
tosih Oct 17, 2016
1a4720f
add big-poppa host
podviaznikov Oct 21, 2016
ce72a20
update khronos.yml
podviaznikov Oct 21, 2016
b0afd50
add npm registry
podviaznikov Oct 21, 2016
dd0c598
Merge pull request #563 from CodeNow/add-big-poppa-to-khronos
podviaznikov Oct 21, 2016
956b256
swarm hovers over 1gb, make it 2gb
Oct 27, 2016
3bb005f
Change khronos version to v4.4.7
thejsj Oct 31, 2016
c71ea86
Remove LTS from everywhere
thejsj Oct 31, 2016
421fd90
bump swarm memory limit
anandkumarpatel Oct 31, 2016
3e00654
Merge pull request #566 from CodeNow/bump-swarm-memory
Myztiq Oct 31, 2016
d66ea62
Change to 4..47
thejsj Nov 2, 2016
60498fb
Merge branch 'master' into fix-node-version-for-khronos
thejsj Nov 2, 2016
3507945
Merge pull request #565 from CodeNow/fix-node-version-for-khronos
thejsj Nov 2, 2016
7a64260
add prom to gamma
Nov 4, 2016
98fa493
fix nits
Nov 4, 2016
b371854
Merge pull request #567 from CodeNow/SAN-5333-add-prom-deplo
anandkumarpatel Nov 4, 2016
8bfc569
add cadvisor and node-exporter
Nov 5, 2016
6520a1d
add 10g limit
Nov 5, 2016
3c7fd85
Roll Stripe API key
thejsj Nov 7, 2016
844ce9a
Merge pull request #570 from CodeNow/roll-stripe-keys
anandkumarpatel Nov 7, 2016
fc38814
add delta-deploy aws key
Nov 7, 2016
23e1673
add ssh key rotation scripts
Nov 8, 2016
b4ed5dc
Merge branch 'master' into SAN-5333-add-prom-deplo
Nov 8, 2016
cc77ca1
add hostIp tag, and alerts
Nov 9, 2016
c61b5ca
add hostIp tag, and alerts
Nov 9, 2016
1ea68ee
move pagerduty key, remove test alert
Nov 10, 2016
bf0ab92
fix remove
Nov 10, 2016
a791d0f
Update hosts
anandkumarpatel Nov 10, 2016
145c79a
fix remove
Nov 10, 2016
c6f0cfa
Merge branch 'SAN-5333-add-prom-deplo' of https://github.com/CodeNow/…
Nov 10, 2016
58ae529
fix text
Nov 10, 2016
bae3432
Merge pull request #571 from CodeNow/SAN-5333-add-prom-deplo
anandkumarpatel Nov 10, 2016
9efc3e8
Added timekeeping event emmitter to cron tab for Khronos.
Nov 4, 2016
9a45e3a
Add time prefix to timekeeper events. Fix naming of older queues tasks.
Nov 5, 2016
21d1b5f
Remove old and invalid comments.
Nov 8, 2016
689e2e3
update prom
Nov 10, 2016
b3d828c
inc intervals
Nov 10, 2016
02c0147
bump to 20g
Nov 11, 2016
e7eafe5
Merge branch 'master' into add-khronos-timekeeping-events
tosih Nov 11, 2016
1101f13
New plays for dock.yml
Nov 11, 2016
b1e6ee6
NOT MY PRESIDENT
Nov 12, 2016
dd3e9d0
Let amazon start its own service
Nov 12, 2016
77109e8
Event naming in same tense.
Nov 15, 2016
2b3bda6
Edited main.yml
henrymollman Nov 17, 2016
7d3f4bf
Update README.md
anandkumarpatel Nov 17, 2016
adba366
Merge pull request #568 from CodeNow/add-khronos-timekeeping-events
tosih Nov 17, 2016
ed608bb
update ssl certs
Nov 21, 2016
ea0b10f
Merge remote-tracking branch 'origin/master' into new-dock-ami
Nov 22, 2016
4049e94
Merge remote-tracking branch 'origin/new-dock-ami' into new-dock-ami
Nov 22, 2016
cf89697
Updated to include weave containers
Nov 23, 2016
e49c10c
remove epsilon hosts
Nov 23, 2016
4af55ff
Added some demo repo images for speed
Nov 23, 2016
c16cd6e
Use host var for registry ip address
Nov 23, 2016
33a15f9
Cleanup docker pull playbook
Nov 23, 2016
1711e86
update palantiri token
Nov 23, 2016
a644b5c
New tags for dock-init
Nov 23, 2016
d3b4f04
Added registry host
Nov 28, 2016
4060d06
Merge pull request #573 from CodeNow/new-dock-ami
henrymollman Nov 29, 2016
cdc9de9
update prom config
Nov 30, 2016
91b5439
Added github varnish connection information and intercom admin id to …
Dec 1, 2016
ea0cae6
Merge pull request #574 from CodeNow/add-github-varnish-and-intercom-…
tosih Dec 1, 2016
6cd79d9
Added new read/write intercom keys.
Dec 1, 2016
2e41e2d
Merge pull request #575 from CodeNow/full-access-keys-for-pheidi
tosih Dec 1, 2016
704837b
Add mixpanel proxy
thejsj Dec 6, 2016
6d1d767
Add mixpanel proxy
thejsj Dec 6, 2016
e1561dd
Add ENV for MIXPANEL_PROXY_URL
thejsj Dec 6, 2016
4467099
Change tag name
thejsj Dec 6, 2016
7064468
Merge pull request #576 from CodeNow/SAN-5498-mixpanel-proxy
thejsj Dec 6, 2016
48b3092
Increased client_max_body_size to fix S3 uploads.
Dec 6, 2016
28c4978
Undo navi change.
Dec 7, 2016
ff5f813
Merge pull request #577 from CodeNow/bigger-client-max-body
anandkumarpatel Dec 7, 2016
27a346b
yo
thejsj Dec 7, 2016
9df98fb
Make nginx changes for mixpanel in order to handle ssl
thejsj Dec 7, 2016
2a1350a
Merge pull request #578 from CodeNow/SAN-5498-mixpanel-proxy
tosih Dec 7, 2016
da2a90a
Change URL
thejsj Dec 7, 2016
468b5b0
Merge pull request #579 from CodeNow/change-mixpanel-proxy-url
anandkumarpatel Dec 8, 2016
5e7ec6a
Chnage 6pm PDT time to 2am PDT time so that p4l orgs don't get deleted
thejsj Dec 9, 2016
da7c58f
Merge pull request #580 from CodeNow/GIVE_US_BACK_OUR_P4L_ORGS
thejsj Dec 9, 2016
fcc9797
Add git branch as ENV
thejsj Dec 13, 2016
467a0d2
Merge pull request #581 from CodeNow/add-branch-as-env
thejsj Dec 14, 2016
e06dc66
Remove epsilon. Remove reviewers
thejsj Dec 14, 2016
f5ee4bc
Merge branch 'master' into fix-pull-request
thejsj Dec 14, 2016
234265e
Merge pull request #582 from CodeNow/fix-pull-request
anandkumarpatel Dec 14, 2016
713c1d4
Add deploy song for pheidi
tosih Dec 16, 2016
9e5b378
bump api to use npm 4.0.3 (#583)
anandkumarpatel Dec 19, 2016
4e6f5b8
add shrinkwrap for api (#584)
anandkumarpatel Dec 19, 2016
b2e4ee6
San 5334 add alerts to prom (#585)
anandkumarpatel Dec 20, 2016
f5422f2
Increase container build timeout to 1 hour from 30 mins.
Dec 20, 2016
f65d1bb
Merge pull request #587 from CodeNow/increase-api-build-timeout
tosih Dec 20, 2016
34a88e1
in comes Deployer (#586)
anandkumarpatel Dec 20, 2016
95d50c2
bump version
Dec 20, 2016
fbd9624
opps
Dec 20, 2016
2072a69
version 2.1.0
Dec 20, 2016
f42a530
hotfix add delta
Dec 20, 2016
47fccdb
fix location
Dec 20, 2016
53c8624
add env to label, bump mem aletings, update alert text (#589)
anandkumarpatel Dec 21, 2016
bf13dff
San 5331 docks iptables (#569)
und1sk0 Jan 11, 2017
97c5b1e
remove bad deps (#590)
anandkumarpatel Jan 11, 2017
c451707
Updates for bear-clone
thejsj Jan 17, 2017
bc1a6f5
Fix psad alerting (#591)
anandkumarpatel Jan 17, 2017
474a412
Merge branch 'master' of github.com:CodeNow/devops-scripts into setup…
thejsj Jan 17, 2017
f4f7ce9
Shiva fixes
thejsj Jan 18, 2017
0d7a700
Fix stuff
thejsj Jan 20, 2017
b864473
Fix vault tokens. Add nginix entry for registry
thejsj Jan 24, 2017
666d5a9
Added github enterprise config
Jan 24, 2017
7f0083f
remove varnish host (#593)
anandkumarpatel Jan 24, 2017
7b95ef9
Added support for github enterprise deploy
Jan 25, 2017
03a0525
Fixed typo
Jan 25, 2017
cf4b2cf
Merge pull request #594 from CodeNow/SAN-5662-gh-varnish
Myztiq Jan 25, 2017
7ae53ce
Added varnish theme song
Jan 25, 2017
498cbc2
Fix docks/shiva stuff
thejsj Jan 25, 2017
fb2c8fa
remove datadog agent for docks (#595)
anandkumarpatel Jan 25, 2017
3263e5e
Added egret and env vars
Jan 26, 2017
5daf273
Changed names to agreeable-egret
Jan 26, 2017
7c9f0fd
PR comments
Jan 26, 2017
83b9959
Fix consul-template and port mapping
thejsj Jan 27, 2017
853bd6a
Merge branch 'master' of github.com:CodeNow/devops-scripts into setup…
thejsj Jan 27, 2017
ab4152c
San 5685 add demo docker images (#596)
tosih Jan 27, 2017
23db873
Merge branch 'master' of github.com:CodeNow/devops-scripts into setup…
thejsj Jan 27, 2017
a9a0ba4
Added user content domain var
Jan 27, 2017
67af2a7
Fixing vault issues and more port issues
thejsj Jan 28, 2017
b11fa1c
W/e
thejsj Jan 28, 2017
a0eaca7
Added port options
Jan 30, 2017
c107c96
Fix missing variables. Fix handling missing intercom/rollbar keys
thejsj Jan 30, 2017
de54f3a
Change node version
Jan 31, 2017
51663ad
Fix container_kill_start role to also use names
thejsj Feb 1, 2017
cc9e202
Fix vault ports
thejsj Feb 1, 2017
cf6722b
Add proxy for NAVI and socket server and make sure it all works
thejsj Feb 1, 2017
a2737c9
Fix redis TLS
thejsj Feb 1, 2017
b590f7d
Include docker-listener
thejsj Feb 1, 2017
a7d2406
Fix proxy stuff for navi, drake, and api
thejsj Feb 1, 2017
0ff00fd
Uncomment stuff in single host
thejsj Feb 1, 2017
4825438
Updated logic for bear-clone
Feb 2, 2017
1ea6ff6
Add >65000 port for deployment
Feb 2, 2017
080d7bc
Container Kill Start
Feb 2, 2017
deca8ff
Added flag for API, changed charon API token to not be hardcoded. Fix…
Feb 3, 2017
c8f2eb1
bump image builder (#602)
anandkumarpatel Feb 3, 2017
f02e85b
Cleanup
thejsj Feb 3, 2017
5adff25
Revert terraform changes
thejsj Feb 3, 2017
7e66a6f
Merge branch 'setup-playbook' of github.com:CodeNow/devops-scripts in…
thejsj Feb 3, 2017
7031632
Merge branch 'master' into setup-playbook
thejsj Feb 3, 2017
03af1eb
Updated access token
Feb 3, 2017
07f4453
Merge remote-tracking branch 'origin/setup-playbook' into setup-playbook
Feb 3, 2017
2eea474
Link no longer has a hardcoded token.
Feb 3, 2017
5479329
Fix osx removal of deamon (#592)
anandkumarpatel Feb 4, 2017
75c11e9
Added github host!
Feb 4, 2017
cc87ba3
Added postgres connect strings
Feb 6, 2017
c6f9fdd
PR comments
Feb 6, 2017
6c2a981
Remove '-'
Feb 6, 2017
eeb3e54
Added github domain to build arg
Feb 6, 2017
1a8e760
Fix typo
Feb 6, 2017
c86845a
Added image builder version env passing
Feb 6, 2017
e236b60
Added hello runnable github id.
Feb 6, 2017
76e6749
Tags
Feb 6, 2017
73bab51
Merge pull request #597 from CodeNow/add-agreeable-egret
henrymollman Feb 6, 2017
6e083ac
Added pheidi runnabot username + token to bear-clone-hosts
Feb 6, 2017
1a2f389
Added github_username for runnabot
Feb 6, 2017
70df764
Added varnish info to pheidi.
Feb 7, 2017
e85ce5b
Remove docks.js file. Remove commented variables
thejsj Feb 9, 2017
07f638e
Merge branch 'setup-playbook' of github.com:CodeNow/devops-scripts in…
thejsj Feb 9, 2017
817fb92
Remove undefined variable
thejsj Feb 9, 2017
c800512
Fix ENVs in api. Create ENV in diff environments
thejsj Feb 10, 2017
bbf62a0
Remove unnecessary defaults in big poppa
thejsj Feb 10, 2017
b7a0de2
Undo binding consul to localhost
thejsj Feb 10, 2017
7529498
Remove echo from findTagRunning
thejsj Feb 10, 2017
51ab574
Remove unnecssary file
thejsj Feb 10, 2017
15663a3
Remove comments
thejsj Feb 10, 2017
4f8a517
Remove old certs
thejsj Feb 10, 2017
394c428
Remove new_container_ids check
thejsj Feb 10, 2017
b8b3f86
Clean up single-host-proxy.yml
thejsj Feb 10, 2017
1f6ed9d
Remove comments
thejsj Feb 10, 2017
abf56a9
Change to ENV
thejsj Feb 10, 2017
e5e6e6e
Bug: Add kill_container_name for consul-template
thejsj Feb 8, 2017
5f107fc
Remove unnecessary registering of var
thejsj Feb 10, 2017
26c2494
Re-add secret file
thejsj Feb 10, 2017
af84a5f
Remove redis TLS
thejsj Feb 10, 2017
5307dce
Remove unnecessary vars
thejsj Feb 10, 2017
da64a4d
Remove redis-tls
thejsj Feb 10, 2017
38313a6
Fix ports
thejsj Feb 10, 2017
c3eef01
Fix var to environment_name
thejsj Feb 10, 2017
4955cdf
Fixed issues with github domain
Feb 10, 2017
3a02dcf
Merge remote-tracking branch 'origin/setup-playbook' into setup-playbook
Feb 10, 2017
a1e5efd
change version (#604)
podviaznikov Feb 24, 2017
cfa5cee
remove epsilon (#605)
anandkumarpatel Feb 27, 2017
e6450f4
Add bucket region for web
thejsj Mar 1, 2017
662dcd8
Merge pull request #606 from CodeNow/add-aws-region-to-fe
Myztiq Mar 1, 2017
945f255
Updated deploy bucket for marketing.
Mar 2, 2017
6bbbeaf
Added marketing deploy keys
Mar 3, 2017
f821c8e
Merge pull request #607 from CodeNow/updated-deploy-bucket
Myztiq Mar 3, 2017
4708bc3
Change region name
thejsj Mar 4, 2017
e9f6737
Merge pull request #609 from CodeNow/change-region-name
thejsj Mar 7, 2017
e73cc13
Merge branch 'master' of github.com:CodeNow/devops-scripts into setup…
thejsj Mar 7, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
18 changes: 18 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
node_modules
*.pem
pass
.pass
hellorunnable
dump.rdb
erl_crash.dump
npm-debug.log
ca.srl
.DS_Store
ansible/roles/hipache/templates/runnable*
ansible/certs/*
*.retry
*.tfstate*
terraform/credentials.tfvars
terraform/.build
ansible/secrets/*
.idea
38 changes: 38 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
FROM ubuntu:14.04

RUN apt-get update -y
RUN DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y -q unzip build-essential python-pip python-dev python-yaml libxml2-dev libxslt1-dev zlib1g-dev git curl sshpass openssh-client
RUN pip install --upgrade pyyaml jinja2 pycrypto

RUN curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash - && \
apt-get install -y nodejs

RUN curl -O https://releases.hashicorp.com/vault/0.6.3/vault_0.6.3_linux_amd64.zip && \
unzip ./vault_0.6.3_linux_amd64.zip -d /bin && \
chmod +x /bin/vault

RUN git clone git://github.com/ansible/ansible.git --recursive /opt/ansible

RUN cd /opt/ansible && \
git checkout v2.1.3.0-1 && \
git submodule update --init --recursive && \
bash -c 'source ./hacking/env-setup'

ENV PATH /opt/ansible/bin:$PATH
ENV PYTHONPATH /opt/ansible/lib:$PYTHONPATH
ENV MANPATH /opt/ansible/docs/man:$MANPATH

ADD ./ssh /root/.ssh
RUN echo 'eval `ssh-agent`' >> /root/start.sh
RUN echo 'ssh-add /root/.ssh/id_rsa' >> /root/start.sh
RUN echo 'npm start' >> /root/start.sh
RUN chmod +x /root/start.sh

ADD ./ansible/ /ansible
RUN cd /ansible && npm install

ADD ./deployer/ /deployer
RUN cd /deployer && npm install

WORKDIR /deployer
CMD /root/start.sh
21 changes: 21 additions & 0 deletions PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[//]: # (Let's get your best description here about what's happend! Here's a list as well, if you like:)

* I removed this function
* I fixed all these things, etc.

#### Dependencies

- [ ] list dependencies (eg, PR from another branch or repo; tags or versions required prior to deployment)

#### Tests

> Test any modifications on one of our environments.

- [ ] tested on _environment_ by _someone_

#### Deployment (post-merge)

> Ensure that all environments have the given changes.

- [ ] deployed to gamma
- [ ] deployed to delta
143 changes: 142 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,145 @@
devops-scripts
==============

devops-scripts
Scripts for managing our deployments.

# How to Deploy at Runnable
## Setup

Before you can deploy you'll need to install the appropriate tools, scripts, and keys on your local machine.
To do so, execute the following steps:

1. Install Ansible v2.1.0.0 (the deploy automation tool we use to deploy projects to production)
http://docs.ansible.com/intro_installation.html

2. Get the latest devops-scripts (the recipes that we use to deploy various projects)
https://github.com/CodeNow/devops-scripts

3. Change to the devops scripts repo directory and run the following command:
`ln -s /<local-path-to-devops-scripts>/ssh/config ~/.ssh/config`

4. Obtain the “Keys of Power” from someone who can already deploy (ask Anand if you don’t know). Depending on what you want to deploy you'll receive `*.pem` files.

5. Move the “Keys of Power” .pem files to your `~/.ssh` directory

6. Install two required tools onto your machine:
```bash
brew update && brew install vault daemon
```

At this point you should be capable of deploying; keep reading to find out how to actually perform a deploy!

## Deploying Services
- **IMPORTANT:** always pull latest devopts-scripts (`git pull origin master`)
- **IMPORTANT:** Before you deploy a new version of any project make sure to determine which version of the project is currently deployed. This way you can quickly revert to the last stable release if something goes wrong after pushing a new version.

### Step 1: Determine the Current Deploy Version
To determine the latest deploy tag for a project please check the project's repository on
github and look for the latest release tag (should be in the form `vX.Y.Z`). Once you've located the tag,
copy it down somewhere that is easily and quickly accessible (you may need to use it quickly if something goes wrong).

### Step 2: Deploy the Project via `ansible-playbook`

- **WARNING:** If you were unable to determine the last deploy tag for a project and cannot revert **STOP**.
Ask someone on the team for help before continuing.
- **IMPORTANT:** All commands should be run from the `devops-script/ansible` directory.

#### Ansible Vault

Please note that there are playbook that require encrypted [ansible vault](http://docs.ansible.com/ansible/playbooks_vault.html) files. If you see the following error:

```bash
ERROR: A vault password must be specified to decrypt # snip
```

you will need to re-run the playbook with:

```bash
--ask-vault-pass
```

#### Latest Tag
Build and deploy a service to the latest tag of its repository. This will build
the docker image needed to run the container on our infrastructure.

#### Branch or Tag
Build and deploy a service to a specific branch or tag on its repository. This performs a build
of the docker image needed to run the service on our architecture.

##### Command
```
ansible-playbook -i ./[inventory_dir] [service-playbook] -e git_branch=[branch-or-tag] -t deploy
```

##### Arguments
- `[inventory_dir]` - The environment inventory files (servers and variables). Should be one of the following:
- `stage-hosts` - Runnable sandbox staging environment services
- `gamma-hosts` - Gamma services (internal use only; production mirror)
- `delta-hosts` - Delta services (real production)
- `[service-playbook]` - The playbook for the service you wish to deploy, ex:
- `api.yml` - Deploys both the api and the api-workers services
- `shiva.yml` - Deploys the shiva micro-service
- `charon.yml` - Deploys a specific version of charon DNS to all docks
- `[branch-or-tag]` - The branch or tag you wish to deploy, ex:
- `-e git_branch=v1.9.9` (version tag)
- `-e git_branch=my-feature-branch` (branch)
- `-e git_branch=3928745892364578623` (specific commit)

##### Rebuild and Deploy Tag or Branch (No Cache)
Forces a rebuild of a docker image for the given service at the given branch or tag and then deploys the
newly created image. This is useful when a previously deployed branch has new changes that need to
be deployed to an environment.

Generally this command is only used with `gamma-hosts/` as it is often used to update code
being tested in the production mirror.

##### Command
```
ansible-playbook -i ./[inventory_dir] [service-playbook] -e git_branch=[branch-or-tag] -e build_args=--no-cache
```

##### Arguments
- `[inventory_dir]` - The environment inventory files (servers and variables).
- `[service-playbook]` - The playbook for the service you wish to deploy.
- `[branch-or-tag]` - The branch or tag you wish to deploy.


## Reverting
If, for some reason, the new deploy is not operating as expected you can quickly revert by referencing the tag you collected in Step 1.
Simply run the appropriate deploy command in the previous section with the last release tag and the new deploy will be reverted.

## Deploy Songs

- **IMPORTANT:** Make sure to play the song loud and proud when deploying!

It is the custom at Runnable to play a song to the entire team when deploying. For each of the repositories here are the respective songs:

| Service | Deploy Song Link |
| ------- | ---------------- |
| api / api-workers | [Push it - Rick Ross](https://www.youtube.com/watch?v=qk2jeE1LOn8) |
| arithmancy | [onerepublic - Counting Stars](https://www.youtube.com/watch?v=hT_nvWreIhg) |
| big poppa | [Big Poppa - The Notorious B.I.G.](https://www.youtube.com/watch?v=phaJXp_zMYM) |
| charon | [Enter Sandman - Metallica](https://www.youtube.com/watch?v=CD-E-LDc384) |
| cream | [C.R.E.A.M. - Wu-Tang Clan](https://www.youtube.com/watch?v=PBwAxmrE194) |
| deployer | [Roll our](https://www.youtube.com/watch?v=t21DFnu00Dc) |
| detention | [Unbreakable Kimmy Schmidt](https://youtu.be/CV9xF8CjhJk?t=21s) |
| docker-listener | [Call Me Maybe - Carly Rae Jepsen](https://www.youtube.com/watch?v=fWNaR-rxAic) |
| drake | [Drake - Hotline Bling](https://www.youtube.com/watch?v=uxpDa-c-4Mc)
| filibuster | [He's a Pirate - Pirates Of The Caribbean](https://www.youtube.com/watch?v=yRh-dzrI4Z4) |
| Full Stack Deploy (`all.yml`) | [The Cleveland Orchestra (George Szell conducting) Ludwig von Beethoven Symphony No. 9 "Chorale (Ode To Joy)" Opus 125 IV.] (https://www.youtube.com/watch?v=4g5770gaais) |
| github-proxy | [Proxy - Martin Garrix](https://www.youtube.com/watch?v=NWB6-PJw4Mk) |
| khronos | [Time After Time - Cyndi Lauper](https://www.youtube.com/watch?v=VdQY7BusJNU) |
| krain | [Men at Work - Down Under](https://www.youtube.com/watch?v=XfR9iY5y94s) |
| link | [Zelda Main Theme Song](https://www.youtube.com/watch?v=cGufy1PAeTU) |
| mavis | [Fairy Tail theme song](https://www.youtube.com/watch?v=R4UFCTMrV-o) |
| navi | [Ocarina of Time: Lost Woods The Legend of Zelda](https://www.youtube.com/watch?v=iOGpdGEEcJM) |
| optimus | [Original Transformers Opening Theme](https://www.youtube.com/watch?v=nLS2N9mHWaw) |
| pheidi | [Chariots of Fire Theme](https://www.youtube.com/watch?v=CSav51fVlKU) |
| runnable-angular | [Push it to the limit - Scarface](https://www.youtube.com/watch?v=9D-QD_HIfjA) |
| sauron | [Sauron theme song from LOTR](https://www.youtube.com/watch?v=V_rk9VBrXMY) |
| Security Groups | [Out of the Woods - Tayor Swift](https://www.youtube.com/watch?v=JLf9q36UsBk)
| shiva | [FFXIV Shiva Theme](https://www.youtube.com/watch?v=noJiH8HLZw4) |
| swarm-deamon | [Pink Floyd - Another Brick In The Wall](https://www.youtube.com/watch?v=5IpYOF4Hi6Q) |
| swarm-manager | [Eric Prydz VS Pink Floyd - 'Proper Education'](https://www.youtube.com/watch?v=IttkDYE33aU) |
| varnish | [Karate Kid Theme Song](https://www.youtube.com/watch?v=VIYqtkdMxQg) |
| vault / vault-values | [Seal - Kiss From A Rose](https://www.youtube.com/watch?v=zP3so2hY4CM) |
2 changes: 2 additions & 0 deletions ansible/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
_cache
*.pyc
15 changes: 15 additions & 0 deletions ansible/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@

Ansible provides a framework for our administration and deployment. It requires an organization for scripts and variables. By design it uses SSH to connect to all hosts before it executes the actions. As such it can be run from any machine. All Ansible provided functionality is idempotent and it strongly encourage custom scripts match that standard.

Here is the organization of the files in `devops-scripts/ansible`

* `*-hosts` - Files naming all the servers
* `*.yml` - The top level ansible actions. These files describe how a host has vars and roles executed on it.
* `/group_vars` - yml files that define variables and values for your ansible scripts. This mostly maps one to one with machine types in AWS. They’re a key value map.
* `/library` - Third party libraries and scripts.
* `/roles` - A set of folders containing the ansible roles. A role defines the executable actions by ansible. The center pieces is the `/tasks/main.yml`. It defines name actions and requirements.
The role can have several sub folders.
* `/handlers` - ???
* `/defaults` - ???
* `/meta` - contains dependencies
* `/template` - templates for any files that need to be generate and delivered.
11 changes: 11 additions & 0 deletions ansible/agreeable-egret.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
- hosts: redis
- hosts: agreeable-egret
vars_files:
- group_vars/alpha-agreeable-egret.yml
roles:
- role: notify
tags: [ notify ]
- role: builder
tags: [ build ]
- role: container_kill_start
3 changes: 3 additions & 0 deletions ansible/ansible.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[defaults]
# Required so `sudo: yes` does not lose the environment variables, which hold the ssh-agent socket
sudo_flags = -HE
34 changes: 34 additions & 0 deletions ansible/api-core.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
- hosts: mongodb
- hosts: navi
- hosts: charon
- hosts: rabbitmq
- hosts: redis
- hosts: big-poppa
- hosts: cream
- hosts: swarm-manager
- hosts: consul

- hosts: api
vars_files:
- group_vars/alpha-api-base.yml
- group_vars/alpha-api.yml
roles:
- role: notify
rollbar_token: "{{ api_rollbar_key }}"
tags: [ notify ]

- role: builder
tags: [ build ]

- role: docker_client

- role: tls-client
tags: [ tls ]
tls_service: mongodb

- role: datadog
tags: [ datadog ]

- role: container_start
number_of_containers: "{{ api_number_of_containers | default(ansible_processor_cores) }}"
3 changes: 3 additions & 0 deletions ansible/api.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
- include: api-core.yml
- include: socket-server.yml
- include: workers.yml
6 changes: 6 additions & 0 deletions ansible/app-services.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
- include: detention.yml git_branch="{{ detention_branch }}"
- include: drake.yml git_branch="{{ drake_branch }}"
- include: eru.yml git_branch="{{ eru_branch }}"
- include: metis.yml git_branch="{{ astral_branch }}"
- include: shiva.yml git_branch="{{ astral_branch }}"
# run with: `--extra-vars "@current_versions.yml"`
13 changes: 13 additions & 0 deletions ansible/arithmancy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
- hosts: rabbitmq
- hosts: consul

- hosts: arithmancy
vars_files:
- group_vars/alpha-arithmancy.yml
roles:
- role: notify
rollbar_token: "{{ arithmancy_rollbar_token }}"
tags: [ notify ]
- { role: builder, tags: [build] }
- { role: container_kill_start }
23 changes: 23 additions & 0 deletions ansible/base.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
- hosts: localhost
connection: local
tasks:
- fail: msg="`host` (target host) needs to be defined to run this role"
when: host is not defined

- add_host:
name={{ host }}
groups=dock

- hosts: "{{ host }}"
roles:
- { role: apt_update }
- { role: package-dock, tags: [ dock, package ] }
- { role: package-aws, tags: [ dock, package ] }
- { role: package_ntp }
- { role: build_essential }
- { role: docker, tags: [ docker ] }
- { role: datadog, tags: [ datadog ] }
- { role: ulimits, tags: [ ulimits ] }
- { role: loggly, tags: [ loggly, clean ] }
- { role: node }
11 changes: 11 additions & 0 deletions ansible/bastion.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
- hosts: consul

- hosts: bastion
roles:
- role: notify
tags: [ notify ],
app_name: bastion_sshd,
git_branch: latest,
name: bastion_sshd
- { role: bastion_sshd, tags: bastion-sshd }
Loading