Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite server deploy script #1793

Merged
merged 11 commits into from
Oct 30, 2018
Merged

Rewrite server deploy script #1793

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
7ab6b5d
Ensure production secrets are not used in development
paulmelnikow Jul 22, 2018
3f865d2
Merge branch 'master' into deploy-server
paulmelnikow Jul 22, 2018
dd98aa4
Add new targets to phony
paulmelnikow Jul 22, 2018
2fd5b85
Merge branch 'master' into deploy-server
paulmelnikow Jul 25, 2018
617ba94
Add to prettierignore
paulmelnikow Jul 25, 2018
6eba3e2
Merge branch 'master' into deploy-server
paulmelnikow Oct 27, 2018
91919c5
Disable pre-commit hooks during deploy
paulmelnikow Oct 27, 2018
f51f37f
Merge branch 'master' into deploy-server
paulmelnikow Oct 29, 2018
07dc2b5
Use a protected branch
paulmelnikow Oct 29, 2018
5556ccf
A couple fixes
paulmelnikow Oct 30, 2018
046d69e
Merge branch 'master' into deploy-server
paulmelnikow Oct 30, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .prettierignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ package-lock.json
/build
/coverage
**/*.md
private/*.json
76 changes: 42 additions & 34 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
SHELL:=/bin/bash

DEPLOY_TEMP=${TMPDIR}shields-deploy
SERVER_TMP=${TMPDIR}shields-server-deploy
FRONTEND_TMP=${TMPDIR}shields-frontend-deploy

all: website favicon test

Expand All @@ -11,57 +12,64 @@ favicon:
website:
LONG_CACHE=false npm run build

# `website` is needed for the server deploys.
deploy: website deploy-s0 deploy-s1 deploy-s2 deploy-gh-pages deploy-gh-pages-clean
deploy: deploy-s0 deploy-s1 deploy-s2 clean-server-deploy deploy-gh-pages deploy-gh-pages-clean

deploy-s0:
deploy-s0: prepare-server-deploy push-s0
deploy-s1: prepare-server-deploy push-s1
deploy-s2: prepare-server-deploy push-s2

prepare-server-deploy: website
# Ship a copy of the front end to each server for debugging.
# https://github.com/badges/shields/issues/1220
git add -f Verdana.ttf private/secret.json build/
git commit -m'MUST NOT BE ON GITHUB'
git push -f s0 HEAD:master
git reset HEAD~1
git checkout master
rm -rf ${SERVER_TMP}
git worktree prune
git worktree add -B production ${SERVER_TMP}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of git worktree feels a bit like tempting the devil. The use of -B in particular risks losing information without realizing.

In general, I prefer having deployment procedures so dumb that it takes a lot for them to go wrong, and when they do, they are dumb enough that it is simple to fix them.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree that deployment procedures should be as easy to reason about as possible. If they fail, they should fail in very safe ways.

To that end, I'd like to avoid modifying the existing working tree during deployment. I'd also like to avoid branch switching to make sure the current ref gets deployed. In the current process, if two servers are deployed at once, I believe the second one will get master. I'd also like to deploy the exact same front-end build to each server. There's no need to build it again and it's one less thing to reason about.

When I was working on #1774 I was initially reluctant to use git worktree, mostly because I don't like adding new tooling that isn't necessary. The other alternative I experimented with was copying or re-cloning to an entirely separate working copy. This was messier and more brittle than using git worktree.

I'm not thrilled with using -B. I took it from the github pages deploy, which starts with a git checkout -B gh-pages.

Before my change, the deploy commits happen on a detached head. Nothing is deleted in the process, however it's deleted right afterward. Without a branch, it's difficult to inspect what was pushed to the server. I don't like deleting data. However, it is not surprising that a deploy process would clobber a branch named production. I'd rather reserve a branch for the production deploy work than do it all on a detached head that gets tossed right after it runs.

Would you be more comfortable using a name that was really explicit like server-deploy-working-branch?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1941 is another issue with the current scripts. git worktree has been working more reliably for deploy-gh-pages than the current deploy is for the servers.

cp -r build ${SERVER_TMP}
git -C ${SERVER_TMP} add build/
git -C ${SERVER_TMP} commit --no-verify -m '[DEPLOY] Add frontend for debugging'
cp private/secret-production.json ${SERVER_TMP}/private/secret.json
cp Verdana.ttf ${SERVER_TMP}
git -C ${SERVER_TMP} add private/secret.json Verdana.ttf
git -C ${SERVER_TMP} commit --no-verify -m '[DEPLOY] MUST NOT BE ON GITHUB'

deploy-s1:
git add -f Verdana.ttf private/secret.json build/
git commit -m'MUST NOT BE ON GITHUB'
git push -f s1 HEAD:master
git reset HEAD~1
git checkout master
clean-server-deploy:
rm -rf ${SERVER_TMP}
git worktree prune

deploy-s2:
git add -f Verdana.ttf private/secret.json build/
git commit -m'MUST NOT BE ON GITHUB'
git push -f s2 HEAD:master
git reset HEAD~1
git checkout master
push-s0:
git push -f s0 production:master

push-s1:
git push -f s1 production:master

push-s2:
git push -f s2 production:master

deploy-gh-pages:
rm -rf ${DEPLOY_TEMP}
rm -rf ${FRONTEND_TMP}
git worktree prune
LONG_CACHE=true \
BASE_URL=https://img.shields.io \
NEXT_ASSET_PREFIX=https://shields.io \
npm run build
git worktree add -B gh-pages ${DEPLOY_TEMP}
git -C ${DEPLOY_TEMP} ls-files | xargs git -C ${DEPLOY_TEMP} rm
git -C ${DEPLOY_TEMP} commit -m '[DEPLOY] Completely clean the index'
cp -r build/* ${DEPLOY_TEMP}
cp favicon.png ${DEPLOY_TEMP}
echo shields.io > ${DEPLOY_TEMP}/CNAME
touch ${DEPLOY_TEMP}/.nojekyll
git -C ${DEPLOY_TEMP} add .
git -C ${DEPLOY_TEMP} commit -m '[DEPLOY] Add built site'
git worktree add -B gh-pages ${FRONTEND_TMP}
git -C ${FRONTEND_TMP} ls-files | xargs git -C ${FRONTEND_TMP} rm
git -C ${FRONTEND_TMP} commit --no-verify -m '[DEPLOY] Completely clean the index'
cp -r build/* ${FRONTEND_TMP}
cp favicon.png ${FRONTEND_TMP}
echo shields.io > ${FRONTEND_TMP}/CNAME
touch ${FRONTEND_TMP}/.nojekyll
git -C ${FRONTEND_TMP} add .
git -C ${FRONTEND_TMP} commit --no-verify -m '[DEPLOY] Add built site'
git push -f origin gh-pages

deploy-gh-pages-clean:
rm -rf $DEPLOY_TEMP
rm -rf ${FRONTEND_TMP}
git worktree prune

deploy-heroku:
git add -f Verdana.ttf private/secret.json build/
git commit -m'MUST NOT BE ON GITHUB'
git commit --no-verify -m'MUST NOT BE ON GITHUB'
git push -f heroku HEAD:master
git reset HEAD~1
(git checkout -B gh-pages && \
Expand All @@ -72,4 +80,4 @@ deploy-heroku:
test:
npm test

.PHONY: all favicon website deploy deploy-s0 deploy-s1 deploy-s2 deploy-gh-pages deploy-heroku setup redis test
.PHONY: all favicon website deploy prepare-server-deploy clean-server-deploy deploy-s0 deploy-s1 deploy-s2 push-s0 push-s1 push-s2 deploy-gh-pages deploy-gh-pages-clean deploy-heroku setup redis test