Continuous releases

[ben]

Looks like Travis has the capability of continuous deployment, which we could use to always be pushing fresh builds to GitHub releases. Some requirements:

• Only run on changes to master
• Tag as 2.1.NNNN (we don't care how big the numbers get, but they should be in the 2.1 series)
• Publish epub, mobi, and PDF binaries
• Update the git-scm site to provide download links (to the newest version)

[jnavila]

The thing is I don't have administrative rights on the repo, only push, even for the French translation. Do you have these rights? If so, could you share them at least for the French repo?

Also, as you mentionned, that would require tagging every time we would need a new release. What would be the policy?

[ben]

Permissions granted on progit2-fr.

I think just a monotonically-increasing patch number, that gets bumped every time a commit lands on master would work okay. I don't mind the versions going up to 2.1.173. I think Travis can create a tag on its own?

[jnavila]

Travis can only tag for npm, it seems. I don't know if we can make a hook on github to tag after each push. Another question is whether Github accepts that we link the files to download from git-scm.com. Yet another question is, what do we do with translations?

[ben]

Looks like the releases API lets you upload your own binaries. And if a direct link to an asset doesn't work when the referrer isn't github.com, maybe a link to the release itself would?

Translation repos could work the same way, managing their own binary releases. We'd need to update git-scm.com to link to them as well.

[jnavila]

@peff What do you think of automatic deployment?

[peff]

Automatic deployment sounds great. We should trust the translation teams to manage their repos and keep their master branch at something sane.

Right now all of the updates to git-scm.com happen through nightly scheduled jobs. It probably wouldn't be too hard to hit the releases API for each translation, checking for new links.

Direct-linking to uploaded assets should be fine, I think.

[jtcmedia]

Were you all in need of a Travis CI file? I'm already am using one on my fork https://github.com/jtcmedia/progit2 if this is of any use.

[ben]

I just pushed that, and it looks like Travis will build whenever we push a tag. So that takes care of the first and third bullet points above.

[jnavila]

I already have a fully automated build+tag+deploy on my personal repo.

https://github.com/jnavila/progit2-fr/blob/3af9c11cb8f04320f4a0f40a4d5c7bc8c9f00b14/.travis.yml

This version needs some more love:

• describe the setup for uploading the artefacts
• generalize the script (there are still some hard references to the repo)

This version needs more administration for the setup, which is one of my major concerns for translations.

I'll try it on progit-fr first (hopefully soon) but you can already try the tagging part.

[ben]

OK, I got this working. It involved updating .travis.yml, writing a script, and generating a new token for the GitHub API, but it works now. Here's a Travis build from a push to master which pushed the 2.1.2 tag, and here's the https://travis-ci.org/progit/progit2/builds/292755132 that published the release.

Going to close this out for now, since everything actionable in this repo is done. The one thing left is for git-scm.com's nightly build to look at the releases and pick the latest one to link to.

[jnavila]

Sorry to bother, but your version of tagging is clearly ligher than mine. Could you rapidly describe the steps, so that they can be reproduced in other repos?

[ben]

Sure. Here's where we compute the next patch number:

LASTPATCH=$(git describe --tags | cut -d- -f1 | cut -d. -f3)
PATCH=$(($LASTPATCH+1))

git describe --tags usually outputs something like 2.1.5-11-g608d7fa. We pipe this to cut, which splits it using hyphens and takes the first result (2.1.5 here), then we cut again, this time splitting by .s and grabbing the third one (5 here). The next line just increments by one.

Here's where we apply the tag:

curl -H "Authorization: token $GITHUB_KEY" \
     -X POST \
     -d "{\"ref\":\"refs/tags/2.1.$PATCH\", \"sha\":\"$TRAVIS_COMMIT\"}" \
     https://api.github.com/repos/progit/progit2/git/refs

This just uses the GitHub api for refs to create a tag. I created a new access token and did travis encrypt to generate the string you see in .travis.yml using GITHUB_KEY=<key> as the input.

Here's the JSON payload we send to GitHub:

{
  "ref": "refs/tags/2.1.6",
  "sha": "098963b50060aebaaf5091779cba78306d1c6e37"
}

Which creates a new tag. That triggers the webhook that Travis has installed, which then runs the build again on that commit and deploys the result to a release. Does that make sense?

[jtcmedia]

That's slick @ben. Thanks for posting!

[YueLinHo]

So... will be a lot of tag?

[ben]

Yes, this will unfortunately result in a lot of tags. I wonder what happens if we remove the tags once the deploy is done…

[ben]

I just manually deleted the 2.1.1 tag, and it turned that GitHub release into a "draft." I don't think that's what we want, so we'll just be accumulating tags. I'm okay with that trade off.

If someone else isn't, it would be possible to rewrite my tag_on_master script to upload built binaries and create a release without adding the tag. I don't have time to tinker with it just now.

[jnavila]

I think we can do something about tidying automatically with octokit

[taichunmin]

It is pretty easier to use GitLab to auto build the book.
I use GitLab CI + docker to build book, then push to GitLab Pages.
Doesn't require useless tags.

https://gitlab.com/taichunmin/progit2-zh-tw/

https://taichunmin.gitlab.io/progit2-zh-tw/