Publish checksums for every artifact

[lidel]

This PR aims to close #152 by exposing CID and additionally generating a plain SHA512 hash for every artifact of new releases.

Rationale

I had a short chat with someone (sorry, forgot the name :() about things that we could improve as an open source project to make it easier to package our software.

Number one issue was that we do not publish checksums and package managers can't verify downloads without additional orchestration. This makes package maintainer's work harder than it should be.

I mentioned all our artifacts are content-addressed and it is possible to get CID for every file but there is a good argument about learning curve of IPFS concepts and multihash/cids not being supported by default userland tools in unix-like systems (yet), while things like shasum being usually around.

Details

This PR exposes two new values in two places:

• in file hierarchy, as additional files with self-describing extension:
  • releases/<dist>/<version>/<dist>_<version>_<platform>.tar.gz.cid
  • releases/<dist>/<version>/<dist>_<version>_<platform>.tar.gz.sha512
• in releases/<dist>/<version>/dist.js as additional fields named cid and sha512:

  "amd64": {
            "link": "/go-ipfs_v0.4.17_linux-amd64.tar.gz",
  +          "cid": "QmSmogJPQKUbDtvtxBsbstE4RtX7u1RRoApCyQgzj9UvD7",
  +          "sha512": "27aa376e0a542aefadc643503bf3f95c4255fbdeff0f6522bba0fa3f2b9145a30c246d7e9c2e2d260381b8d7b9b96dbc0f29649e2732af8ddce204c205a2f770"
           },

Flat files enable bulk verification via simple shell scripts or just plain shasum -c *.sha512.
Fields in dist.js enable more sophisticated setups to do the same.

How to test locally

  # fetch historical releases of go-ipfs
$ make go-ipfs
$ cd dists/go-ipfs
  # remove last release 
$ rm -rf ../../releases/go-ipfs/v0.4.17
  # rebuild v0.4.17
$ bash ../../build-go.sh go-ipfs github.com/ipfs/go-ipfs/cmd/ipfs versions
  # inspect generated artifacts
$ ls ../../releases/go-ipfs/v0.4.17/
$ view ../../releases/go-ipfs/v0.4.17/dist.json

What about historical releases?

Due to the way we fetch historical data from IPNS those changes will be applied only to new releases. Adding this additional metadata to historical archives requires additional work and QA and if it is really needed it can be tackled in a separate PR, so this one can be merged before go-ipfs v0.4.18 is released.

Demo

(2018-11-02) go-ipfs 0.4.18 just got published on the website and has .cid and .sha512 files:

• https://ipfs.io/ipfs/Qmc62xHE51VourcXcuc1g81ct5PwfVb99Czy4zUnLwKazH/go-ipfs/v0.4.18

[dignifiedquire]

@lidel have you tested this on both macos and linux? just making sure because sometimes builtin shell commands don't work exactly the same

[lidel]

@dignifiedquire tested on linux, don't have mac around unfortunately, however shasum -a 512 should work the same according to this.

[dignifiedquire]

I tested shasum -a 512 locally and it worked.

[dignifiedquire]

LGTM, nice improvement

[lidel]

@dignifiedquire I don't have power to merge this, are you able to do it? (or grant me the rights)