pkg/stash: allow nonsemver requests to be resolved properly

[marwan-at-work]

NOTE: Requires go1.12 beta or above

Fixes #699

1. Say a user does GOPROXY=<athens> go get module@master
2. Go will ping Athens as such: GET <athens>/module/@v/master.info
3. Athens will internally call go mod download -json module@master
4. Go will return a json that includes the following property "Version": "v0.0.0-timestamp-sha"

-- the next step is where this PR comes in --

5. If the given version (master) does not equal the version that mod download returns v0.0.0-timestamp-sha, then we should check if that new version already exists in our storage or not.
6. If v0.0.0-timestamp-sha exists, then we already have it, just point the download.Protocol to it.
7. If v0.0.0-timestamp-sha does not exist, persist it, then point the download.Protocol to it.

Edge cases to contemplate:

1. What happens when a user asks for master but the repo is removed? I think we should error out as this PR does because if a user asks for "master" we assume their intention is to get the latest of an active branch.

2. What happens when a user asks for <long-commit-sha> and the repo is removed? I think we should manually resolve the commit sha to a pseudo version and see if it exists. The problem is that pseudo versions carry the latest tag in them so maybe we should ignore that.

[codecov-io]

Codecov Report

Merging #1015 into master will increase coverage by 0.93%.
The diff coverage is 75%.

@@            Coverage Diff             @@
##           master    #1015      +/-   ##
==========================================
+ Coverage   62.25%   63.18%   +0.93%     
==========================================
  Files          71       71              
  Lines        2379     2393      +14     
==========================================
+ Hits         1481     1512      +31     
+ Misses        767      745      -22     
- Partials      131      136       +5

Impacted Files	Coverage Δ
pkg/module/go_get_fetcher.go	60.3% <100%> (+0.3%)	⬆️
pkg/stash/with_singleflight.go	100% <100%> (ø)	⬆️
pkg/stash/stasher.go	63.41% <61.53%> (+63.41%)	⬆️
pkg/download/protocol.go	70.17% <66.66%> (-0.1%)	⬇️
pkg/stash/with_pool.go	96.96% <80%> (+0.09%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d94af2...120aecd. Read the comment docs.

[marpio]

LGTM
If I understood it correctly we will run go mod download just to get the pseudo version even if we already have the zip/mod/info bits in our storage. I guess there isn't any better way at the moment but it would be great if this mapping functionality would be extracted from go into it's own module at some point.

[marwan-at-work]

@marpio Only if the given version is non-semver, which makes sense because on every call to master we really have to go all the way to upstream to see "what's latest at master?" -- letting go mod download do that for us has more benefits than us copying the code.

[marpio]

👍 I was thinking about a specific commit hash but yes it makes sense for master/branch Also I hope go team will extract some code rather than use copying it.

[marwan-at-work]

@marpio even with commit hashes there are two pieces of information that sort-of require us to clone a repo to find out:

1. The the latest semver tag, because for whatever reason the psuedo version is the latest semver tag plus a minor upgrade. So if you want the tip of a repo that has a latest relase of v0.8.0, you'll get v0.8.1-timestamp-latestCommit
2. The timestamp requires getting the commit info as well.

I had always hoped we'd use APIs directly such as GitHub API, Bitbucket's API etc to know this kind of stuff, but oh well :)

We can potentially have smart logic to figure a bunch of stuff out, but we really don't need that kinda complexity because all of this is only if a user does the less-usual go get pkg@laskdjflskdfjlkdjf or go get pkg@master instead of go get pkg@latest

[marpio]

Got it. Thanks for the explanation.