Move to go modules

[markmandel]

Apparently Go Modules are a thing now 😄 and where Go is headingin the near future (1.13 I think they will be default).

We should convert over from dep to the modules at some point in the future.

[heartrobotninja]

I will be working on this shortly.

[aLekSer]

@heartrobotninja Please don't forget about #414 which currently was manually fixed in /vendor
kubernetes/kubernetes#70277 is merged, but not sure about target Kubernetes version.

[aLekSer]

@heartrobotninja also note that we can use vendors with modules at the same time to mitigate mentioned above issue:
https://github.com/golang/go/wiki/Modules#how-do-i-use-vendoring-with-modules-is-vendoring-going-away

[heartrobotninja]

Here is what we have so far:

Moving straight to go modules is ideal, but not doable right now. We have code in vendor that contains a fix that will not be released until March 25th, and there is no guarantee when the platforms will update to 1.14 since they are still on 1.12.

Given this, it might be best that we move to go modules while also keeping the vendor directory. This would require re-implementing the fix from #414. If we are ok with doing that, I can do both parts fairly quickly.

[markmandel]

It sounds like a way forward, but what does "re-implementing the fix" entail? Just making sure it's pasted into the new vendor directory?

Question - with go modules,without a vendor directory, does that mean on each CI build, we would have to download all our dependencies every time?

[heartrobotninja]

re-implementing would just be re-applying the fix. I know there is a cache for go modules, but I am currently trying to hunt down how that works with CI builds.

[markmandel]

Gotcha. We'd likely want to store that in GCS and move it back and forth.

[heartrobotninja]

As for CI:
We could either cache all dependencies prior to building. This still makes us download still, but we do it in advance of building. There might be some magic we can do there where we keep the cache warmed.

The other option is doing module-aware vendoring. This is mostly what we do now, so will likely consist of the least changes. Both are supported and I would propose we just do module-aware vendoring for now.

[markmandel]

I would propose we just do module-aware vendoring for now.

SGTM 👍

[heartrobotninja]

Currently have modules working for LOCAL_GO=true. Currently working out some issues with the non-local version failing. Been looking into why this would be a problem for docker run.

[heartrobotninja]

What I am currently running into: The intermediate containers which get instantiated by RUN aren't inheriting environment variables.

[heartrobotninja]

So, the go.mod file isn't visible in the intermediate container. That is the actual problem.

[heartrobotninja]

I finally have gotten things to get past the missing mod file, but now I am trying to sort through the differences in local vs docker because while local building works fine, docker has some issues finding certain packages. The current one is:
build agones.dev/agones/cmd/controller: cannot load k8s.io/api/admissionregistration/v1alpha1: cannot find module providing package k8s.io/api/admissionregistration/v1alpha1

[heartrobotninja]

Its apparently by design per kubernetes/client-go#551 they removed v1alpha1 from master, but we still have an indirect dependency on it via an import of an import.

[heartrobotninja]

Fun fact, when running go build as we do for build-controller-binary, I can't get the build to recognize that a go.mod file exists unless I put a command before go build. For instance if I put pwd && go build -mod=vendor it works.

[heartrobotninja]

Progress! I have make build and make test working. I need to do some e2e testing and documentation and I will have a pull ready.

[heartrobotninja]

e2e testing worked. Fixing some issues with the gen-grpc files that have cropped up due to vendoring.

[heartrobotninja]

Design Doc:
https://goo.gl/uNhtJ2

[markmandel]

Can this be closed now?

[heartrobotninja]

Yep!

[roberthbailey]

Should the GOPATH section of the build/README be removed (or updated) in light of this change or is is still necessary to install inside of GOPATH for IDE integration?