Provide a way to externally provide overriding aliases

[mattfarina]

Imagine the case where you have a build environment disconnected from the Internet and using a mirror or mirror like server. Now you have a project with an existing glide yaml file. To point to the mirror should that be a fork of the glide yaml file or some outside config to glide (e.g., an environment variable to a config file aliasing repos)?

I'm thinking of the enterprise use cases where vendoring is used to avoid going out to the Internet regularly to pull dependencies.

Thoughts?

[technosophos]

I can see the following as valid options:

Check _vendor Into Your VCS

In this case, your build env doesn't have to do any remote fetching.

Advatages:

• No changes to Glide at all

Disadvantages:

• You basically have to fix the thing with repos checked into your repo, which will probably devolve into submodules or subtrees, and then nobody will like you.

Use multiple glide.yaml files

The use case here is actually similar to what we had in mind when we first wrote glide pin. In that case, we wanted dev to always get the most recent version of packages, but then we wanted to be able to freeze the file for deployments.

Basically, someone could post-process a Glide file (maybe with a plugin) and set up the import paths to use mirrors.

Advantages:

• Meh

Disadvantages:

• Meh

Support environment variable interpolation in glide.yaml

The idea here is that glide.yaml files can contain environment variables.

Advantages:

• Sounds elegant on paper

Disadvantages:

• Is icky in practice.

Support Local Packages

The idea here is that Glide allows you to set the VCS to something like local, which then looks somewhere on your filesystem for a package and creates a symbolic link into _vendor for that. I built a plugin to do this for GPM, and it actually worked better than I expected.

Advantages:

• Conceptually simple
• Solves some other problems (like sharing common repositories across multiple projects)

Disadvantages:

• Defers the problem of actually fetching those packages to something else. Of course, you could create special glide projects for production and dev that fetch the actual repos from their sources and then reference those.

[mattfarina]

@technosophos nice go at coming up with advantages and disadvantages of my idea that I just wrote down off the cuff.

The issue is, I think, more complicated than a local system. More than one developer may be working on the project and you may use a CI/CD system with multiple nodes. Each environment would need access to the mirror.

I think that nixes the idea of local packages.

[wav]

I think this would make sense as some sorta plugin that "re-writes" the package information that glide uses for sourcing. The plugin would be executed before glide does a fetch.

The plugin can then simply be a script that takes the package information and does a lookup.

[mattfarina]

@wav I like the idea of this being pluggable. That way alternative approaches could be used.

[mhoglan]

Could be good to also look at how this is handled on systems currently. There are two ways I typically handle this:

• DNS redirection, whether that is through /etc/hosts on the machine or the DNS server it is utilizing
• local .gitconfig that performs insteadOf on the URL
  • this is also typically the way you make go use SSH always even if the URL is referencing https

e.g. .gitconfig way

[url "git@my-git-mirror:"]
        insteadOf = https://github.com/

One use case that I think this starts a path down, is a similar concept of the repo.xml that maven has and the ability to change out the central repository you are working. While there is no central repository here, you are changing the source of your dependencies to come from a mirror instead of the original. That is the same concept of changing the repository that artifacts are coming from in maven.

[mattfarina]

@mhoglan I'd like a glide native way to do this that's easily sharable and works across systems.

There are lots of environments where it's not allowed or wise to much with the /etc/hosts file. Mucking with that feels like a bit of a hack.

A Git specific mechanism doesn't work for SVN, Bzr, or Mercurial. All three of those are supported by Go and Glide.

I'd like to have something elegant that supports all the supported VCS.

[mhoglan]

Agree, I was just mentioning those as ways that people do it today, just in case there are some insights that can be gained by looking at those work arounds and the use cases they are used for. And to help make sure a solution alleviates having to use those work arounds.

I do think getting to where you can have a site style file which can span projects for specifying mappings is beneficial for several use cases. It also could be the place you could start to define policies. I would like to be able to give every development environment a configuration file that would help dictate their usage, such as only these imports are allowed, or these are not allowed. Becomes important when considering GPL or other licensing concerns. One of the primary reasons you use a mirror and disallow internet access to confine dependencies to a controlled list that has been reviewed and approved.

Another thought, those restrictions / policies are not necessarily related to the application, but to the usage of the application. dev usage is restricted this way. scm usage is restrict another way. etc... Some third party external using my open source application may have zero restrictions. That to me helps point that this solution should be environment based versus in the application source. Very "profile" like. Passing in a profile to use when executing.

[arschles]

Would aliases extend to transitive dependencies as well?

Following on my comment at #372 (comment) since that issue was closed.

[ipfans]

Is there any plans to configure VCS rewrites? I think it's very useful for users from countries those has internet access restrictions. For example, in China, we need to redirect requests to sites that we can access. Some packages from Google servers will block by government.

Like pip for python or npm for nodejs, it support a arg -i to pypi mirror:

pip install -i https://mirrors.pypiorsomething.com flask

[mattfarina]

This was just merged to master. See the mirror command and subcommands.