Create "glaze" BUILD file generator for Go

[damienmg]

See bazelbuild/bazel#758 for more context

[kamalmarhubi]

@damienmg Issue title needs a small fix :-)

Is the intention that Google will adapt and release the internal tool, or will this be an entirely new tool for the open source Bazel?

[damienmg]

oops done. The idea is to adapt the internal tools AFAIU.

[kamalmarhubi]

Will the tool allow generating BUILD files for a project with vendored libraries? This is the issue I'm currently coming up against. (See also my mailing list posting.)

[justinsb]

Is there any timeline on when this might happen? I'm new to bazel, but if there are problems with open sourcing the internal tool, I wonder if maybe we could recreate it as open source (the go parsing libraries are pretty good and have lots of examples because of generators now). I even wonder if we could write it as a go generator...

What I would find very helpful are some example outputs of glaze so that we know what we're aiming for. For example, I think having the glaze-output for https://github.com/golang/glog would be a nice trivial example, and https://github.com/golang/protobuf would be a good non-trivial example (and both libraries are fairly widely used, I presume also inside google...)

[yugui]

Is there any timeline on when this might happen

No, but I'm trying to show some limited version of glaze.

I don't think there's a timeline for opensourcing glaze itself for now.
This is because currently we have none inside Google who maintain rules_go -- while I am maintaining it as an external contributor. It means none who is working on rules_go has access to their internal source codes.
So we have to wait for someone inside Google volunteering in order to get glaze opensourced.

To unblock this situation, I have just started to prototype a limited version of glaze which generates BUILD files for external dependencies. It would generate something similar to glog.BUILD or goprotobuf.BUILD.
I think the prototype will be also a good start point of generating BUILD files for your project.

[damienmg]

We have no timeline for getting glaze open-sourced. This requires open-sourcing our BUILD file parsing library that will come sooner that glaze. Sorry for the delay but as @yugui has explained we have no go rules maintainer for now on the Google side and we need to fix that first.

[justinsb]

@yugui thank you! I would love to help (even if only by testing) on your version of the glaze tool. Is it on github?

Based on your examples, I was able to get a bazel build (all the way to docker images!) for one of my simpler projects...

kopeio/aws-controller@a5e7812

I used the vendor approach, as described in https://github.com/bazelbuild/rules_go I am not sure of any disadvantages this would have vs the external approach you seem to favor. One potential advantage is that a go vendoring tool could also create the BUILD files in the vendor directory.

It was tedious (looks like we have to create a target for every package & subpackage), but it was largely mechanical, so I am optimistic that we can create a tool for the simpler cases!

[justinsb]

PS and thank you to @damienmg also - knowing that the glaze tool is not imminent is also helpful. Perhaps if we are heading in a totally contrary direction to how glaze works you can repoint us though!

[jmhodges]

@damienmg I'm guessing that parser tool you're speaking of is not the one inside of https://github.com/bazelbuild/buildifier/?

[damienmg]

@jmhodges I was thinking of another one but glaze actually use buildifier so it is the good one :)

[chenxiaoqino]

Just adding a quick note, we should to also get goimports running with Bazel, as usually we do goimports && glaze everyday at Google.

This is now possible because Google have tentatively open-sourced Grok as Kythe.

(More specifically, goimports can index the Bazel workspace and automatically add import statements for packages across the workspace, similar to the internal tool?)

[bshi]

While we're all waiting for glaze, here is a quick and dirty script that automates away most of the tedium of writing BUILD files for a large collection of dependencies.

pre-glaze-pkg.py

Requires buildifier installed. I was able to get a project with ~300 vendored packages (mostly kubernetes client code) building with bazel with relatively few manual fixups required. YMMV!

[yugui]

@pmbethe09

I plan to send some more PRs toward go_vendor rule and gazelle in general. I'd really appreciate it if you review them when you have some time.

• Let Gazelle also generate rule visibility
• Support the case that the repository root is not a buildable go package
• Add another tool which fetches remote go repository from the given import path.
• Add go_vendor rule
• Add "flat" style of vendor BUILD file to support the pattern 2 in Support specifying custom package name #16 (comment)
• Respect manually added attributes and non-Go rules when regenerate BUILD files.