Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The buildbuddy repo is incompatible with OSS tooling #16

Closed
achew22 opened this issue May 4, 2020 · 2 comments · Fixed by #17
Closed

The buildbuddy repo is incompatible with OSS tooling #16

achew22 opened this issue May 4, 2020 · 2 comments · Fixed by #17

Comments

@achew22
Copy link
Contributor

achew22 commented May 4, 2020

The BuildBuddy repo uses the Google convention of having a go_library named for each package you would like declared. Unfortunately this is incompatible with the opensource golang ecosystem which makes importing it to a standard golang tree impossible.

Would you be open to a PR that moves to a more standard OSS golang import path?

@siggisim
Copy link
Member

siggisim commented May 4, 2020

We're just a couple of ex-Googlers who went with what we're used to 😂

I think we'd be open to this at some point in the future - but right now we're moving quickly and think the migration would slow us down a bit more than we'd like.

The go_default_library naming is probably our biggest annoyance with the OSS standard at the moment.

Where exactly do you see the breakage? With gazelle, or some other tool? I wonder if there is some workaround we can implement in the meantime.

achew22 pushed a commit to achew22/buildbuddy that referenced this issue May 6, 2020
This looks like a huge change, but I promise it's quite simple. In fact
it's so simple that I was able to sync it with several days of commit
from your mainline repo and only had one merge conflict (`server/BUILD`,
which I replaced entirely) + updating the new imports you had made to
point at their new path. I promise this won't negatively impact your
productivity.

If a file previously was declaring a unique package, put it under the
equivalent path. For example:

`server/backends/cachedb.go` was creating a package
`github.com/buildbuddy-io/buildbuddy/cachedb`, so I simply moved it to
the directory that would be associated with it.

Additionally, I manually mapped the `go_proto_library`s in `/proto` to
have an `importpath` that was relative to the repo root
(`github.com/buildbuddy-io/buildbuddy`). This means that Gazelle knows
about the existance of these proto files and can automatically add
dependencies on them.

Why is this worth merging?

 * You will now be able to get the beautiful godoc.org docs when you're
   working on your project.
 * I made Gazelle work, so now you'll no longer have to manually update
   your build files.
 * You will also be able to use more of the go ecosystem around linting,
   code fixes (eg), and more!
 * Having code that uses the standard tools decreases the burden on
   external contributors to your project (like me) who would be
   delighted to pitch in and fix issues here and there. After all, this
   is the whole reason to do open source, right?

But isn't `go_default_library` ugly?

 * Yes, yes it is. It's also a hack that has been in place in `rules_go`
   for a while that is not beloved by the maintainer. In the olden days
   of rules_go, before there was even a Gazelle, the package name was
   inferred from the path to the file. You would declare your repo root
   when you loaded rules_go in your WORKSPACE. This, over time, was
   deemed to be insufficiently flexible for open-source where you can't
   make modifications in a `third_party` context. They have a different,
   now deprecated system called `vendors` that behaves similarly, but
   differently enough that it caused problems.  In order to address
   this, `importpath` was added as a stopgap to allow the file path and
   the `importpath` to be inconsistent. Then there was an issue around
   creating crosscompilation targets for linux from osx. This issue has
   since been resolved in the master for rules_go using transitions, and
   as of the next release of rules_go.

   Fixing `go_default_library` is now just a matter of someone going
   into gazelle and updating it to have a flag to select naming things
   name things `go_default_library` or the directory that contains them.
   I think that is probably something I'm going to do in the near
   future.

Testing strategy:

I've ensured that the entire repo is able to build/test completely. Note
that there are no tests checked into this repo, so the `bazel test`
returns failure. The `bazel build` was successful, however.

```
$ bazelisk test //...
INFO: Invocation ID: e4f2626b-88ef-4e32-a52c-1c0bdbe10780
INFO: Streaming build results to: https://app.buildbuddy.io/invocation/e4f2626b-88ef-4e32-a52c-1c0bdbe10780
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
INFO: Analyzed 117 targets (0 packages loaded, 0 targets configured).
INFO: Found 117 targets and 0 test targets...
INFO: Elapsed time: 0.262s, Critical Path: 0.00s
INFO: 0 processes.
ERROR: No test targets were found, yet testing was requested
INFO: Streaming build results to: https://app.buildbuddy.io/invocation/e4f2626b-88ef-4e32-a52c-1c0bdbe10780
INFO: Build completed successfully, 1 total action
$ bazelisk build //...
INFO: Invocation ID: d1abc034-6186-4f9c-9b0a-0763c3725dc3
INFO: Streaming build results to: https://app.buildbuddy.io/invocation/d1abc034-6186-4f9c-9b0a-0763c3725dc3
INFO: Build option --test_sharding_strategy has changed, discarding analysis cache.
INFO: Analyzed 117 targets (0 packages loaded, 16507 targets configured).
INFO: Found 117 targets...
INFO: Elapsed time: 1.096s, Critical Path: 0.07s
INFO: 0 processes.
INFO: Streaming build results to: https://app.buildbuddy.io/invocation/d1abc034-6186-4f9c-9b0a-0763c3725dc3
INFO: Build completed successfully, 1 total action
```

Fixed: buildbuddy-io#16
@achew22
Copy link
Contributor Author

achew22 commented May 6, 2020

We're just a couple of ex-Googlers who went with what we're used to joy

Oh yeah, I know that feeling 😉

Where exactly do you see the breakage? With gazelle, or some other tool? I wonder if there is some workaround we can implement in the meantime.

For starters, none of the godocs work which makes it much harder as an outsider to delve into the project. In addition I get a lot of complaining from Gazelle when I run it.

It sounds to me like the big blocker is finding dev resources to do this, so I threw some 20% time with some bash and a couple of for loops change to make you conformant with the expectations of the OSS world. I have prepared #17 to demonstrate how it could be done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants