Make bazel-bin more reliable

[gregestren]

Description of the problem / feature request:

Targets don't appear in bazel-bin in mixed-configuration builds. See, for example, #12389.

--use_top_level_targets_for_symlinks partially solves this problem, but not completely.

Any other information, logs, or outputs that you want to share?

I explored this and wrote some prototype code.

Conclusion: there's no trivial way to support multi-config bazel-bin, if we consider "trivial" = "super-low chance of depot breakages"

The code I wrote added a new postprocessing step: if an output isn't in the top-level configuration, add a new symlink under bazel-bin to it. i.e.:

bazel-bin/mydir/myapp -> bazel-out/x86-fastbuild/bin/mydir/myapp

This roughly works but sacrifices Bazel's correctness guarantees:

1. What if a legitimate file already exists at bazel-bin/mydir/myapp?
2. We could adapt the logic to "only create the symlink when bazel-bin/mydir/myapp doesn't exist
3. But what if bazel-bin/mydir/myapp is the symlink from the last build and we need to update it?
4. We could adapt the logic to "delete existing bazel-bin/mydir/myapp" if it's a symlink
5. What if the targets legitimately output symlinks?
6. We could adapt the logic to check if the symlink reaches out to a different bazel-out root
7. But target outputs can reach out to different roots (like if they inherit outputs from their children)
8. If we accidentally preserve an old symlink we should have deleted, it may point to the wrong value for the current build
9. So it's an unresolvable mess trying to preserve correctness.

Since by definition this problem occurs when mixing configs, there is no perfect solution in which bazel-bin points to a single config base.

Therefore, to stay correct we'd need to refactor bazel-bin to mix configs. This is fine: bazel-bin -> bazel-out/mixedconfigs/bin and then symlink forest all target outputs. And clean out bazel-out/mixedconfigs/ for each build.

But which outputs do we symlink from mixedconfigs? Just the targets? Also its runfiles? Do debuggers need debug symbols? We can no longer take for granted that every output in bazel-bin/ can access all its peer outputs in the same directory. Only if we symlink them all.

As a worst-case backup we could symlink the total union of all bazel-out/... paths, and give up if there are any conflicts. This seems like overkill though.

The ultimate answer depends on how people really depend on bazel-bin in practice.

[gregestren]

I suspect we can get creative and get past those complications.

But it sounds like more than a quick fix.

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.