Proper use of --platforms for 'bazel run' and --run_under

[moroten]

Is there any use case of running bazel run --platforms=//:not_my_host_platform //:my_target?

1. Is it only useful if running with --script_path and packing the build output?
2. Maybe extending the default host platform with e.g. GPU constraints? Shouldn't that be done using --host_platform instead?

For --run_under, why is the run-under target built for the target platform and not the host platform? Under remote execution, shouldn't bazel test --run_under= use a third run-under execution platform, different from --platforms and --host_platform?

The use case I'm considering is bazel run --run_under=@rules_oci//:push --platforms=//:linux_armv8 //:my_image where the image platform (armv8) is not necessarily the same as my host (x86-64). Currently, Push targets (executables) are created with transition to a wanted target platform for the image.

@katre Do you have any insight?

[katre]

bazel run is currently not working well with platforms.

I agree that the right approach would be for run to ignore the target platform (or ensure it matches the host) and use --host_platform, since it's implicitly running on the bazel host. We haven't ever prioritized doing anything special here because the default is that --platforms defaults to --host_platform, so it's mostly there. (But I'd be happy to review a PR!)

The intersection of bazel run, --platforms, and --run_under is much more confusing and needs more thought though, but my first instinct is:

1. The actual thing that runs is whatever --run_under points to (@rules_oci//:push in your example), and that's running on the host platform (and should be built for the execution platform)
2. The actual argument (//:my_image in your example) is effectively data at this point, and bazel can't tell what platform it should be built for (so using --platforms to tell it makes sense)
3. In your example, how does @rules_oci//:push know what to do with an image built for //:linux_armv8? Some kind of image metadata? Further out-of-band signals?

It'd be nice to clean this up, but I'm not sure anyone on the Bazel team has time to tackle it. This definitely feels like a situation where we could accept community patches, however.

[moroten]

Further ideas we got here today was to have --features=<platform-name>:warnings_as_errors, i.e. replace the host_* flags with a platform naming scheme. --experimental_override_name_platform_in_output_dir already gives nice names to the platforms.

[anguslees]

Just to say it out loud: //:my_image in this example probably could/should include a transition to build the image for a specific image platform (ie: //:my_image_linux_amd64). In particular, transitions are required when building a multi-arch OCI image from a single bazel build. Using a transition like this essentially makes this --run_under issue go away, since the default --platform is --host_platform (which is correct for this case), and the image would be transitioned by bazel config to the intended image platform eg linux_amd64 regardless of host (which is also correct for this case).

This doesn't materially change the original question though: --run_under should unambiguously be the host platform, and (when using transitions) the --platform arg would basically be ignored for the image target. I think this means we could/should make --platform the host platform for bazel run. Fwiw, the approach I use locally (as @katre similarly points out) is to leave --platform unspecified since it defaults to host_platform (yay), and override --platform in my RBE-enabled .bazelrc only for specific RBE-only bazel operations like test.

[gregestren]

We're going to try to clean this up, at least for test: #22624.

[gregestren]

I have a Gerrit fix for $ bazel test --run_under at https://bazel-review.googlesource.com/c/bazel/+/255473.

I think we ultimately want:

1. $ bazel run //foo - Fail if the target and exec configurations don't match. Rationale: this is basically shorthand for $ bazel build //foo && blaze-bin/foo. I don't think it should produce a different binary than $ bazel build //foo. If you swap out build with run at the terminal it'd be awfully subtle if you're no longer building the same binary. But it makes no sense to run a binary on the host machine if it's not built for the host.
2. $ bazel run //foo --run_under=//runner - Build //foo for the target config, //runner for the exec config. Since the host machine doesn't run //foo it no longer matters if //foo is built for a different config.
3. $ bazel test //foo - Build //foo for the exec config. By definition this runs on an exec machine.
4. $ bazel test //foo --run_under=//runner - Build //runner for the exec config (as my change does). I'm not sure about //foo. I believe bazel still requires that //foo must be a test. We might need a stronger API to distinguish this vs. some binary that should now be staged with //runner.