config_settings() needs a way to discriminate based on platform/OS in addition to CPU

[steven-johnson]

At present, the apparently suggested solution is to examine 'cpu', but this is insufficient; it appears that the only possibilities currently are for linux (either x86-32 or x86-64), darwin (cpu type and size unspecified), or armeabi (which apparently can refer to either android or ios); there doesn't appear to be a way to specify android-x86 or arm64-anything.

To use Bazel effectively, we need to be able to discriminate based on the target platform as well as the CPU type and bit-width; this is absolutely essential for working with prebuilt non-Bazel libraries (e.g. pulled via new_http_archive).

[gregestren]

Agreed. We need a stronger concept of "platform" than exists today - cpu is obviously inadequate. The challenge is to make sure we can properly encapsulate everyone's concept of "platform", in all its manifestations.

To be clear, which platforms specifically are important to you?

[steven-johnson]

which platforms specifically are important to you?

OSX (x86-32, x86-64)
Windows (x86-32, x86-64)
Linux (x86-32, x86-64)
Android (arm32, arm64, x86-32, x86-64 -- we can live without MIPS for now)
iOS (arm32, arm64)

[gregestren]

Work is ensuing over this summer to define a stronger notion of "platform" that I think should cover practical nuances.

[steven-johnson]

Is there any update on this?

[steven-johnson]

No activity on this for a few months. Is it still on the radar?

[kchodorow]

Yes! Sorry about the lack of updates, there is actually a bunch of activity on this front. Let me find someone who knows more about this area to chime in.

[katre]

Work is in progress on this. We've done some planning and I will start implementing soon. The public design doc is at https://docs.google.com/document/d/1-G-VPRLEj9VyfC6VrQBiR8To-dZjnBSQS66Y4nargGM/edit#heading=h.al54v2ddwqzv

[katre]

Platforms now exist in bazel, using the platform() rule. The next step to solve this problem is to use platform and the constraint_setting/constraint_value rules in select() calls, which is a feature we plan to add in the near future.

[katre]

This functionality is now present, marking as closed.

[steeve]

Is there any way to use the constraint system to match on Android or iOS target ? It seems matching on @bazel_tools//platforms: is insufficient.
I mean in for instance when building apple_* targets with for instance --ios_multi_cpu.

It seems at the moment the only way is to use config_setting by matching on ios_{armv7,arm64} or crosstool: external/android....

[katre]

Currently, the ios (and android) rules don't work with the platforms system. We'll address this in the future when those rules move to using platforms and toolchains directly.

[sergiocampama]

Do you have any more information on how the Apple rules should interface with the platform rule?

[steeve]

So right now there is no real way to define a toolchain (or constraint) that could be, say compatible with iOS?
I'm asking because we've been discussing it in bazel-contrib/rules_go#1264 as the rules_go toolchain use constraints heavily and I'm doing very dirty hack with config_setting to sort of work around this.

[katre]

@steeve You can writ a platform definition to describe an iOS target, by defining new constraint_setting targets to describe the various features you need. If that platform is used with the --platforms flag, then config_setting targets can match against it. However, it won't change what iOS toolchain is used, because the iOS rules are not currently platform-aware.

[katre]

@sergiocampama Take a look at the toolchains documentation, but the short way to add toolchain (and platform) support is to:

1. Define a new new toolchain_type target.
2. Define a new ios_toolchain rule (or re-use one that exists, if it exists) to also create a platform_common.ToolchainInfo provider with all platform-specific data that a rule would need.
3. Register instances of the ios_toolchain rule using toolchain and register_toolchain
4. Add toolchains = ['label of toolchain_type'] to your rules that need to access the toolchains, and access them by using ctx.toolchain['label of toolchain type']

The major work will be defining your constraint settings so users can define meaningful platforms for different types of iOS device they would like to target builds for.

[steeve]

@katre looks good, but the issue i have is that iOS and Android targets are builting for multiple architectures (fat binaries or apk) via a single target (by propagating the arch change to deps, it seems).
Would there be a way to handle that?

[katre]

Yes, part of the work will be handling fat binaries by using the correct config split transition.

[sergiocampama]

how is the toolchain support different from the CROSSTOOL?

[steeve]

Is there a way do that already?
To be clear, I'm trying to get rules_go to select the proper toolchain (which is constraint based) based on the current architecture building.
Basically rules_go builds a simple arch .a file, which will would get merged into a multi-arch fat binary via the ios_ rule.

EDIT: Also, this is already the case for all the @bazel_tools cpu and os (which does not define ios or arm64, however https://github.com/bazelbuild/rules_go/blob/master/go/platform/list.bzl#L15)

Something like this:

                +-------------+
                |             |
                |             |
          +----^+   IOS       +^------+
          |     |             |       |
          |     +-------------+       |
          |                           |
          |                           |
          |                           |
 +----------------+           +----------------+
 |                |           |                |
 |                |           |                |
 |   GO ARMV7     |           |   GO ARM64     |
 |                |           |                |
 |                |           |                |
 +----------------+           +----------------+

[katre]

Please take a look at the docs, including the original design doc. The short version is that toolchains works for all rule types, not just CC rules, and has a better idea of what makes different platforms distinct from each other.

[katre]

@steeve There's code to automatically convert from the --cpu flag to the auto-detected target platform (@bazel_tools//platforms:target_platform). This should work properly with split transitions like this, but I've never tested it, so if you're seeing errors there's a bug somewhere.

I'd like to encourage you to file this as a separate issue and I'll put it on my backlog, but it's unlikely to be something I can get to in the near future.

[steeve]

@katre sure thing, thank you for the quick answers
However, it seems the ios_* cpus are defined by the CROSSTOOL, so I'm not sure it will work.
Also, ios and arm64 are not considered OS or CPU at the moment (perhaps that would be the fastest route to take, however).

Can you point me to the code? I might take a look.

[katre]

The auto-configuration for platforms happens here: https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/platform/Platform.java;l=55

And is based on the set of cpus and oses available for autoconfig, set here: https://source.bazel.build/bazel/+/master:tools/platforms/platforms.BUILD;l=81?q=tools%2Fplatforms