Skylark support for custom configuration transitions

[gregestren]

Part of the Skylark build configuration project.

This is a tracking issue for the ability of Skylark writers to trigger configuration transitions, which essentially means having a dependency use different flags than its parent. One common example is building a dep with a different --cpu.

Practically, we may limit (via code restrictions or best practice guidance) initial usage to platform transitions. This is because the platform effort is a complementary project aimed at consolidating --cpu, --crosstool_top, etc. into a a single platform entity. And most configuration transitions in practice trigger those kinds of settings. So we want to be careful about fragmenting the transition space too widely too early.

This complements the Bazel Configurability Roadmap. Write questions / comments / status updates here. See there for the bigger picture.

[kamalmarhubi]

@gregestren just checking in after bazelcon: is this the right issue to track for split transitions from starlark?

[gregestren]

For splits specifically, #5575 is probably a closer match. I intended #5574 to capture just 1 -> 1 transitions, which is a prerequisite step.

It's ultimately up to @juliexxia. But to be safe, track both? :)

[kamalmarhubi]

Ah excellent. Tracking both, and for good measure scanning things under the configurability category.

[gregestren]

Commit: "Introduce the transitions() function and "transition" type in Starlark" (e0efc14, @c-parsons)

[gregestren]

Discussion context for parameterized transitions (transitions that change a flag on dependencies but read some attribute on the attached rule to determine what to change to): https://groups.google.com/d/msg/bazel-dev/Lvmz6f_p9jY/XlzyT3T8AwAJ

[gregestren]

2018 EOY update:

This feature is partially done.

• It's now possible to define custom transitions in Starlark that, for example, change the --cpu for a rule's transitive dependencies.
• This currently requires `--experimental_starlark_config_transitions. But that flag should go away by the next release or two.
• This also requires documentation to explain the performance & memory consequences. This can easily make the build graph much larger, and harm action caching & general build speed. So it needs to be used with care.
• Proper documentation is planned for 2019.
• Certain kinds of transitions with extra power are still in pre-implementation phase.
• Transitions over user-defined flags is still in progress.

[gregestren]

April '19 update:

• --experimental_starlark_config_transitions is no longer required for basic transitions. Some more advanced functionality still needs that flag. The goal is to shift more logic out of that flag and into production transitions incrementally over upcoming months.
• Documentation still remains a TODO. Not top priority at the moment. Still a priority by end of summer.
• Transitions on user-defined flags have made strides and work in at least simple form. Still pieces missing, notably selecton user-defined flags, that @juliexxia is vigilantly moving forward.
• Also good progress recently on reducing needs for --define, which is basically a legacy flag that user-defined flags should ultimately replace and be better at in every way
• Parameterized transitions work (ones that read attributes), but certain corner cases like attribute-reading rule transitions (my_rule = rule(cfg = _transition_reading_my_rules_attributes)) don't work if the rule has a select.

@juliexxia is at the cutting edge of all this and can correct any parts I got wrong.

[steeve]

hey @gregestren, is there a reason the whitelist (@//tools/whitelists/function_transition_whitelist) is still needed for transition now that the flag is no longer required ?

[gregestren]

Hi @steeve - those are somewhat orthogonal.

Non-experimental means the API is stable and functional, and basically usable for production code. But the performance challenges remain real. "Unskilled" use of transitions could blow up build graphs and make builds way more sluggish than expected.

We want to give people the power to evaluate this impact for their own projects, vs. blocking the feature outright until all theoretical scale problems are solved. But since these risks remain we kept the whitelist requirement in case project owners feel the need to lock things down. In other words, the whitelist defaults to everything:

bazel/tools/whitelists/function_transition_whitelist/BUILD

Line 6 in c73aa26

packages = ["//..."],

but project owners can further restrict that if they feel that's safer.

It looks like external dependencies may still need whitelist support FYI.

Also, I don't think this is documented anywhere. That should be a TODO.

[gregestren]

Regarding:

• Documentation still remains a TODO. Not top priority at the moment. Still a priority by end of summer.

Starting documentation available at https://docs.bazel.build/versions/master/skylark/config.html.

[steeve]

That is a very nice and detailed explanation, thank you!

[juliexxia]

May '20 update:

• --experimental_starlark_config_transitions is no longer required at all
• Documentation is mostly complete (including whitelist details) at https://docs.bazel.build/versions/master/skylark/config.html
• Select on user-defined flags is implemented
• ctx.split_attr. is implemented, pending a new bazel release for the code to go live
• Work being done (this week!) to toss a bunch of cook book examples into the bazel examples repo: https://github.com/bazelbuild/examples

Closing this issue as done! 🎉

[oehme]

The documentation for "configurable build attributes" should be updated now that this is implemented. It currently still says that "config_setting currently only supports built-in Bazel flags". Overall the docs on configurability are in a bit of a confusing state at the moment, leaving me to wonder on what the best practices are and how to use all these new features.

[gregestren]

@oehme - can you elaborate more on general docs confusion? Feedback would be enormously helpful. We'd love to clarify best practices where not clear.

[bttk]

What @oehme writes seems right. The statement "config_setting currently only supports built-in Bazel flags" is not true: rule config_setting now has an argument flag_values described as The same as values but for Starlark-defined flags.

Use of flag --define is not recommended, see General Rules doc:

If you need to define conditions that aren't modeled by built-in Bazel flags, use Starlark-defined flags. You can also use --define, but this offers weaker support and is not recommended.

Examples using --define shouldn't be as prominent, and Starlark-defined flags should be preferred. Here is an example use of config_setting: https://docs.bazel.build/versions/master/skylark/config.html#build-settings-and-select

[oehme]

My main feedback is that --define is still widely used in the docs, even though it is discouraged. Also it's not clear how the new custom configs play together with platforms (if at all). E.g. when would I use a constraint_setting vs config_setting. Some examples (that aren't just using abstract names like "myRule", "myConfig") would go a long way. Maybe I just missed a link to them.

On a more general note, I don't like documentation that says "this will be possible in the future", "this will be added in this PR" (which has since been closed, but without updating the docs) or which presents incomplete new APIs mixed in with the existing ones. I'd much rather have the docs reflect the current best practices and then be updated when new best practices are fully ready. At that point the old practices can be moved to their own "outdated" chapter, so newcomers don't run straight into them.

For context, I'm the kind of person who read's a tool's user guide back-to-back to get an overview of everything that is possible and on what the current best practices are. This is to avoid cargo-culting by just copying what older projects have done. But right now the configurability chapter is a mix of old and new APIs that makes it hard for me to figure out what to do and what to avoid in a new project.

[moroten]

@oehme I asked about constraint_setting vs config_setting on bazel-discuss which includes a guideline from @gregestren:

We generally recommend trying to stay pure to the intention of platforms: something belongs in a platform if running the same build on two different machines might produce different results ("exec platform" differences) or running the same built binary on different machines produces different output ("target platform" differences).

[oehme]

Just reading this section about platforms and config_settings makes me wonder why everything is specified twice. I.e. why do I create a basalt build_setting and a basalt_platform? Why don't I directly select on the platform or directly specify the build_setting on the command line? What problem does this additional indirection solve? Explaining this with a more elaborate example would probably help newcomers like me understand the concepts better.

Edit: I've finally found the chapter on the new config system much later in the user guide. However, it is still unclear to me how these new build_settings relate to config_settings, constraint_settings and platforms. Are all of these concepts still relevant? When would I use which?

[gregestren]

I've scheduled some time to chat with the main devs of these features this Friday. This is great feedback. There are definitely some steps we can take to address some of these points. In the meantime, some random thoughts:

• --define is a widely used pattern today whether we like it or not. I think we need to at least acknowledge it in the docs for those who are committed to using it. But you're absolutely right we have examples that lean on it too much, which were written before build settings were available.
• One impediment to simply deprecating --define today is it's such tight syntax vs. build settings. e.g. --define foo=bar vs. --@some_repo//flag_defs/my_flags:foo=bar. Has anyone used build settings and felt this pain? I'd especially love feedback on this. This is exactly what the shorthand flag aliases project is for. Please weigh in there if that's interesting to you!
• I hear your point about "best practices" docs vs. "coming in the future" docs. I'll make sure that gets discussed this week.
• However, there is still some subtlety. Platforms is the most prominent example. We consider the new platforms API current best practices. But there remains a long tail of migration support for the rule sets that defined their own APIs before this one was available. This creates the subtlety that your best practices depend on which rules your org uses & how. The more you depend on legacy APIs the more you'd necessarily need to lean into them. Building with platforms tries to provide a coherent overview of the situation. This is also why we're up-prioritizing just getting those migrations done for the big rule sets (C++, Java) to decisively commit everyone globally to the new API.
• build_setting, config_setting, constraint_setting, and platform are all distinct concepts that should principally work together in coherent ways. I accept your argument about making this distinction clearer, maybe with some dedicated docs to clarify the difference.

Anyway, again this is all really helpful. And really happy to hear you're not a cargo culter! Cargo culting indeed makes more problems for everyone, and it's such a common pattern to slip into.

[oehme]

I understand, it's hard to make both new and existing users happy. I think a major problem for me was that the information about these different configuration aspects is spread over several chapters, some of them quite recent, others dated. This makes it hard to get an overview of everything that is there and what each concept is used for. I'd definitely appreciate clarification on the way that build_setting, config_setting, constraint_setting and platform work together. Thanks for taking the time to improve this!

[gregestren]

FYI @oehme and others, we have some changes coming through the pipeline to address your doc comments (in #11967).

We probably won't address everything perfectly, but can at least get a few good steps up from the status quo.

I'm still torn on how decisive we should be about discouraging --define, but I would like to offer more decisive opinion on that.

Thanks again for your input.