Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproduce bazel transitions bug, where build settings delta is used to determine output directory hash #124

Closed
wants to merge 1 commit into from

Conversation

amberdixon
Copy link
Collaborator

Bazel computes the output directory hash for artifacts under bazel-out when there is a transition. It computes the hash by only looking at the transition build settings which have changed. As a result, the same target built with the same build setting can be placed into different output directories, depending on which target transitioned into it. This makes it impossible to determine where we can expect to find a built artifact.

Here's an example of how to repro this problem from this branch:

# First, build the framework FW. Note that it gets placed in the directory with hash `7bf874b56ea0`
~/Development/rules_ios master * bazelisk build tests/ios/app:FW
INFO: Analyzed target //tests/ios/app:FW (1 packages loaded, 82 targets configured).
INFO: Found 1 target...
Target //tests/ios/app:FW up-to-date:
  bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework/FW
  bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework/Modules/module.modulemap
  bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework/Headers/FW.h
  bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework/Headers/FW-umbrella.h
  bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework/PrivateHeaders/FW_Private.h
INFO: Elapsed time: 3.435s, Critical Path: 3.18s
INFO: 10 processes: 9 darwin-sandbox, 1 local.
INFO: Build completed successfully, 14 total actions

# Then, build the application App that depends on FW
~/Development/rules_ios master * bazelisk build tests/ios/app:App
INFO: Analyzed target //tests/ios/app:App (0 packages loaded, 121 targets configured).
INFO: Found 1 target...
Target //tests/ios/app:App up-to-date:
  bazel-bin/tests/ios/app/App.ipa
INFO: Elapsed time: 4.517s, Critical Path: 4.10s
INFO: 15 processes: 13 darwin-sandbox, 2 local.
INFO: Build completed successfully, 21 total actions

# Notice that there are now two configs for FW. They have also been placed in two different output directories:
~/Development/rules_ios master * bazelisk cquery tests/ios/app:FW
INFO: Analyzed target //tests/ios/app:FW (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
//tests/ios/app:FW (584265259504960925847f78c1f32d5d5a091546f08035f4c215c2f07a7a4bf4)
//tests/ios/app:FW (660c7b4ee866aa5f0e022dfc89175514e11a05e1605d8ad86d5d336098469799)
INFO: Elapsed time: 0.094s
INFO: 0 processes.
INFO: Build completed successfully, 0 total actions

~/Development/rules_ios amber/repro-bazel-transitions-bug ls bazel-out/*/bin/tests/ios/app/FW/FW.framework
bazel-out/applebin_ios-ios_x86_64-dbg-ST-7bf874b56ea0/bin/tests/ios/app/FW/FW.framework:
FW             Headers        Modules        PrivateHeaders

bazel-out/applebin_ios-ios_x86_64-dbg-ST-e763699e1aa1/bin/tests/ios/app/FW/FW.framework:
FW             Headers        Modules        PrivateHeaders

# diffing the bazel configs indicates that the same copy of FW was placed in different output directories.
~/Development/rules_ios amber/repro-bazel-transitions-bug bazelisk config 584265259504960925847f78c1f32d5d5a091546f08035f4c215c2f07a7a4bf4 > /tmp/build-fw-alone
~/Development/rules_ios amber/repro-bazel-transitions-bug bazelisk config 660c7b4ee866aa5f0e022dfc89175514e11a05e1605d8ad86d5d336098469799 > /tmp/build-fw-with-app

~/Development/rules_ios amber/repro-bazel-transitions-bug diff /tmp/build-fw-alone /tmp/build-fw-with-app 
1,2c1,2
< BuildConfiguration 584265259504960925847f78c1f32d5d5a091546f08035f4c215c2f07a7a4bf4:
< Skyframe Key: BuildConfigurationValue.Key[584265259504960925847f78c1f32d5d5a091546f08035f4c215c2f07a7a4bf4]
---
> BuildConfiguration 660c7b4ee866aa5f0e022dfc89175514e11a05e1605d8ad86d5d336098469799:
> Skyframe Key: BuildConfigurationValue.Key[660c7b4ee866aa5f0e022dfc89175514e11a05e1605d8ad86d5d336098469799]
24c24
<   affected by starlark transition: [apple_split_cpu]
---
>   affected by starlark transition: [apple configuration distinguisher, apple_platform_type, cpu, ios_minimum_os]
79c79
<   transition directory name fragment: ST-e763699e1aa1
---
>   transition directory name fragment: ST-7bf874b56ea0

# The only difference is the set of build settings that was changed by the transition. The values of the build settings themselves were the exact same for both builds.

…t when there is a transition. It computes the hash by only looking at the transition build settings which have changed. As a result, the same target built with the same build setting can be placed into different output directories, depending on which target transitioned into it. This makes it impossible to determine where we can expect to find a built artifact.
@amberdixon
Copy link
Collaborator Author

THis PR was just for demonstration of a bazel bug, so closing it.

@amberdixon amberdixon closed this Oct 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant