[6.5.0] Restart at most once when prepopulating repository rule environment #20667
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When a repository rule is fetch attributes are iterated over in `enforceLabelAttributes` to prepopulate the environment, restarting the fetch each time a new dependency is discovered. This is faster than calling `repository_ctx.path(...)` early in the repository rule implementation function but still has considerable overhead. This PR defers throwing `NeedsSkyframeRestartException` till after every attribute has been processed, greatly reducing the number of restarts and iterations. In an internal project these optimisations are particularly noticeable. Before: 2min 8s, 2min 7s After: 1min 35s, 1min 27s Closes bazelbuild#20434. PiperOrigin-RevId: 588090528 Change-Id: I7917b137d6e60b6d6a73189cf396418a85b3ec28
bazel-io
added
team-ExternalDeps
Wyverald
approved these changes
Jan 2, 2024
iancha1992
pushed a commit
that referenced
this pull request
Jan 23, 2024
Baseline: 50b61e3 Release Notes: + Fix tree file materialized as symlink to another file when building without the bytes. (#20409) + Don't pass --add-opens= to javac (#20472) + Flip --incompatible_visibility_private_attributes_at_definition (#20520) + Fix extraction of tar archives containing sparse files. (#20531) + RemoteSpawnRunner: record inbetween phases in timing profile (#20550) + Add profiling to `remoteActionBuildingSemaphore.acquire()` (#20549) + The label API shakeup & docs cleanup (#20590) + Disable rewriter test (#20758) + Disable PyTest.testSmoke on macOS (#20729) + Upgrade abseil-cpp to fix build on macos_arm64 (#20785) + Ignore read-only errors when updating the `mtime` of the `install_base` (#20568) + Restart at most once when prepopulating repository rule environment (#20667) + Fix bootstrapped Bazel binary (#20804) + Add flag `experimental_throttle_remote_action_building` (#20861) + Fix versioned shared libraries for macOS toolchain (#20847) + Proto toolchainisation cherrypicks (#20925) Acknowledgements: This release contains contributions from many people at Google, as well as bazel.build machine account, Brentley Jones, Fabian Meumertzheim, Jordan Mele, Mai Hussien, oquenchil, Rahul Butani, Son Luong Ngoc, Xùdōng Yáng.
copybara-service bot
pushed a commit
that referenced
this pull request
Jan 23, 2024
Baseline: 50b61e3 Release Notes: + Fix tree file materialized as symlink to another file when building without the bytes. (#20409) + Don't pass --add-opens= to javac (#20472) + Flip --incompatible_visibility_private_attributes_at_definition (#20520) + Fix extraction of tar archives containing sparse files. (#20531) + RemoteSpawnRunner: record inbetween phases in timing profile (#20550) + Add profiling to `remoteActionBuildingSemaphore.acquire()` (#20549) + The label API shakeup & docs cleanup (#20590) + Disable rewriter test (#20758) + Disable PyTest.testSmoke on macOS (#20729) + Upgrade abseil-cpp to fix build on macos_arm64 (#20785) + Ignore read-only errors when updating the `mtime` of the `install_base` (#20568) + Restart at most once when prepopulating repository rule environment (#20667) + Fix bootstrapped Bazel binary (#20804) + Add flag `experimental_throttle_remote_action_building` (#20861) + Fix versioned shared libraries for macOS toolchain (#20847) + Proto toolchainisation cherrypicks (#20925) Acknowledgements: This release contains contributions from many people at Google, as well as bazel.build machine account, Brentley Jones, Fabian Meumertzheim, Jordan Mele, Mai Hussien, oquenchil, Rahul Butani, Son Luong Ngoc, Xùdōng Yáng.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a repository rule is fetch attributes are iterated over in
enforceLabelAttributesto prepopulate the environment, restarting the fetch each time a new dependency is discovered.This is faster than calling
repository_ctx.path(...)early in the repository rule implementation function but still has considerable overhead.This PR defers throwing
NeedsSkyframeRestartExceptiontill after every attribute has been processed, greatly reducing the number of restarts and iterations.In an internal project these optimisations are particularly noticeable.
Before: 2min 8s, 2min 7s
After: 1min 35s, 1min 27s
Closes #20434.
Commit e8ac96a
PiperOrigin-RevId: 588090528
Change-Id: I7917b137d6e60b6d6a73189cf396418a85b3ec28