Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.1.0] Repo file/dir watching API #21435

Merged
merged 9 commits into from
Feb 21, 2024
Merged

[7.1.0] Repo file/dir watching API #21435

merged 9 commits into from
Feb 21, 2024

Conversation

Wyverald
Copy link
Member

RELNOTES: Added new APIs to watch arbitrary files or directory trees repository_ctx.watch, repository_ctx.watch_tree.

Marker files today store all predeclared inputs as one hash (on the first line of the file), and then each recorded input as a following line in the `TYPE:KEY VALUE` format. This commit refactors the parsing/stringification logic of recorded inputs so that they're not all clumped in big methods in `RepositoryFunction`, to pave the way for more recorded input types (watching directories, etc) and more places to write recorded input data (for the true repo cache).

- The StarlarkSemantics object is no longer treated as a recorded input (only recorded for Starlark repo rules, ignored for native repo rules), but as a predeclared input instead (i.e. hashed on the first line).
  - This slightly simplifies logic, and since the existing native repo rules are either local (local_repository, new_local_repository, local_config_platform) or being Starlarkified (the two Android repo rules), it will have minimal visible impact.
- Each type of recorded inputs is a subclass of `RepoRecordedInput`, which knows how to stringify itself, verify its own up-to-date-ness, and how to parse itself from a string.
- We also try to collect as many SkyKeys needed to verify up-to-date-ness as possible in one go and do a mass Skyframe evaluation. This avoids a fair amount of Skyframe restarts (unlikely to have super big impact on performance, but is a nice thing to do).

Work towards #20952 and #12227.

Closes #21182.

PiperOrigin-RevId: 604522692
Change-Id: Idc18ab202adb601cda47914c48642a6c9039da40
* Starlark API changes
  * New method `rctx.watch()` that watches a path. As of right now, said path must exist and be a regular file, but that will change in follow-ups.
  * Existing method `rctx.read()` gets a new optional string parameter `watch` that defaults to `auto`. This causes the file read to be watched as well by default, even if it's not addressed by a label. See `rctx.read()` docs for the semantics of `watch`.
  * Both changes apply to `mctx` as well, except that no files outside the current Bazel workspace may be watched for `mctx`.
* Marker file format changes
  * `FILE:` marker file entries can now be `FILE:` followed by either an absolute path (starts with a singular '/'), or a label-like ... thing (which is a label but has a slash in place of a colon). This is because we might be watching something inside a repo that is _not_ under a package.
  * Same change will happen to the `accumulatedFileDigests` sections of the module lockfile.
* Code changes
  * `RepoRecordedInput.File` subclasses are renamed to `FileOutsideWorkspace` (absolute paths) and `FileInsideWorkspace` (label-like things) instead to better reflect their meaning.
  * Unified the file watching/digest checking logic in SingleExtensionEvalFunction with the RepoRecordedInput stuff. Follow-ups will unify more.
  * Moved RepoRecordedInput to its own BUILD target as it's being used outside the repo fetching machinery (namely, module extensions)
  * Since `FileOutsideWorkspace` has to be an absolute path (never relative), we don't need a "base directory" for the RepoRecordedInput parser. What we really need is just the FileSystem object, which we can get from the PathPackageLocator available in Skyframe. Refactored code to reflect that.

Work towards #20952.

Closes #21230.

RELNOTES: Added a new method, `repository_ctx.watch()`, which asks Bazel to watch for changes to an arbitrary file. When said file changes, Bazel will refetch the repo.
PiperOrigin-RevId: 607097422
Change-Id: Ib96a49461deddd7f4c786bd1c227de18b2dd71d7
- `rctx.watch()` now supports watching a path even if it's a directory or nonexistent.
  - a path's status changing counts triggers a refetch.
- added `path.is_dir` so that users can tell whether a path points to a directory or a file.
- added `watch_X` parameters to the following methods, with behavior similar to the `watch` parameter in `rctx.read()`
  - `rctx.extract()`, `rctx.patch()`, `rctx.template()` (for the template file), `rctx.symlink()` (for the symlink target)

Work towards #20952.

Closes #21339.

PiperOrigin-RevId: 607415094
Change-Id: Iebb6bc28174d05277a034ba3cf5e0c9bf90ce027
- Added a new parameter `watch` to `path.readdir()` that allows one to watch for changes in entries under a given directory.
  - only names are watched; 'entry types' are not. This somewhat matches the return value of `path.readdir()`, which only contains entry names
  - same restrictions as `rctx.watch()` in terms of which paths can be watched and which cannot; similarly for `mctx`.
- Made a big-ish refactor that eliminated the two `RepoRecordedInput.File` subclasses, and pulled the difference into a separate `RepoRecordedInput.RepoCacheFriendlyPath` instead. This new path class is used by the new `RepoRecordedInput.Dirents` as well.

Work towards #20952.

Closes #21341.

PiperOrigin-RevId: 607772207
Change-Id: Ibba2b3389acd23e0a703818fec2cd58321a9b896
…SPACE

Mappings for repos defined in WORKSPACE are extremely annoying to verify given the chunked loading (we'd have to record which chunk the repo mapping was used in, and then load that chunk while verifying). This is extremely not worth the effort (especially since nobody really uses repo mappings in WORKSPACE) so we just don't record it.

This also means, during verification, we can safely use the main repo mapping without WORKSPACE (since repos defined in Bzlmod can't see stuff from WORKSPACE anyway).

Fixes #21289.

Closes #21393.

PiperOrigin-RevId: 608258493
Change-Id: Ife6a01221e6f5e1685d859eaa5acc8b370ab8483
- Added `rctx.watch_tree()` to watch a directory tree, which includes all transitive descendants' names, and if they're files, their contents.
  -  In the future we could add glob patterns to this method.
- Added a new SkyFunction DirectoryTreeDigestFunction to do the heavy lifting.
  - In the future, for performance, we could try to get this skyfunction to have a mode where it only digests stat(), to use as heuristics (similar to #21044)

Work towards #20952.

Closes #21362.

PiperOrigin-RevId: 608667062
Change-Id: Ibacbb7af4cf4d7628fe8fcf06e2c4fa50e811e4e
@iancha1992 iancha1992 added this pull request to the merge queue Feb 20, 2024
Merged via the queue into release-7.1.0 with commit 18ba449 Feb 21, 2024
34 checks passed
@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Feb 21, 2024
iancha1992 pushed a commit that referenced this pull request Mar 11, 2024
Baseline:  8f4b115

Release Notes:

+ Modify the error message that occurs when a requested target does not… (#20636)
+ Cherry-pick all presubmit.yml changes (#20736)
+ Accept labels of aliases in config_setting. (#20649)
+ Improve `use_repo_rule` error when not referencing a `repository_rule` (#20732)
+ Attempt to make main repo mapping inverse more efficient (#20633)
+ Retry binding to ipv6 localhost (#20755)
+ Print interactive sandboxed shell command with `--sandbox_debug` (#20769)
+ Fix two issues with --incompatible_sandbox_hermetic_tmp that manifested themselves when the output base was under /tmp (#20766)
+ Optimize prefetchInputs. (#20719)
+ Fix crash on `bazel mod` error containing `%` (#20651)
+ Cover missing cases during module extension label normalization (#20630)
+ Do not print errors when repository rules are interrupted (#20662)
+ Restart at most once when prepopulating repository rule environment (#20643)
+ Add profiles to the call sites of `updateRunfiles` (#20803)
+ Fixes for Bazel's own integration tests fail locally on Linux (#20822)
+ Remove unnecessary `cc_test` coverage handling (#20641)
+ Fix NPE in BzlmodRepoRuleFunction (#20829)
+ Return labels instead of strings from DescribableExecutionUnit methods. (#20788)
+ Introduce a SpawnLogContext interface. (#20842)
+ Fix `common` `.bazelrc` behavior for flag expansions (#20844)
+ Add a profiler span for fetching repositories. (#20852)
+ Make Bazel's RAM estimate container aware (#20644)
+ Auto-create deploy jars for Bazel `java_test` targets if requested (#20762)
+ Ignore read-only errors when updating the `mtime` of the `install_base` (#20648)
+ Add profiling to `remoteActionBuildingSemaphore.acquire()` (#20645)
+ DigestUtils: avoid throwing on invalid digest function name (#20650)
+ Use a larger buffer size for `java.util.zip.*Stream` classes (#20642)
+ Flip flag `--experimental_use_semaphore_for_jobs`. (#20646)
+ RemoteSpawnRunner: record inbetween phases in timing profile (#20647)
+ Add fastutil 7.2.1 dependency (#20854)
+ Allow repo rules to download multiple things in parallel. (#20856)
+ Add support for tmpfs mounts under `/tmp` with hermetic tmp (#20859)
+ Add new flag `--enable_workspace` that allows us to disable WORKSPACE… (#20855)
+ Add support for bind mounts under `/tmp` with hermetic tmp (#20772)
+ Document `--digest_function` startup flag (#20864)
+ Add flag `experimental_throttle_remote_action_building` (#20858)
+ Remove suffix from fastutil alias. (#20872)
+ Move StableSort into the exec package.
+ Test that missing spawn outputs are logged correctly.
+ Simplify computeDigest.
+ Add SpawnBuilder#with{Inputs,Tools} overloads accepting a NestedSet.
+ Correctly log paths for runfiles and filesets.
+ Report empty files in the spawn log.
+ Propagate the tool bit to logged directory inputs.
+ Move some more common logic into SpawnLogContext.
+ Introduce a new compact execution log format.
+ Apply zstd compression to the compact execution log.
+ Avoid unnecessary overhead when determining whether an action input is a directory.
+ Offer Shell completion for top-level packages without subpackages (#20879)
+ Increase maximal length of profile span for repository function calls (#20907)
+ Still generate a WORKSPACE file in repo rules if --enable_workspace is set (#20914)
+ Read authentication information from .netrc (#20915)
+ deps: rules_python 0.4.0 -> 0.22.0 (#20916)
+ Avoid emitting canonical labels into generated repos (#20917)
+ python: make incompatible_python_disallow_native_rules work for top-level external repo targets (#20923)
+ Remove flag guarding for the AndroidIdeInfo provider (#20932)
+ Point _virtual_includes to stable locations so IDE integrations survive builds (#20946)
+ [rfc] Allow repository rules to lazily declare environment variable deps (#20944)
+ Replaced usage of rev with awk in bash runfiles (#20934)
+ Cherry pick a few changes to address flaky tests (#20956)
+ Cherry-pick the change to reduce repository invalidations to Bazel 7.1 (#20949)
+ The label API shakeup & docs cleanup (#20977)
+ New docs for labels, repos, etc (#20978)
+ Add support for arbitrary headers to rctx.download[_and_extract] (#20979)
+ Show a warning message when the credential helper invocation fails (#20992)
+ Fix singlejar resource mapping for external repositories (#20989)
+ Remove user specific path from the lockfile (Fixes #19621) (#21009)
+ Also report cycles involving WORKSPACE from BzlmodRepoCycleReporter (#21013)
+ Fix -fatal_warnings on macOS (#21018)
+ Cherry-picks for module extension repo mapping usage tracking (#21033)
+ bzlmod: support git repos in source.json (#21036)
+ Add `bazel mod dump_repo_mapping` (#21023)
+ Cherry-picks for elimination of repo rule restarts (#21082)
+ Fix inconsistent dep graph stubs in Bzlmod tests (#21085)
+ Distinguish the disk and remote caches in the action progress status. (#21084)
+ Clarify where to find the definition of the --experimental_remote_scrubbing_config configuration format. (#21089)
+ Disable `--legacy_external_runfiles` in Bazel tests (#21086)
+ Follow directory symlink in RemoteActionFileSystem#getDirectoryEntries(). (#21088)
+ Treat the inability to load the Windows filesystem JNI as an error. (#21090)
+ Fix up permissions error in getInputStream, like we already do for getOutputStream. (#21087)
+ Force output checking for incremental run commands without the bytes. (#20988)
+ Remove visionos_x86_64 CPUs (#21022)
+ Close test.err before deleteing it (#21020)
+ Fix linker feature detection being performed on wrong linker (#20990)
+ Add an option to set a minimum size threshold for zstd blob compression. (#21124)
+ Publish RCs to GitHub (#21127)
+ Avoid using `InputStream.available()` to detect EOF while reading delimited protos. (#21143)
+ Starlark: reuse positional array in native calls where possible (#21144)
+ Harmonize BUILD files. (#21145)
+ Add bash completion for external targets (#21149)
+ Make some minor adjustments to the compact execution log format and document it better. (#21146)
+ Optimize the execution log sorter by using reference equality. (#21147)
+ Update to Turbine 0.4.0 (#21161)
+ Split StableSort into a separate target. (#21152)
+ Document that the compact execution log isn't guaranteed to be serialized in increasing ID order. (#21165)
+ Fix the comment for MessageOutputStream#write(). (#21166)
+ Make repo marker files sensitive to repo mapping changes (#21172)
+ Include the digest hash function in the compact execution log. (#21174)
+ Report unresolved symlinks as such in the execution log. (#21177)
+ Correctly handle unresolved symlinks when they appear in the inputs. (#21181)
+ Add missing close(). (#21183)
+ Add a profile span for building the upload manifest. (#21184)
+ Remove obsolete comments and dividers. (#21185)
+ Implement a new execution log conversion tool. (#21187)
+ Implement a new execution log conversion tool. (#21192)
+ Introduce a MessageInputStream abstraction, mirroring MessageOutputStream. (#21207)
+ Upgrade to use Bazel 7.0.2 (#21208)
+ Do not store the repository name in `RepoSpec` (#21209)
+ Make sure we build as well as test //src/tools/execlog/... on CI. (#21216)
+ Teach ExecLogConverter to read the compact format. (#21223)
+ Switch macOS minimum version flag to gcc compatible version (#21219)
+ Update default visionOS CPU to sim_arm64 (#21240)
+ Avoid exception-based control flow in RemoteActionFileSystem#stat. (#21236)
+ Cherry-pick: linker_param_file only added to command line if it starts with "@" (#21235)
+ Fixes for experimental extend rule and subrule functionality (#21237)
+ Fix NPE in ResourceManager when collecting local resource estimation in the profiler. (#21229)
+ Optimize RemoteActionFileSystem#readdir for the tree artifact input case. (#21251)
+ Document --incompatible_disallow_unsound_directory_outputs. (#21252)
+ Also path map transitive header jar paths with direct classpath optimization (#21227)
+ Error on invalid path characters in `.bazelignore` (#21259)
+ Mark gcc-<version> as `gcc` instead of `compiler` in Unix CC toolchain (#21224)
+ Avoid exception-based control flow in RAFS#getDigest and RAFS#getFastDigest. (#21264)
+ Add `add_exports/add_opens` to bazel java_binary deploy jars (#21270)
+ Manipulate the local filesystem directly in the writeLocalFile test helper. (#21272)
+ Improve the documentation for PathFragment methods dealing with segments. (#21275)
+ Canonicalize the parent path in RemoteActionFileSystem#delete. (#21282)
+ Revert "Also path map transitive header jar paths with direct classpath optimization" (#21281)
+ Make it possible to toggle cache key scrubbing by rule kind (#21276)
+ Fix a hanging issue with skymeld & `--combined_report=lcov`. (#21271)
+ Canonicalize the parent path in RemoteActionFileSystem#renameTo. (#21285)
+ Exclude `//src/test/py/bazel:mod_command_test` from RBE tests due to frequent flaky timeouts.
+ Add `bazel mod tidy` (#21265)
+ Don't use worker threads for repo fetching during Skyframe er… (#21305)
+ Fix flakiness in //src/test/shell/bazel:starlark_repository_test (#21309)
+ Document best practice of avoiding extensions directly specifying repository names (#21300)
+ Allow `@repo_name` labels in override attributes (#21313)
+ Reproducible extension (#21306)
+ Omit unique module versions from canonical repo names (#21316)
+ Add `Label.to_display_form()` (#21312)
+ Clarify the purpose and overall behavior of RemoteActionFileSystem. (#21294)
+ Make SpawnLogConvert an abstract class instead of an interface. (#21325)
+ Add support for additional command profiler event types. (#21327)
+ Remove the fileSize parameter from DigestUtils. (#21328)
+ Optimize RemoteActionFileSystem#resolveSymbolicLinks by caching intermediate results in a trie. (#21333)
+ Mark `use_repo_rule` extension as reproducible (#21335)
+ Make SpawnLogContext interruptible. (#21337)
+ Document --cache_computed_file_digests. (#21326)
+ Generate a lockfile for the distribution archive on the fly (#21338)
+ Introduce --local_resources flag (#21331)
+ Cherry-pick recent changes to fix CI flakiness and breakages (#21349)
+ Clear the file digests cache on clean. (#21346)
+ Parallelize TreeArtifactValue.visitTree across files instead of subdirectories. (#21347)
+ Temporarily hardcode rules_java repository name (#21356)
+ Remove unnecessary test assertions to fix flakiness. (#21354)
+ Make it possible to avoid an extra stat() when obtaining a digest from the cache. (#21353)
+ Collect directory contents in parallel in CompactSpawnLogContext. (#21361)
+ Introduce --default_test_resources flag (#21311)
+ python: rules_python 0.22.0 -> 0.22.1 soas to register Python toolchain by default (#21369)
+ Add vendor mode (#21366)
+ Clarify the behavior of --incompatible_remote_symlinks in the presence of a dangling symlink. (#21363)
+ Handle symlinks in a more consistent manner in UploadManifest. (#21371)
+ Set the executable bit on files in output directories uploaded to a disk or remote cache. (#21376)
+ Call out that TreeArtifactVisitor.visit is called in a nondeterministic order. (#21377)
+ Optimize out a stat call. (#21388)
+ Compute output directories in parallel when building the upload manifest. (#21386)
+ Fix rule definition environment for repo rules (#21397)
+ Share classpath `NestedSet` between full and header compile actions (#21389)
+ Emit labels in display form in Java rules (#21395)
+ Fetch refactor and mod command fix (#21385)
+ Implement `describeKey` for more actions (#21421)
+ Let scrubbed actions fall back to local execution when remote execution is enabled. (#21384)
+ Publish the new execution log format to the build event protocol. (#21417)
+ Ensure that the mtime of an AC entry is smaller, not larger, than the CAS blobs it references. (#21416)
+ Pass the name of the classpath manifest jar to JacocoCoverageRunner (#21413)
+ Traverse symlinks to directories while collecting a TreeArtifactValue. (#21418)
+ Correctly handle file inputs/outputs with directory contents in the execution log. (#21427)
+ Upgrade to async-profiler v3.0. (#21428)
+ Avoid a superfluous stat() in DigestUtil. (#21400)
+ [credentialhelper] Respect `expires` field from helper (#21429)
+ Improve performance of --reuse_sandbox_directories (#21433)
+ [credentialhelper] Update flag doc to point to more convenient usage instructions (#21441)
+ Repo file/dir watching API (#21435)
+ Clarify the meaning of Dirent.Type.UNKNOWN. (#21434)
+ Add a native image of turbine to the prebuilt Java tools (#21426)
+ Update java_tools v13.4 / rules_java 7.4.0 (#21359)
+ Automated rollback of commit b11fa7a. (#21448)
+ Remove the restriction that relative symlinks in a tree artifact may not point outside the tree. (#21449)
+ Revert "Add `Label.to_display_form()`" (#21454)
+ Do not record any repo mapping entries in the RepoMappingRecorder for WORKSPACE repo rules (#21457)
+ Reland "Also path map transitive header jar paths with direct classpath optimization" (#21458)
+ Backport CI test configs (#21456)
+ Use execution info instead of hard-coded mnemonics for Java path mapping (#21461)
+ Always decide whether to scrub an input by its effective path. (#21472)
+ Set RC branch when creating GitHub releases (#21477)
+ Fix vendor existing repo (#21487)
+ [test][windows] Export BAZEL_TEST=1 on windows (#21494)
+ Enable aar_import JNI libs to work with --android_platforms. (#21502)
+ Fix stale trash dir not cleaned up on worker creation (#21510)
+ Fix genrule autostamping in bazel (#21512)
+ Remove --host_jvm_args=-Djava.net.preferIPv6Addresses=true (#21546)
+ Passthrough HTTP headers to remote downloader service (#21503)
+ [credentialhelper] Support paths relative to `%install_base%` (#21532)
+ Update LibrariesToLinkCollector.java for .dll suffix stripping (#21524)
+ Backport changes for updating default lockfile used in integration tests. (#21547)
+ Fix a flaky test by avoiding leaking the eager capability RPC thread. (#21550)
+ Add a profiler span for the findMissingDigests call associated with an upload. (#21552)
+ Move the disk cache reads and writes into a thread pool. (#21551)
+ Lazily open files to be uploaded to an HTTP cache. (#21549)
+ Exclude convenience symlinks after changing the output base (#21505)
+ StarlarkBaseExternalContext.java: propagate error message when deleting temporary directory failed (#21555)
+ Fix `bazel fetch` by replacing query with cquery for underlying implementation (#21567)
+ Fix watching paths in undefined repos in repo rules (#21575)
+ Implicit dependencies should be visible to rule/aspect definitions in `.bzl` files in the same package (#21577)
+ Disable some tests because of JDK21 (#21595)
+ Expose the ApkInfo provider constructor to Starlark. (#21588)
+ Add multiplex sandboxing support to JavaBuilder (#21598)
+ Attempt to fix cancellation crash in repo fetching w/ worker thread (#21599)
+ Move compile StarlarkMethod back to CcModuleAPI (#21605)
+ Expose AndroidIdeInfo in android_common (#21607)

Acknowledgements:

This release contains contributions from many people at Google, as well as Alessandro Patti, Artem V. Navrotskiy, bazel.build machine account, Brentley Jones, Cameron Martin, Chi Wawng, Christian Scott, Cristin Donoso, David Ostrovsky, Ed Schouten, Fabian Meumertzheim, Gunnar Wagenknecht, Jordan Mele, Keith Smiley, lberki, Nikhil Kalige, oquenchil, Patrick Balestra, Rahul Butani, Ryan Beasley, Son Luong Ngoc, Sushain Cherivirala, thesayyn, Viktor Kustov, Xdng Yng, Xùdōng Yáng, Yannic, Yannic Bonenberger.
copybara-service bot pushed a commit that referenced this pull request Mar 11, 2024
Baseline:  8f4b115

Release Notes:

+ Modify the error message that occurs when a requested target does not… (#20636)
+ Cherry-pick all presubmit.yml changes (#20736)
+ Accept labels of aliases in config_setting. (#20649)
+ Improve `use_repo_rule` error when not referencing a `repository_rule` (#20732)
+ Attempt to make main repo mapping inverse more efficient (#20633)
+ Retry binding to ipv6 localhost (#20755)
+ Print interactive sandboxed shell command with `--sandbox_debug` (#20769)
+ Fix two issues with --incompatible_sandbox_hermetic_tmp that manifested themselves when the output base was under /tmp (#20766)
+ Optimize prefetchInputs. (#20719)
+ Fix crash on `bazel mod` error containing `%` (#20651)
+ Cover missing cases during module extension label normalization (#20630)
+ Do not print errors when repository rules are interrupted (#20662)
+ Restart at most once when prepopulating repository rule environment (#20643)
+ Add profiles to the call sites of `updateRunfiles` (#20803)
+ Fixes for Bazel's own integration tests fail locally on Linux (#20822)
+ Remove unnecessary `cc_test` coverage handling (#20641)
+ Fix NPE in BzlmodRepoRuleFunction (#20829)
+ Return labels instead of strings from DescribableExecutionUnit methods. (#20788)
+ Introduce a SpawnLogContext interface. (#20842)
+ Fix `common` `.bazelrc` behavior for flag expansions (#20844)
+ Add a profiler span for fetching repositories. (#20852)
+ Make Bazel's RAM estimate container aware (#20644)
+ Auto-create deploy jars for Bazel `java_test` targets if requested (#20762)
+ Ignore read-only errors when updating the `mtime` of the `install_base` (#20648)
+ Add profiling to `remoteActionBuildingSemaphore.acquire()` (#20645)
+ DigestUtils: avoid throwing on invalid digest function name (#20650)
+ Use a larger buffer size for `java.util.zip.*Stream` classes (#20642)
+ Flip flag `--experimental_use_semaphore_for_jobs`. (#20646)
+ RemoteSpawnRunner: record inbetween phases in timing profile (#20647)
+ Add fastutil 7.2.1 dependency (#20854)
+ Allow repo rules to download multiple things in parallel. (#20856)
+ Add support for tmpfs mounts under `/tmp` with hermetic tmp (#20859)
+ Add new flag `--enable_workspace` that allows us to disable WORKSPACE… (#20855)
+ Add support for bind mounts under `/tmp` with hermetic tmp (#20772)
+ Document `--digest_function` startup flag (#20864)
+ Add flag `experimental_throttle_remote_action_building` (#20858)
+ Remove suffix from fastutil alias. (#20872)
+ Move StableSort into the exec package.
+ Test that missing spawn outputs are logged correctly.
+ Simplify computeDigest.
+ Add SpawnBuilder#with{Inputs,Tools} overloads accepting a NestedSet.
+ Correctly log paths for runfiles and filesets.
+ Report empty files in the spawn log.
+ Propagate the tool bit to logged directory inputs.
+ Move some more common logic into SpawnLogContext.
+ Introduce a new compact execution log format.
+ Apply zstd compression to the compact execution log.
+ Avoid unnecessary overhead when determining whether an action input is a directory.
+ Offer Shell completion for top-level packages without subpackages (#20879)
+ Increase maximal length of profile span for repository function calls (#20907)
+ Still generate a WORKSPACE file in repo rules if --enable_workspace is set (#20914)
+ Read authentication information from .netrc (#20915)
+ deps: rules_python 0.4.0 -> 0.22.0 (#20916)
+ Avoid emitting canonical labels into generated repos (#20917)
+ python: make incompatible_python_disallow_native_rules work for top-level external repo targets (#20923)
+ Remove flag guarding for the AndroidIdeInfo provider (#20932)
+ Point _virtual_includes to stable locations so IDE integrations survive builds (#20946)
+ [rfc] Allow repository rules to lazily declare environment variable deps (#20944)
+ Replaced usage of rev with awk in bash runfiles (#20934)
+ Cherry pick a few changes to address flaky tests (#20956)
+ Cherry-pick the change to reduce repository invalidations to Bazel 7.1 (#20949)
+ The label API shakeup & docs cleanup (#20977)
+ New docs for labels, repos, etc (#20978)
+ Add support for arbitrary headers to rctx.download[_and_extract] (#20979)
+ Show a warning message when the credential helper invocation fails (#20992)
+ Fix singlejar resource mapping for external repositories (#20989)
+ Remove user specific path from the lockfile (Fixes #19621) (#21009)
+ Also report cycles involving WORKSPACE from BzlmodRepoCycleReporter (#21013)
+ Fix -fatal_warnings on macOS (#21018)
+ Cherry-picks for module extension repo mapping usage tracking (#21033)
+ bzlmod: support git repos in source.json (#21036)
+ Add `bazel mod dump_repo_mapping` (#21023)
+ Cherry-picks for elimination of repo rule restarts (#21082)
+ Fix inconsistent dep graph stubs in Bzlmod tests (#21085)
+ Distinguish the disk and remote caches in the action progress status. (#21084)
+ Clarify where to find the definition of the --experimental_remote_scrubbing_config configuration format. (#21089)
+ Disable `--legacy_external_runfiles` in Bazel tests (#21086)
+ Follow directory symlink in RemoteActionFileSystem#getDirectoryEntries(). (#21088)
+ Treat the inability to load the Windows filesystem JNI as an error. (#21090)
+ Fix up permissions error in getInputStream, like we already do for getOutputStream. (#21087)
+ Force output checking for incremental run commands without the bytes. (#20988)
+ Remove visionos_x86_64 CPUs (#21022)
+ Close test.err before deleteing it (#21020)
+ Fix linker feature detection being performed on wrong linker (#20990)
+ Add an option to set a minimum size threshold for zstd blob compression. (#21124)
+ Publish RCs to GitHub (#21127)
+ Avoid using `InputStream.available()` to detect EOF while reading delimited protos. (#21143)
+ Starlark: reuse positional array in native calls where possible (#21144)
+ Harmonize BUILD files. (#21145)
+ Add bash completion for external targets (#21149)
+ Make some minor adjustments to the compact execution log format and document it better. (#21146)
+ Optimize the execution log sorter by using reference equality. (#21147)
+ Update to Turbine 0.4.0 (#21161)
+ Split StableSort into a separate target. (#21152)
+ Document that the compact execution log isn't guaranteed to be serialized in increasing ID order. (#21165)
+ Fix the comment for MessageOutputStream#write(). (#21166)
+ Make repo marker files sensitive to repo mapping changes (#21172)
+ Include the digest hash function in the compact execution log. (#21174)
+ Report unresolved symlinks as such in the execution log. (#21177)
+ Correctly handle unresolved symlinks when they appear in the inputs. (#21181)
+ Add missing close(). (#21183)
+ Add a profile span for building the upload manifest. (#21184)
+ Remove obsolete comments and dividers. (#21185)
+ Implement a new execution log conversion tool. (#21187)
+ Implement a new execution log conversion tool. (#21192)
+ Introduce a MessageInputStream abstraction, mirroring MessageOutputStream. (#21207)
+ Upgrade to use Bazel 7.0.2 (#21208)
+ Do not store the repository name in `RepoSpec` (#21209)
+ Make sure we build as well as test //src/tools/execlog/... on CI. (#21216)
+ Teach ExecLogConverter to read the compact format. (#21223)
+ Switch macOS minimum version flag to gcc compatible version (#21219)
+ Update default visionOS CPU to sim_arm64 (#21240)
+ Avoid exception-based control flow in RemoteActionFileSystem#stat. (#21236)
+ Cherry-pick: linker_param_file only added to command line if it starts with "@" (#21235)
+ Fixes for experimental extend rule and subrule functionality (#21237)
+ Fix NPE in ResourceManager when collecting local resource estimation in the profiler. (#21229)
+ Optimize RemoteActionFileSystem#readdir for the tree artifact input case. (#21251)
+ Document --incompatible_disallow_unsound_directory_outputs. (#21252)
+ Also path map transitive header jar paths with direct classpath optimization (#21227)
+ Error on invalid path characters in `.bazelignore` (#21259)
+ Mark gcc-<version> as `gcc` instead of `compiler` in Unix CC toolchain (#21224)
+ Avoid exception-based control flow in RAFS#getDigest and RAFS#getFastDigest. (#21264)
+ Add `add_exports/add_opens` to bazel java_binary deploy jars (#21270)
+ Manipulate the local filesystem directly in the writeLocalFile test helper. (#21272)
+ Improve the documentation for PathFragment methods dealing with segments. (#21275)
+ Canonicalize the parent path in RemoteActionFileSystem#delete. (#21282)
+ Revert "Also path map transitive header jar paths with direct classpath optimization" (#21281)
+ Make it possible to toggle cache key scrubbing by rule kind (#21276)
+ Fix a hanging issue with skymeld & `--combined_report=lcov`. (#21271)
+ Canonicalize the parent path in RemoteActionFileSystem#renameTo. (#21285)
+ Exclude `//src/test/py/bazel:mod_command_test` from RBE tests due to frequent flaky timeouts.
+ Add `bazel mod tidy` (#21265)
+ Don't use worker threads for repo fetching during Skyframe er… (#21305)
+ Fix flakiness in //src/test/shell/bazel:starlark_repository_test (#21309)
+ Document best practice of avoiding extensions directly specifying repository names (#21300)
+ Allow `@repo_name` labels in override attributes (#21313)
+ Reproducible extension (#21306)
+ Omit unique module versions from canonical repo names (#21316)
+ Add `Label.to_display_form()` (#21312)
+ Clarify the purpose and overall behavior of RemoteActionFileSystem. (#21294)
+ Make SpawnLogConvert an abstract class instead of an interface. (#21325)
+ Add support for additional command profiler event types. (#21327)
+ Remove the fileSize parameter from DigestUtils. (#21328)
+ Optimize RemoteActionFileSystem#resolveSymbolicLinks by caching intermediate results in a trie. (#21333)
+ Mark `use_repo_rule` extension as reproducible (#21335)
+ Make SpawnLogContext interruptible. (#21337)
+ Document --cache_computed_file_digests. (#21326)
+ Generate a lockfile for the distribution archive on the fly (#21338)
+ Introduce --local_resources flag (#21331)
+ Cherry-pick recent changes to fix CI flakiness and breakages (#21349)
+ Clear the file digests cache on clean. (#21346)
+ Parallelize TreeArtifactValue.visitTree across files instead of subdirectories. (#21347)
+ Temporarily hardcode rules_java repository name (#21356)
+ Remove unnecessary test assertions to fix flakiness. (#21354)
+ Make it possible to avoid an extra stat() when obtaining a digest from the cache. (#21353)
+ Collect directory contents in parallel in CompactSpawnLogContext. (#21361)
+ Introduce --default_test_resources flag (#21311)
+ python: rules_python 0.22.0 -> 0.22.1 soas to register Python toolchain by default (#21369)
+ Add vendor mode (#21366)
+ Clarify the behavior of --incompatible_remote_symlinks in the presence of a dangling symlink. (#21363)
+ Handle symlinks in a more consistent manner in UploadManifest. (#21371)
+ Set the executable bit on files in output directories uploaded to a disk or remote cache. (#21376)
+ Call out that TreeArtifactVisitor.visit is called in a nondeterministic order. (#21377)
+ Optimize out a stat call. (#21388)
+ Compute output directories in parallel when building the upload manifest. (#21386)
+ Fix rule definition environment for repo rules (#21397)
+ Share classpath `NestedSet` between full and header compile actions (#21389)
+ Emit labels in display form in Java rules (#21395)
+ Fetch refactor and mod command fix (#21385)
+ Implement `describeKey` for more actions (#21421)
+ Let scrubbed actions fall back to local execution when remote execution is enabled. (#21384)
+ Publish the new execution log format to the build event protocol. (#21417)
+ Ensure that the mtime of an AC entry is smaller, not larger, than the CAS blobs it references. (#21416)
+ Pass the name of the classpath manifest jar to JacocoCoverageRunner (#21413)
+ Traverse symlinks to directories while collecting a TreeArtifactValue. (#21418)
+ Correctly handle file inputs/outputs with directory contents in the execution log. (#21427)
+ Upgrade to async-profiler v3.0. (#21428)
+ Avoid a superfluous stat() in DigestUtil. (#21400)
+ [credentialhelper] Respect `expires` field from helper (#21429)
+ Improve performance of --reuse_sandbox_directories (#21433)
+ [credentialhelper] Update flag doc to point to more convenient usage instructions (#21441)
+ Repo file/dir watching API (#21435)
+ Clarify the meaning of Dirent.Type.UNKNOWN. (#21434)
+ Add a native image of turbine to the prebuilt Java tools (#21426)
+ Update java_tools v13.4 / rules_java 7.4.0 (#21359)
+ Automated rollback of commit b11fa7a. (#21448)
+ Remove the restriction that relative symlinks in a tree artifact may not point outside the tree. (#21449)
+ Revert "Add `Label.to_display_form()`" (#21454)
+ Do not record any repo mapping entries in the RepoMappingRecorder for WORKSPACE repo rules (#21457)
+ Reland "Also path map transitive header jar paths with direct classpath optimization" (#21458)
+ Backport CI test configs (#21456)
+ Use execution info instead of hard-coded mnemonics for Java path mapping (#21461)
+ Always decide whether to scrub an input by its effective path. (#21472)
+ Set RC branch when creating GitHub releases (#21477)
+ Fix vendor existing repo (#21487)
+ [test][windows] Export BAZEL_TEST=1 on windows (#21494)
+ Enable aar_import JNI libs to work with --android_platforms. (#21502)
+ Fix stale trash dir not cleaned up on worker creation (#21510)
+ Fix genrule autostamping in bazel (#21512)
+ Remove --host_jvm_args=-Djava.net.preferIPv6Addresses=true (#21546)
+ Passthrough HTTP headers to remote downloader service (#21503)
+ [credentialhelper] Support paths relative to `%install_base%` (#21532)
+ Update LibrariesToLinkCollector.java for .dll suffix stripping (#21524)
+ Backport changes for updating default lockfile used in integration tests. (#21547)
+ Fix a flaky test by avoiding leaking the eager capability RPC thread. (#21550)
+ Add a profiler span for the findMissingDigests call associated with an upload. (#21552)
+ Move the disk cache reads and writes into a thread pool. (#21551)
+ Lazily open files to be uploaded to an HTTP cache. (#21549)
+ Exclude convenience symlinks after changing the output base (#21505)
+ StarlarkBaseExternalContext.java: propagate error message when deleting temporary directory failed (#21555)
+ Fix `bazel fetch` by replacing query with cquery for underlying implementation (#21567)
+ Fix watching paths in undefined repos in repo rules (#21575)
+ Implicit dependencies should be visible to rule/aspect definitions in `.bzl` files in the same package (#21577)
+ Disable some tests because of JDK21 (#21595)
+ Expose the ApkInfo provider constructor to Starlark. (#21588)
+ Add multiplex sandboxing support to JavaBuilder (#21598)
+ Attempt to fix cancellation crash in repo fetching w/ worker thread (#21599)
+ Move compile StarlarkMethod back to CcModuleAPI (#21605)
+ Expose AndroidIdeInfo in android_common (#21607)

Acknowledgements:

This release contains contributions from many people at Google, as well as Alessandro Patti, Artem V. Navrotskiy, bazel.build machine account, Brentley Jones, Cameron Martin, Chi Wawng, Christian Scott, Cristin Donoso, David Ostrovsky, Ed Schouten, Fabian Meumertzheim, Gunnar Wagenknecht, Jordan Mele, Keith Smiley, lberki, Nikhil Kalige, oquenchil, Patrick Balestra, Rahul Butani, Ryan Beasley, Son Luong Ngoc, Sushain Cherivirala, thesayyn, Viktor Kustov, Xdng Yng, Xùdōng Yáng, Yannic, Yannic Bonenberger.
@Wyverald Wyverald deleted the wyv-cp-watch branch March 14, 2024 20:46
iancha1992 pushed a commit that referenced this pull request May 8, 2024
Baseline:  d798ebd

Release Notes:

+ Consider MODULE.bazel for workspace detection in bazel.sh (#20594)
+ Auto-create deploy jars for Bazel `java_test` targets if requested (#20602)
+ `java_binary` wrapper should forward `restricted_to` (#20611)
+ Mount user-specified bind mounts before Bazel's own magic. (#20609)
+ Fix bootstrapped Bazel binary (#20612)
+ Modify the error message that occurs when a requested target does not… (#20636)
+ Cherry-pick all presubmit.yml changes (#20736)
+ Accept labels of aliases in config_setting. (#20649)
+ Improve `use_repo_rule` error when not referencing a `repository_rule` (#20732)
+ Attempt to make main repo mapping inverse more efficient (#20633)
+ Retry binding to ipv6 localhost (#20755)
+ Print interactive sandboxed shell command with `--sandbox_debug` (#20769)
+ Fix two issues with --incompatible_sandbox_hermetic_tmp that manifested themselves when the output base was under /tmp (#20766)
+ Optimize prefetchInputs. (#20719)
+ Fix crash on `bazel mod` error containing `%` (#20651)
+ Cover missing cases during module extension label normalization (#20630)
+ Do not print errors when repository rules are interrupted (#20662)
+ Restart at most once when prepopulating repository rule environment (#20643)
+ Add profiles to the call sites of `updateRunfiles` (#20803)
+ Fixes for Bazel's own integration tests fail locally on Linux (#20822)
+ Remove unnecessary `cc_test` coverage handling (#20641)
+ Fix NPE in BzlmodRepoRuleFunction (#20829)
+ Return labels instead of strings from DescribableExecutionUnit methods. (#20788)
+ Introduce a SpawnLogContext interface. (#20842)
+ Fix `common` `.bazelrc` behavior for flag expansions (#20844)
+ Add a profiler span for fetching repositories. (#20852)
+ Make Bazel's RAM estimate container aware (#20644)
+ Auto-create deploy jars for Bazel `java_test` targets if requested (#20762)
+ Ignore read-only errors when updating the `mtime` of the `install_base` (#20648)
+ Add profiling to `remoteActionBuildingSemaphore.acquire()` (#20645)
+ DigestUtils: avoid throwing on invalid digest function name (#20650)
+ Use a larger buffer size for `java.util.zip.*Stream` classes (#20642)
+ Flip flag `--experimental_use_semaphore_for_jobs`. (#20646)
+ RemoteSpawnRunner: record inbetween phases in timing profile (#20647)
+ Add fastutil 7.2.1 dependency (#20854)
+ Allow repo rules to download multiple things in parallel. (#20856)
+ Add support for tmpfs mounts under `/tmp` with hermetic tmp (#20859)
+ Add new flag `--enable_workspace` that allows us to disable WORKSPACE… (#20855)
+ Add support for bind mounts under `/tmp` with hermetic tmp (#20772)
+ Document `--digest_function` startup flag (#20864)
+ Add flag `experimental_throttle_remote_action_building` (#20858)
+ Remove suffix from fastutil alias. (#20872)
+ Move StableSort into the exec package.
+ Test that missing spawn outputs are logged correctly.
+ Simplify computeDigest.
+ Add SpawnBuilder#with{Inputs,Tools} overloads accepting a NestedSet.
+ Correctly log paths for runfiles and filesets.
+ Report empty files in the spawn log.
+ Propagate the tool bit to logged directory inputs.
+ Move some more common logic into SpawnLogContext.
+ Introduce a new compact execution log format.
+ Apply zstd compression to the compact execution log.
+ Avoid unnecessary overhead when determining whether an action input is a directory.
+ Offer Shell completion for top-level packages without subpackages (#20879)
+ Increase maximal length of profile span for repository function calls (#20907)
+ Still generate a WORKSPACE file in repo rules if --enable_workspace is set (#20914)
+ Read authentication information from .netrc (#20915)
+ deps: rules_python 0.4.0 -> 0.22.0 (#20916)
+ Avoid emitting canonical labels into generated repos (#20917)
+ python: make incompatible_python_disallow_native_rules work for top-level external repo targets (#20923)
+ Remove flag guarding for the AndroidIdeInfo provider (#20932)
+ Point _virtual_includes to stable locations so IDE integrations survive builds (#20946)
+ [rfc] Allow repository rules to lazily declare environment variable deps (#20944)
+ Replaced usage of rev with awk in bash runfiles (#20934)
+ Cherry pick a few changes to address flaky tests (#20956)
+ Cherry-pick the change to reduce repository invalidations to Bazel 7.1 (#20949)
+ The label API shakeup & docs cleanup (#20977)
+ New docs for labels, repos, etc (#20978)
+ Add support for arbitrary headers to rctx.download[_and_extract] (#20979)
+ Show a warning message when the credential helper invocation fails (#20992)
+ Fix singlejar resource mapping for external repositories (#20989)
+ Remove user specific path from the lockfile (Fixes #19621) (#21009)
+ Also report cycles involving WORKSPACE from BzlmodRepoCycleReporter (#21013)
+ Fix -fatal_warnings on macOS (#21018)
+ Cherry-picks for module extension repo mapping usage tracking (#21033)
+ bzlmod: support git repos in source.json (#21036)
+ Add `bazel mod dump_repo_mapping` (#21023)
+ Cherry-picks for elimination of repo rule restarts (#21082)
+ Fix inconsistent dep graph stubs in Bzlmod tests (#21085)
+ Distinguish the disk and remote caches in the action progress status. (#21084)
+ Clarify where to find the definition of the --experimental_remote_scrubbing_config configuration format. (#21089)
+ Disable `--legacy_external_runfiles` in Bazel tests (#21086)
+ Follow directory symlink in RemoteActionFileSystem#getDirectoryEntries(). (#21088)
+ Treat the inability to load the Windows filesystem JNI as an error. (#21090)
+ Fix up permissions error in getInputStream, like we already do for getOutputStream. (#21087)
+ Force output checking for incremental run commands without the bytes. (#20988)
+ Remove visionos_x86_64 CPUs (#21022)
+ Close test.err before deleteing it (#21020)
+ Fix linker feature detection being performed on wrong linker (#20990)
+ Add an option to set a minimum size threshold for zstd blob compression. (#21124)
+ Publish RCs to GitHub (#21127)
+ Avoid using `InputStream.available()` to detect EOF while reading delimited protos. (#21143)
+ Starlark: reuse positional array in native calls where possible (#21144)
+ Harmonize BUILD files. (#21145)
+ Add bash completion for external targets (#21149)
+ Make some minor adjustments to the compact execution log format and document it better. (#21146)
+ Optimize the execution log sorter by using reference equality. (#21147)
+ Update to Turbine 0.4.0 (#21161)
+ Split StableSort into a separate target. (#21152)
+ Document that the compact execution log isn't guaranteed to be serialized in increasing ID order. (#21165)
+ Fix the comment for MessageOutputStream#write(). (#21166)
+ Make repo marker files sensitive to repo mapping changes (#21172)
+ Include the digest hash function in the compact execution log. (#21174)
+ Report unresolved symlinks as such in the execution log. (#21177)
+ Correctly handle unresolved symlinks when they appear in the inputs. (#21181)
+ Add missing close(). (#21183)
+ Add a profile span for building the upload manifest. (#21184)
+ Remove obsolete comments and dividers. (#21185)
+ Implement a new execution log conversion tool. (#21187)
+ Implement a new execution log conversion tool. (#21192)
+ Introduce a MessageInputStream abstraction, mirroring MessageOutputStream. (#21207)
+ Upgrade to use Bazel 7.0.2 (#21208)
+ Do not store the repository name in `RepoSpec` (#21209)
+ Make sure we build as well as test //src/tools/execlog/... on CI. (#21216)
+ Teach ExecLogConverter to read the compact format. (#21223)
+ Switch macOS minimum version flag to gcc compatible version (#21219)
+ Update default visionOS CPU to sim_arm64 (#21240)
+ Avoid exception-based control flow in RemoteActionFileSystem#stat. (#21236)
+ Cherry-pick: linker_param_file only added to command line if it starts with "@" (#21235)
+ Fixes for experimental extend rule and subrule functionality (#21237)
+ Fix NPE in ResourceManager when collecting local resource estimation in the profiler. (#21229)
+ Optimize RemoteActionFileSystem#readdir for the tree artifact input case. (#21251)
+ Document --incompatible_disallow_unsound_directory_outputs. (#21252)
+ Also path map transitive header jar paths with direct classpath optimization (#21227)
+ Error on invalid path characters in `.bazelignore` (#21259)
+ Mark gcc-<version> as `gcc` instead of `compiler` in Unix CC toolchain (#21224)
+ Avoid exception-based control flow in RAFS#getDigest and RAFS#getFastDigest. (#21264)
+ Add `add_exports/add_opens` to bazel java_binary deploy jars (#21270)
+ Manipulate the local filesystem directly in the writeLocalFile test helper. (#21272)
+ Improve the documentation for PathFragment methods dealing with segments. (#21275)
+ Canonicalize the parent path in RemoteActionFileSystem#delete. (#21282)
+ Revert "Also path map transitive header jar paths with direct classpath optimization" (#21281)
+ Make it possible to toggle cache key scrubbing by rule kind (#21276)
+ Fix a hanging issue with skymeld & `--combined_report=lcov`. (#21271)
+ Canonicalize the parent path in RemoteActionFileSystem#renameTo. (#21285)
+ Exclude `//src/test/py/bazel:mod_command_test` from RBE tests due to frequent flaky timeouts.
+ Add `bazel mod tidy` (#21265)
+ Don't use worker threads for repo fetching during Skyframe er… (#21305)
+ Fix flakiness in //src/test/shell/bazel:starlark_repository_test (#21309)
+ Document best practice of avoiding extensions directly specifying repository names (#21300)
+ Allow `@repo_name` labels in override attributes (#21313)
+ Reproducible extension (#21306)
+ Omit unique module versions from canonical repo names (#21316)
+ Add `Label.to_display_form()` (#21312)
+ Clarify the purpose and overall behavior of RemoteActionFileSystem. (#21294)
+ Make SpawnLogConvert an abstract class instead of an interface. (#21325)
+ Add support for additional command profiler event types. (#21327)
+ Remove the fileSize parameter from DigestUtils. (#21328)
+ Optimize RemoteActionFileSystem#resolveSymbolicLinks by caching intermediate results in a trie. (#21333)
+ Mark `use_repo_rule` extension as reproducible (#21335)
+ Make SpawnLogContext interruptible. (#21337)
+ Document --cache_computed_file_digests. (#21326)
+ Generate a lockfile for the distribution archive on the fly (#21338)
+ Introduce --local_resources flag (#21331)
+ Cherry-pick recent changes to fix CI flakiness and breakages (#21349)
+ Clear the file digests cache on clean. (#21346)
+ Parallelize TreeArtifactValue.visitTree across files instead of subdirectories. (#21347)
+ Temporarily hardcode rules_java repository name (#21356)
+ Remove unnecessary test assertions to fix flakiness. (#21354)
+ Make it possible to avoid an extra stat() when obtaining a digest from the cache. (#21353)
+ Collect directory contents in parallel in CompactSpawnLogContext. (#21361)
+ Introduce --default_test_resources flag (#21311)
+ python: rules_python 0.22.0 -> 0.22.1 soas to register Python toolchain by default (#21369)
+ Add vendor mode (#21366)
+ Clarify the behavior of --incompatible_remote_symlinks in the presence of a dangling symlink. (#21363)
+ Handle symlinks in a more consistent manner in UploadManifest. (#21371)
+ Set the executable bit on files in output directories uploaded to a disk or remote cache. (#21376)
+ Call out that TreeArtifactVisitor.visit is called in a nondeterministic order. (#21377)
+ Optimize out a stat call. (#21388)
+ Compute output directories in parallel when building the upload manifest. (#21386)
+ Fix rule definition environment for repo rules (#21397)
+ Share classpath `NestedSet` between full and header compile actions (#21389)
+ Emit labels in display form in Java rules (#21395)
+ Fetch refactor and mod command fix (#21385)
+ Implement `describeKey` for more actions (#21421)
+ Let scrubbed actions fall back to local execution when remote execution is enabled. (#21384)
+ Publish the new execution log format to the build event protocol. (#21417)
+ Ensure that the mtime of an AC entry is smaller, not larger, than the CAS blobs it references. (#21416)
+ Pass the name of the classpath manifest jar to JacocoCoverageRunner (#21413)
+ Traverse symlinks to directories while collecting a TreeArtifactValue. (#21418)
+ Correctly handle file inputs/outputs with directory contents in the execution log. (#21427)
+ Upgrade to async-profiler v3.0. (#21428)
+ Avoid a superfluous stat() in DigestUtil. (#21400)
+ [credentialhelper] Respect `expires` field from helper (#21429)
+ Improve performance of --reuse_sandbox_directories (#21433)
+ [credentialhelper] Update flag doc to point to more convenient usage instructions (#21441)
+ Repo file/dir watching API (#21435)
+ Clarify the meaning of Dirent.Type.UNKNOWN. (#21434)
+ Add a native image of turbine to the prebuilt Java tools (#21426)
+ Update java_tools v13.4 / rules_java 7.4.0 (#21359)
+ Automated rollback of commit b11fa7a. (#21448)
+ Remove the restriction that relative symlinks in a tree artifact may not point outside the tree. (#21449)
+ Revert "Add `Label.to_display_form()`" (#21454)
+ Do not record any repo mapping entries in the RepoMappingRecorder for WORKSPACE repo rules (#21457)
+ Reland "Also path map transitive header jar paths with direct classpath optimization" (#21458)
+ Backport CI test configs (#21456)
+ Use execution info instead of hard-coded mnemonics for Java path mapping (#21461)
+ Always decide whether to scrub an input by its effective path. (#21472)
+ Set RC branch when creating GitHub releases (#21477)
+ Fix vendor existing repo (#21487)
+ [test][windows] Export BAZEL_TEST=1 on windows (#21494)
+ Enable aar_import JNI libs to work with --android_platforms. (#21502)
+ Fix stale trash dir not cleaned up on worker creation (#21510)
+ Fix genrule autostamping in bazel (#21512)
+ Remove --host_jvm_args=-Djava.net.preferIPv6Addresses=true (#21546)
+ Passthrough HTTP headers to remote downloader service (#21503)
+ [credentialhelper] Support paths relative to `%install_base%` (#21532)
+ Update LibrariesToLinkCollector.java for .dll suffix stripping (#21524)
+ Backport changes for updating default lockfile used in integration tests. (#21547)
+ Fix a flaky test by avoiding leaking the eager capability RPC thread. (#21550)
+ Add a profiler span for the findMissingDigests call associated with an upload. (#21552)
+ Move the disk cache reads and writes into a thread pool. (#21551)
+ Lazily open files to be uploaded to an HTTP cache. (#21549)
+ Exclude convenience symlinks after changing the output base (#21505)
+ StarlarkBaseExternalContext.java: propagate error message when deleting temporary directory failed (#21555)
+ Fix `bazel fetch` by replacing query with cquery for underlying implementation (#21567)
+ Fix watching paths in undefined repos in repo rules (#21575)
+ Implicit dependencies should be visible to rule/aspect definitions in `.bzl` files in the same package (#21577)
+ Disable some tests because of JDK21 (#21595)
+ Expose the ApkInfo provider constructor to Starlark. (#21588)
+ Add multiplex sandboxing support to JavaBuilder (#21598)
+ Attempt to fix cancellation crash in repo fetching w/ worker thread (#21599)
+ Move compile StarlarkMethod back to CcModuleAPI (#21605)
+ Expose AndroidIdeInfo in android_common (#21607)
+ Release 7.1.0 (2024-03-11)
+ Update centos7 platform in build_bazel_binaries.yml (#21644)
+ Fix `bazel mod tidy` failure with no changes (#21662)
+ Update .bazelversion to 7.1.0 (#21664)
+ Let native Turbine image find `ct.sym` with non-hermetic `java_runtime` (#21670)
+ Actually use shouldPublish() to determine whether to publish the execution log to the BEP. (#21671)
+ Also inject a failure for createWritableDirectory when testing that ActionOutputDirectoryHelper propagates exceptions. (#21683)
+ Fix race condition and add more logging for null entry error message (#21692)
+ Allow any canonical repo name to be used with `bazel mod show_repo` (#21694)
+ Fix two `bazel mod tidy` crashes (#21700)
+ Cherry-pick Java execution info improvements (#21703)
+ Disable //src/test/shell/bazel:srcs_test on Intel macOS (#21707)
+ Fix sandbox cleanup crashing after server restart (#21733)
+ Revert "Fix `bazel fetch` by replacing query with cquery for … (#21735)
+ Release 7.1.1 (2024-03-21)
+ Implement RemoteActionFileSystem#statIfFound correctly when the path cannot be canonicalized (#21889)
+ Don't upload remote input to remote cache (#21941)
+ Do not watch `.netrc` in `read_netrc` (#22186)
+ Set public visibility for R8 desugar binary (#22176)

Acknowledgements:

This release contains contributions from many people at Google, as well as Alessandro Patti, Artem V. Navrotskiy, bazel.build machine account, Brentley Jones, Cameron Martin, Chi Wawng, Christian Scott, Cristin Donoso, David Ostrovsky, Ed Schouten, Fabian Meumertzheim, Gunnar Wagenknecht, Jordan Mele, Keith Smiley, lberki, Nikhil Kalige, oquenchil, Patrick Balestra, Rahul Butani, Ryan Beasley, Siddhartha Bagaria, Son Luong Ngoc, Sushain Cherivirala, thesayyn, Tianyu Geng, Viktor Kustov, Xdng Yng, Xùdōng Yáng, Yannic, Yannic Bonenberger.
copybara-service bot pushed a commit that referenced this pull request May 8, 2024
Baseline:  d798ebd

Release Notes:

+ Consider MODULE.bazel for workspace detection in bazel.sh (#20594)
+ Auto-create deploy jars for Bazel `java_test` targets if requested (#20602)
+ `java_binary` wrapper should forward `restricted_to` (#20611)
+ Mount user-specified bind mounts before Bazel's own magic. (#20609)
+ Fix bootstrapped Bazel binary (#20612)
+ Modify the error message that occurs when a requested target does not… (#20636)
+ Cherry-pick all presubmit.yml changes (#20736)
+ Accept labels of aliases in config_setting. (#20649)
+ Improve `use_repo_rule` error when not referencing a `repository_rule` (#20732)
+ Attempt to make main repo mapping inverse more efficient (#20633)
+ Retry binding to ipv6 localhost (#20755)
+ Print interactive sandboxed shell command with `--sandbox_debug` (#20769)
+ Fix two issues with --incompatible_sandbox_hermetic_tmp that manifested themselves when the output base was under /tmp (#20766)
+ Optimize prefetchInputs. (#20719)
+ Fix crash on `bazel mod` error containing `%` (#20651)
+ Cover missing cases during module extension label normalization (#20630)
+ Do not print errors when repository rules are interrupted (#20662)
+ Restart at most once when prepopulating repository rule environment (#20643)
+ Add profiles to the call sites of `updateRunfiles` (#20803)
+ Fixes for Bazel's own integration tests fail locally on Linux (#20822)
+ Remove unnecessary `cc_test` coverage handling (#20641)
+ Fix NPE in BzlmodRepoRuleFunction (#20829)
+ Return labels instead of strings from DescribableExecutionUnit methods. (#20788)
+ Introduce a SpawnLogContext interface. (#20842)
+ Fix `common` `.bazelrc` behavior for flag expansions (#20844)
+ Add a profiler span for fetching repositories. (#20852)
+ Make Bazel's RAM estimate container aware (#20644)
+ Auto-create deploy jars for Bazel `java_test` targets if requested (#20762)
+ Ignore read-only errors when updating the `mtime` of the `install_base` (#20648)
+ Add profiling to `remoteActionBuildingSemaphore.acquire()` (#20645)
+ DigestUtils: avoid throwing on invalid digest function name (#20650)
+ Use a larger buffer size for `java.util.zip.*Stream` classes (#20642)
+ Flip flag `--experimental_use_semaphore_for_jobs`. (#20646)
+ RemoteSpawnRunner: record inbetween phases in timing profile (#20647)
+ Add fastutil 7.2.1 dependency (#20854)
+ Allow repo rules to download multiple things in parallel. (#20856)
+ Add support for tmpfs mounts under `/tmp` with hermetic tmp (#20859)
+ Add new flag `--enable_workspace` that allows us to disable WORKSPACE… (#20855)
+ Add support for bind mounts under `/tmp` with hermetic tmp (#20772)
+ Document `--digest_function` startup flag (#20864)
+ Add flag `experimental_throttle_remote_action_building` (#20858)
+ Remove suffix from fastutil alias. (#20872)
+ Move StableSort into the exec package.
+ Test that missing spawn outputs are logged correctly.
+ Simplify computeDigest.
+ Add SpawnBuilder#with{Inputs,Tools} overloads accepting a NestedSet.
+ Correctly log paths for runfiles and filesets.
+ Report empty files in the spawn log.
+ Propagate the tool bit to logged directory inputs.
+ Move some more common logic into SpawnLogContext.
+ Introduce a new compact execution log format.
+ Apply zstd compression to the compact execution log.
+ Avoid unnecessary overhead when determining whether an action input is a directory.
+ Offer Shell completion for top-level packages without subpackages (#20879)
+ Increase maximal length of profile span for repository function calls (#20907)
+ Still generate a WORKSPACE file in repo rules if --enable_workspace is set (#20914)
+ Read authentication information from .netrc (#20915)
+ deps: rules_python 0.4.0 -> 0.22.0 (#20916)
+ Avoid emitting canonical labels into generated repos (#20917)
+ python: make incompatible_python_disallow_native_rules work for top-level external repo targets (#20923)
+ Remove flag guarding for the AndroidIdeInfo provider (#20932)
+ Point _virtual_includes to stable locations so IDE integrations survive builds (#20946)
+ [rfc] Allow repository rules to lazily declare environment variable deps (#20944)
+ Replaced usage of rev with awk in bash runfiles (#20934)
+ Cherry pick a few changes to address flaky tests (#20956)
+ Cherry-pick the change to reduce repository invalidations to Bazel 7.1 (#20949)
+ The label API shakeup & docs cleanup (#20977)
+ New docs for labels, repos, etc (#20978)
+ Add support for arbitrary headers to rctx.download[_and_extract] (#20979)
+ Show a warning message when the credential helper invocation fails (#20992)
+ Fix singlejar resource mapping for external repositories (#20989)
+ Remove user specific path from the lockfile (Fixes #19621) (#21009)
+ Also report cycles involving WORKSPACE from BzlmodRepoCycleReporter (#21013)
+ Fix -fatal_warnings on macOS (#21018)
+ Cherry-picks for module extension repo mapping usage tracking (#21033)
+ bzlmod: support git repos in source.json (#21036)
+ Add `bazel mod dump_repo_mapping` (#21023)
+ Cherry-picks for elimination of repo rule restarts (#21082)
+ Fix inconsistent dep graph stubs in Bzlmod tests (#21085)
+ Distinguish the disk and remote caches in the action progress status. (#21084)
+ Clarify where to find the definition of the --experimental_remote_scrubbing_config configuration format. (#21089)
+ Disable `--legacy_external_runfiles` in Bazel tests (#21086)
+ Follow directory symlink in RemoteActionFileSystem#getDirectoryEntries(). (#21088)
+ Treat the inability to load the Windows filesystem JNI as an error. (#21090)
+ Fix up permissions error in getInputStream, like we already do for getOutputStream. (#21087)
+ Force output checking for incremental run commands without the bytes. (#20988)
+ Remove visionos_x86_64 CPUs (#21022)
+ Close test.err before deleteing it (#21020)
+ Fix linker feature detection being performed on wrong linker (#20990)
+ Add an option to set a minimum size threshold for zstd blob compression. (#21124)
+ Publish RCs to GitHub (#21127)
+ Avoid using `InputStream.available()` to detect EOF while reading delimited protos. (#21143)
+ Starlark: reuse positional array in native calls where possible (#21144)
+ Harmonize BUILD files. (#21145)
+ Add bash completion for external targets (#21149)
+ Make some minor adjustments to the compact execution log format and document it better. (#21146)
+ Optimize the execution log sorter by using reference equality. (#21147)
+ Update to Turbine 0.4.0 (#21161)
+ Split StableSort into a separate target. (#21152)
+ Document that the compact execution log isn't guaranteed to be serialized in increasing ID order. (#21165)
+ Fix the comment for MessageOutputStream#write(). (#21166)
+ Make repo marker files sensitive to repo mapping changes (#21172)
+ Include the digest hash function in the compact execution log. (#21174)
+ Report unresolved symlinks as such in the execution log. (#21177)
+ Correctly handle unresolved symlinks when they appear in the inputs. (#21181)
+ Add missing close(). (#21183)
+ Add a profile span for building the upload manifest. (#21184)
+ Remove obsolete comments and dividers. (#21185)
+ Implement a new execution log conversion tool. (#21187)
+ Implement a new execution log conversion tool. (#21192)
+ Introduce a MessageInputStream abstraction, mirroring MessageOutputStream. (#21207)
+ Upgrade to use Bazel 7.0.2 (#21208)
+ Do not store the repository name in `RepoSpec` (#21209)
+ Make sure we build as well as test //src/tools/execlog/... on CI. (#21216)
+ Teach ExecLogConverter to read the compact format. (#21223)
+ Switch macOS minimum version flag to gcc compatible version (#21219)
+ Update default visionOS CPU to sim_arm64 (#21240)
+ Avoid exception-based control flow in RemoteActionFileSystem#stat. (#21236)
+ Cherry-pick: linker_param_file only added to command line if it starts with "@" (#21235)
+ Fixes for experimental extend rule and subrule functionality (#21237)
+ Fix NPE in ResourceManager when collecting local resource estimation in the profiler. (#21229)
+ Optimize RemoteActionFileSystem#readdir for the tree artifact input case. (#21251)
+ Document --incompatible_disallow_unsound_directory_outputs. (#21252)
+ Also path map transitive header jar paths with direct classpath optimization (#21227)
+ Error on invalid path characters in `.bazelignore` (#21259)
+ Mark gcc-<version> as `gcc` instead of `compiler` in Unix CC toolchain (#21224)
+ Avoid exception-based control flow in RAFS#getDigest and RAFS#getFastDigest. (#21264)
+ Add `add_exports/add_opens` to bazel java_binary deploy jars (#21270)
+ Manipulate the local filesystem directly in the writeLocalFile test helper. (#21272)
+ Improve the documentation for PathFragment methods dealing with segments. (#21275)
+ Canonicalize the parent path in RemoteActionFileSystem#delete. (#21282)
+ Revert "Also path map transitive header jar paths with direct classpath optimization" (#21281)
+ Make it possible to toggle cache key scrubbing by rule kind (#21276)
+ Fix a hanging issue with skymeld & `--combined_report=lcov`. (#21271)
+ Canonicalize the parent path in RemoteActionFileSystem#renameTo. (#21285)
+ Exclude `//src/test/py/bazel:mod_command_test` from RBE tests due to frequent flaky timeouts.
+ Add `bazel mod tidy` (#21265)
+ Don't use worker threads for repo fetching during Skyframe er… (#21305)
+ Fix flakiness in //src/test/shell/bazel:starlark_repository_test (#21309)
+ Document best practice of avoiding extensions directly specifying repository names (#21300)
+ Allow `@repo_name` labels in override attributes (#21313)
+ Reproducible extension (#21306)
+ Omit unique module versions from canonical repo names (#21316)
+ Add `Label.to_display_form()` (#21312)
+ Clarify the purpose and overall behavior of RemoteActionFileSystem. (#21294)
+ Make SpawnLogConvert an abstract class instead of an interface. (#21325)
+ Add support for additional command profiler event types. (#21327)
+ Remove the fileSize parameter from DigestUtils. (#21328)
+ Optimize RemoteActionFileSystem#resolveSymbolicLinks by caching intermediate results in a trie. (#21333)
+ Mark `use_repo_rule` extension as reproducible (#21335)
+ Make SpawnLogContext interruptible. (#21337)
+ Document --cache_computed_file_digests. (#21326)
+ Generate a lockfile for the distribution archive on the fly (#21338)
+ Introduce --local_resources flag (#21331)
+ Cherry-pick recent changes to fix CI flakiness and breakages (#21349)
+ Clear the file digests cache on clean. (#21346)
+ Parallelize TreeArtifactValue.visitTree across files instead of subdirectories. (#21347)
+ Temporarily hardcode rules_java repository name (#21356)
+ Remove unnecessary test assertions to fix flakiness. (#21354)
+ Make it possible to avoid an extra stat() when obtaining a digest from the cache. (#21353)
+ Collect directory contents in parallel in CompactSpawnLogContext. (#21361)
+ Introduce --default_test_resources flag (#21311)
+ python: rules_python 0.22.0 -> 0.22.1 soas to register Python toolchain by default (#21369)
+ Add vendor mode (#21366)
+ Clarify the behavior of --incompatible_remote_symlinks in the presence of a dangling symlink. (#21363)
+ Handle symlinks in a more consistent manner in UploadManifest. (#21371)
+ Set the executable bit on files in output directories uploaded to a disk or remote cache. (#21376)
+ Call out that TreeArtifactVisitor.visit is called in a nondeterministic order. (#21377)
+ Optimize out a stat call. (#21388)
+ Compute output directories in parallel when building the upload manifest. (#21386)
+ Fix rule definition environment for repo rules (#21397)
+ Share classpath `NestedSet` between full and header compile actions (#21389)
+ Emit labels in display form in Java rules (#21395)
+ Fetch refactor and mod command fix (#21385)
+ Implement `describeKey` for more actions (#21421)
+ Let scrubbed actions fall back to local execution when remote execution is enabled. (#21384)
+ Publish the new execution log format to the build event protocol. (#21417)
+ Ensure that the mtime of an AC entry is smaller, not larger, than the CAS blobs it references. (#21416)
+ Pass the name of the classpath manifest jar to JacocoCoverageRunner (#21413)
+ Traverse symlinks to directories while collecting a TreeArtifactValue. (#21418)
+ Correctly handle file inputs/outputs with directory contents in the execution log. (#21427)
+ Upgrade to async-profiler v3.0. (#21428)
+ Avoid a superfluous stat() in DigestUtil. (#21400)
+ [credentialhelper] Respect `expires` field from helper (#21429)
+ Improve performance of --reuse_sandbox_directories (#21433)
+ [credentialhelper] Update flag doc to point to more convenient usage instructions (#21441)
+ Repo file/dir watching API (#21435)
+ Clarify the meaning of Dirent.Type.UNKNOWN. (#21434)
+ Add a native image of turbine to the prebuilt Java tools (#21426)
+ Update java_tools v13.4 / rules_java 7.4.0 (#21359)
+ Automated rollback of commit b11fa7a. (#21448)
+ Remove the restriction that relative symlinks in a tree artifact may not point outside the tree. (#21449)
+ Revert "Add `Label.to_display_form()`" (#21454)
+ Do not record any repo mapping entries in the RepoMappingRecorder for WORKSPACE repo rules (#21457)
+ Reland "Also path map transitive header jar paths with direct classpath optimization" (#21458)
+ Backport CI test configs (#21456)
+ Use execution info instead of hard-coded mnemonics for Java path mapping (#21461)
+ Always decide whether to scrub an input by its effective path. (#21472)
+ Set RC branch when creating GitHub releases (#21477)
+ Fix vendor existing repo (#21487)
+ [test][windows] Export BAZEL_TEST=1 on windows (#21494)
+ Enable aar_import JNI libs to work with --android_platforms. (#21502)
+ Fix stale trash dir not cleaned up on worker creation (#21510)
+ Fix genrule autostamping in bazel (#21512)
+ Remove --host_jvm_args=-Djava.net.preferIPv6Addresses=true (#21546)
+ Passthrough HTTP headers to remote downloader service (#21503)
+ [credentialhelper] Support paths relative to `%install_base%` (#21532)
+ Update LibrariesToLinkCollector.java for .dll suffix stripping (#21524)
+ Backport changes for updating default lockfile used in integration tests. (#21547)
+ Fix a flaky test by avoiding leaking the eager capability RPC thread. (#21550)
+ Add a profiler span for the findMissingDigests call associated with an upload. (#21552)
+ Move the disk cache reads and writes into a thread pool. (#21551)
+ Lazily open files to be uploaded to an HTTP cache. (#21549)
+ Exclude convenience symlinks after changing the output base (#21505)
+ StarlarkBaseExternalContext.java: propagate error message when deleting temporary directory failed (#21555)
+ Fix `bazel fetch` by replacing query with cquery for underlying implementation (#21567)
+ Fix watching paths in undefined repos in repo rules (#21575)
+ Implicit dependencies should be visible to rule/aspect definitions in `.bzl` files in the same package (#21577)
+ Disable some tests because of JDK21 (#21595)
+ Expose the ApkInfo provider constructor to Starlark. (#21588)
+ Add multiplex sandboxing support to JavaBuilder (#21598)
+ Attempt to fix cancellation crash in repo fetching w/ worker thread (#21599)
+ Move compile StarlarkMethod back to CcModuleAPI (#21605)
+ Expose AndroidIdeInfo in android_common (#21607)
+ Release 7.1.0 (2024-03-11)
+ Update centos7 platform in build_bazel_binaries.yml (#21644)
+ Fix `bazel mod tidy` failure with no changes (#21662)
+ Update .bazelversion to 7.1.0 (#21664)
+ Let native Turbine image find `ct.sym` with non-hermetic `java_runtime` (#21670)
+ Actually use shouldPublish() to determine whether to publish the execution log to the BEP. (#21671)
+ Also inject a failure for createWritableDirectory when testing that ActionOutputDirectoryHelper propagates exceptions. (#21683)
+ Fix race condition and add more logging for null entry error message (#21692)
+ Allow any canonical repo name to be used with `bazel mod show_repo` (#21694)
+ Fix two `bazel mod tidy` crashes (#21700)
+ Cherry-pick Java execution info improvements (#21703)
+ Disable //src/test/shell/bazel:srcs_test on Intel macOS (#21707)
+ Fix sandbox cleanup crashing after server restart (#21733)
+ Revert "Fix `bazel fetch` by replacing query with cquery for … (#21735)
+ Release 7.1.1 (2024-03-21)
+ Implement RemoteActionFileSystem#statIfFound correctly when the path cannot be canonicalized (#21889)
+ Don't upload remote input to remote cache (#21941)
+ Do not watch `.netrc` in `read_netrc` (#22186)
+ Set public visibility for R8 desugar binary (#22176)

Acknowledgements:

This release contains contributions from many people at Google, as well as Alessandro Patti, Artem V. Navrotskiy, bazel.build machine account, Brentley Jones, Cameron Martin, Chi Wawng, Christian Scott, Cristin Donoso, David Ostrovsky, Ed Schouten, Fabian Meumertzheim, Gunnar Wagenknecht, Jordan Mele, Keith Smiley, lberki, Nikhil Kalige, oquenchil, Patrick Balestra, Rahul Butani, Ryan Beasley, Siddhartha Bagaria, Son Luong Ngoc, Sushain Cherivirala, thesayyn, Tianyu Geng, Viktor Kustov, Xdng Yng, Xùdōng Yáng, Yannic, Yannic Bonenberger.
Kila2 pushed a commit to Kila2/bazel that referenced this pull request May 13, 2024
Baseline:  d798ebd

Release Notes:

+ Consider MODULE.bazel for workspace detection in bazel.sh (bazelbuild#20594)
+ Auto-create deploy jars for Bazel `java_test` targets if requested (bazelbuild#20602)
+ `java_binary` wrapper should forward `restricted_to` (bazelbuild#20611)
+ Mount user-specified bind mounts before Bazel's own magic. (bazelbuild#20609)
+ Fix bootstrapped Bazel binary (bazelbuild#20612)
+ Modify the error message that occurs when a requested target does not… (bazelbuild#20636)
+ Cherry-pick all presubmit.yml changes (bazelbuild#20736)
+ Accept labels of aliases in config_setting. (bazelbuild#20649)
+ Improve `use_repo_rule` error when not referencing a `repository_rule` (bazelbuild#20732)
+ Attempt to make main repo mapping inverse more efficient (bazelbuild#20633)
+ Retry binding to ipv6 localhost (bazelbuild#20755)
+ Print interactive sandboxed shell command with `--sandbox_debug` (bazelbuild#20769)
+ Fix two issues with --incompatible_sandbox_hermetic_tmp that manifested themselves when the output base was under /tmp (bazelbuild#20766)
+ Optimize prefetchInputs. (bazelbuild#20719)
+ Fix crash on `bazel mod` error containing `%` (bazelbuild#20651)
+ Cover missing cases during module extension label normalization (bazelbuild#20630)
+ Do not print errors when repository rules are interrupted (bazelbuild#20662)
+ Restart at most once when prepopulating repository rule environment (bazelbuild#20643)
+ Add profiles to the call sites of `updateRunfiles` (bazelbuild#20803)
+ Fixes for Bazel's own integration tests fail locally on Linux (bazelbuild#20822)
+ Remove unnecessary `cc_test` coverage handling (bazelbuild#20641)
+ Fix NPE in BzlmodRepoRuleFunction (bazelbuild#20829)
+ Return labels instead of strings from DescribableExecutionUnit methods. (bazelbuild#20788)
+ Introduce a SpawnLogContext interface. (bazelbuild#20842)
+ Fix `common` `.bazelrc` behavior for flag expansions (bazelbuild#20844)
+ Add a profiler span for fetching repositories. (bazelbuild#20852)
+ Make Bazel's RAM estimate container aware (bazelbuild#20644)
+ Auto-create deploy jars for Bazel `java_test` targets if requested (bazelbuild#20762)
+ Ignore read-only errors when updating the `mtime` of the `install_base` (bazelbuild#20648)
+ Add profiling to `remoteActionBuildingSemaphore.acquire()` (bazelbuild#20645)
+ DigestUtils: avoid throwing on invalid digest function name (bazelbuild#20650)
+ Use a larger buffer size for `java.util.zip.*Stream` classes (bazelbuild#20642)
+ Flip flag `--experimental_use_semaphore_for_jobs`. (bazelbuild#20646)
+ RemoteSpawnRunner: record inbetween phases in timing profile (bazelbuild#20647)
+ Add fastutil 7.2.1 dependency (bazelbuild#20854)
+ Allow repo rules to download multiple things in parallel. (bazelbuild#20856)
+ Add support for tmpfs mounts under `/tmp` with hermetic tmp (bazelbuild#20859)
+ Add new flag `--enable_workspace` that allows us to disable WORKSPACE… (bazelbuild#20855)
+ Add support for bind mounts under `/tmp` with hermetic tmp (bazelbuild#20772)
+ Document `--digest_function` startup flag (bazelbuild#20864)
+ Add flag `experimental_throttle_remote_action_building` (bazelbuild#20858)
+ Remove suffix from fastutil alias. (bazelbuild#20872)
+ Move StableSort into the exec package.
+ Test that missing spawn outputs are logged correctly.
+ Simplify computeDigest.
+ Add SpawnBuilder#with{Inputs,Tools} overloads accepting a NestedSet.
+ Correctly log paths for runfiles and filesets.
+ Report empty files in the spawn log.
+ Propagate the tool bit to logged directory inputs.
+ Move some more common logic into SpawnLogContext.
+ Introduce a new compact execution log format.
+ Apply zstd compression to the compact execution log.
+ Avoid unnecessary overhead when determining whether an action input is a directory.
+ Offer Shell completion for top-level packages without subpackages (bazelbuild#20879)
+ Increase maximal length of profile span for repository function calls (bazelbuild#20907)
+ Still generate a WORKSPACE file in repo rules if --enable_workspace is set (bazelbuild#20914)
+ Read authentication information from .netrc (bazelbuild#20915)
+ deps: rules_python 0.4.0 -> 0.22.0 (bazelbuild#20916)
+ Avoid emitting canonical labels into generated repos (bazelbuild#20917)
+ python: make incompatible_python_disallow_native_rules work for top-level external repo targets (bazelbuild#20923)
+ Remove flag guarding for the AndroidIdeInfo provider (bazelbuild#20932)
+ Point _virtual_includes to stable locations so IDE integrations survive builds (bazelbuild#20946)
+ [rfc] Allow repository rules to lazily declare environment variable deps (bazelbuild#20944)
+ Replaced usage of rev with awk in bash runfiles (bazelbuild#20934)
+ Cherry pick a few changes to address flaky tests (bazelbuild#20956)
+ Cherry-pick the change to reduce repository invalidations to Bazel 7.1 (bazelbuild#20949)
+ The label API shakeup & docs cleanup (bazelbuild#20977)
+ New docs for labels, repos, etc (bazelbuild#20978)
+ Add support for arbitrary headers to rctx.download[_and_extract] (bazelbuild#20979)
+ Show a warning message when the credential helper invocation fails (bazelbuild#20992)
+ Fix singlejar resource mapping for external repositories (bazelbuild#20989)
+ Remove user specific path from the lockfile (Fixes bazelbuild#19621) (bazelbuild#21009)
+ Also report cycles involving WORKSPACE from BzlmodRepoCycleReporter (bazelbuild#21013)
+ Fix -fatal_warnings on macOS (bazelbuild#21018)
+ Cherry-picks for module extension repo mapping usage tracking (bazelbuild#21033)
+ bzlmod: support git repos in source.json (bazelbuild#21036)
+ Add `bazel mod dump_repo_mapping` (bazelbuild#21023)
+ Cherry-picks for elimination of repo rule restarts (bazelbuild#21082)
+ Fix inconsistent dep graph stubs in Bzlmod tests (bazelbuild#21085)
+ Distinguish the disk and remote caches in the action progress status. (bazelbuild#21084)
+ Clarify where to find the definition of the --experimental_remote_scrubbing_config configuration format. (bazelbuild#21089)
+ Disable `--legacy_external_runfiles` in Bazel tests (bazelbuild#21086)
+ Follow directory symlink in RemoteActionFileSystem#getDirectoryEntries(). (bazelbuild#21088)
+ Treat the inability to load the Windows filesystem JNI as an error. (bazelbuild#21090)
+ Fix up permissions error in getInputStream, like we already do for getOutputStream. (bazelbuild#21087)
+ Force output checking for incremental run commands without the bytes. (bazelbuild#20988)
+ Remove visionos_x86_64 CPUs (bazelbuild#21022)
+ Close test.err before deleteing it (bazelbuild#21020)
+ Fix linker feature detection being performed on wrong linker (bazelbuild#20990)
+ Add an option to set a minimum size threshold for zstd blob compression. (bazelbuild#21124)
+ Publish RCs to GitHub (bazelbuild#21127)
+ Avoid using `InputStream.available()` to detect EOF while reading delimited protos. (bazelbuild#21143)
+ Starlark: reuse positional array in native calls where possible (bazelbuild#21144)
+ Harmonize BUILD files. (bazelbuild#21145)
+ Add bash completion for external targets (bazelbuild#21149)
+ Make some minor adjustments to the compact execution log format and document it better. (bazelbuild#21146)
+ Optimize the execution log sorter by using reference equality. (bazelbuild#21147)
+ Update to Turbine 0.4.0 (bazelbuild#21161)
+ Split StableSort into a separate target. (bazelbuild#21152)
+ Document that the compact execution log isn't guaranteed to be serialized in increasing ID order. (bazelbuild#21165)
+ Fix the comment for MessageOutputStream#write(). (bazelbuild#21166)
+ Make repo marker files sensitive to repo mapping changes (bazelbuild#21172)
+ Include the digest hash function in the compact execution log. (bazelbuild#21174)
+ Report unresolved symlinks as such in the execution log. (bazelbuild#21177)
+ Correctly handle unresolved symlinks when they appear in the inputs. (bazelbuild#21181)
+ Add missing close(). (bazelbuild#21183)
+ Add a profile span for building the upload manifest. (bazelbuild#21184)
+ Remove obsolete comments and dividers. (bazelbuild#21185)
+ Implement a new execution log conversion tool. (bazelbuild#21187)
+ Implement a new execution log conversion tool. (bazelbuild#21192)
+ Introduce a MessageInputStream abstraction, mirroring MessageOutputStream. (bazelbuild#21207)
+ Upgrade to use Bazel 7.0.2 (bazelbuild#21208)
+ Do not store the repository name in `RepoSpec` (bazelbuild#21209)
+ Make sure we build as well as test //src/tools/execlog/... on CI. (bazelbuild#21216)
+ Teach ExecLogConverter to read the compact format. (bazelbuild#21223)
+ Switch macOS minimum version flag to gcc compatible version (bazelbuild#21219)
+ Update default visionOS CPU to sim_arm64 (bazelbuild#21240)
+ Avoid exception-based control flow in RemoteActionFileSystem#stat. (bazelbuild#21236)
+ Cherry-pick: linker_param_file only added to command line if it starts with "@" (bazelbuild#21235)
+ Fixes for experimental extend rule and subrule functionality (bazelbuild#21237)
+ Fix NPE in ResourceManager when collecting local resource estimation in the profiler. (bazelbuild#21229)
+ Optimize RemoteActionFileSystem#readdir for the tree artifact input case. (bazelbuild#21251)
+ Document --incompatible_disallow_unsound_directory_outputs. (bazelbuild#21252)
+ Also path map transitive header jar paths with direct classpath optimization (bazelbuild#21227)
+ Error on invalid path characters in `.bazelignore` (bazelbuild#21259)
+ Mark gcc-<version> as `gcc` instead of `compiler` in Unix CC toolchain (bazelbuild#21224)
+ Avoid exception-based control flow in RAFS#getDigest and RAFS#getFastDigest. (bazelbuild#21264)
+ Add `add_exports/add_opens` to bazel java_binary deploy jars (bazelbuild#21270)
+ Manipulate the local filesystem directly in the writeLocalFile test helper. (bazelbuild#21272)
+ Improve the documentation for PathFragment methods dealing with segments. (bazelbuild#21275)
+ Canonicalize the parent path in RemoteActionFileSystem#delete. (bazelbuild#21282)
+ Revert "Also path map transitive header jar paths with direct classpath optimization" (bazelbuild#21281)
+ Make it possible to toggle cache key scrubbing by rule kind (bazelbuild#21276)
+ Fix a hanging issue with skymeld & `--combined_report=lcov`. (bazelbuild#21271)
+ Canonicalize the parent path in RemoteActionFileSystem#renameTo. (bazelbuild#21285)
+ Exclude `//src/test/py/bazel:mod_command_test` from RBE tests due to frequent flaky timeouts.
+ Add `bazel mod tidy` (bazelbuild#21265)
+ Don't use worker threads for repo fetching during Skyframe er… (bazelbuild#21305)
+ Fix flakiness in //src/test/shell/bazel:starlark_repository_test (bazelbuild#21309)
+ Document best practice of avoiding extensions directly specifying repository names (bazelbuild#21300)
+ Allow `@repo_name` labels in override attributes (bazelbuild#21313)
+ Reproducible extension (bazelbuild#21306)
+ Omit unique module versions from canonical repo names (bazelbuild#21316)
+ Add `Label.to_display_form()` (bazelbuild#21312)
+ Clarify the purpose and overall behavior of RemoteActionFileSystem. (bazelbuild#21294)
+ Make SpawnLogConvert an abstract class instead of an interface. (bazelbuild#21325)
+ Add support for additional command profiler event types. (bazelbuild#21327)
+ Remove the fileSize parameter from DigestUtils. (bazelbuild#21328)
+ Optimize RemoteActionFileSystem#resolveSymbolicLinks by caching intermediate results in a trie. (bazelbuild#21333)
+ Mark `use_repo_rule` extension as reproducible (bazelbuild#21335)
+ Make SpawnLogContext interruptible. (bazelbuild#21337)
+ Document --cache_computed_file_digests. (bazelbuild#21326)
+ Generate a lockfile for the distribution archive on the fly (bazelbuild#21338)
+ Introduce --local_resources flag (bazelbuild#21331)
+ Cherry-pick recent changes to fix CI flakiness and breakages (bazelbuild#21349)
+ Clear the file digests cache on clean. (bazelbuild#21346)
+ Parallelize TreeArtifactValue.visitTree across files instead of subdirectories. (bazelbuild#21347)
+ Temporarily hardcode rules_java repository name (bazelbuild#21356)
+ Remove unnecessary test assertions to fix flakiness. (bazelbuild#21354)
+ Make it possible to avoid an extra stat() when obtaining a digest from the cache. (bazelbuild#21353)
+ Collect directory contents in parallel in CompactSpawnLogContext. (bazelbuild#21361)
+ Introduce --default_test_resources flag (bazelbuild#21311)
+ python: rules_python 0.22.0 -> 0.22.1 soas to register Python toolchain by default (bazelbuild#21369)
+ Add vendor mode (bazelbuild#21366)
+ Clarify the behavior of --incompatible_remote_symlinks in the presence of a dangling symlink. (bazelbuild#21363)
+ Handle symlinks in a more consistent manner in UploadManifest. (bazelbuild#21371)
+ Set the executable bit on files in output directories uploaded to a disk or remote cache. (bazelbuild#21376)
+ Call out that TreeArtifactVisitor.visit is called in a nondeterministic order. (bazelbuild#21377)
+ Optimize out a stat call. (bazelbuild#21388)
+ Compute output directories in parallel when building the upload manifest. (bazelbuild#21386)
+ Fix rule definition environment for repo rules (bazelbuild#21397)
+ Share classpath `NestedSet` between full and header compile actions (bazelbuild#21389)
+ Emit labels in display form in Java rules (bazelbuild#21395)
+ Fetch refactor and mod command fix (bazelbuild#21385)
+ Implement `describeKey` for more actions (bazelbuild#21421)
+ Let scrubbed actions fall back to local execution when remote execution is enabled. (bazelbuild#21384)
+ Publish the new execution log format to the build event protocol. (bazelbuild#21417)
+ Ensure that the mtime of an AC entry is smaller, not larger, than the CAS blobs it references. (bazelbuild#21416)
+ Pass the name of the classpath manifest jar to JacocoCoverageRunner (bazelbuild#21413)
+ Traverse symlinks to directories while collecting a TreeArtifactValue. (bazelbuild#21418)
+ Correctly handle file inputs/outputs with directory contents in the execution log. (bazelbuild#21427)
+ Upgrade to async-profiler v3.0. (bazelbuild#21428)
+ Avoid a superfluous stat() in DigestUtil. (bazelbuild#21400)
+ [credentialhelper] Respect `expires` field from helper (bazelbuild#21429)
+ Improve performance of --reuse_sandbox_directories (bazelbuild#21433)
+ [credentialhelper] Update flag doc to point to more convenient usage instructions (bazelbuild#21441)
+ Repo file/dir watching API (bazelbuild#21435)
+ Clarify the meaning of Dirent.Type.UNKNOWN. (bazelbuild#21434)
+ Add a native image of turbine to the prebuilt Java tools (bazelbuild#21426)
+ Update java_tools v13.4 / rules_java 7.4.0 (bazelbuild#21359)
+ Automated rollback of commit b11fa7a. (bazelbuild#21448)
+ Remove the restriction that relative symlinks in a tree artifact may not point outside the tree. (bazelbuild#21449)
+ Revert "Add `Label.to_display_form()`" (bazelbuild#21454)
+ Do not record any repo mapping entries in the RepoMappingRecorder for WORKSPACE repo rules (bazelbuild#21457)
+ Reland "Also path map transitive header jar paths with direct classpath optimization" (bazelbuild#21458)
+ Backport CI test configs (bazelbuild#21456)
+ Use execution info instead of hard-coded mnemonics for Java path mapping (bazelbuild#21461)
+ Always decide whether to scrub an input by its effective path. (bazelbuild#21472)
+ Set RC branch when creating GitHub releases (bazelbuild#21477)
+ Fix vendor existing repo (bazelbuild#21487)
+ [test][windows] Export BAZEL_TEST=1 on windows (bazelbuild#21494)
+ Enable aar_import JNI libs to work with --android_platforms. (bazelbuild#21502)
+ Fix stale trash dir not cleaned up on worker creation (bazelbuild#21510)
+ Fix genrule autostamping in bazel (bazelbuild#21512)
+ Remove --host_jvm_args=-Djava.net.preferIPv6Addresses=true (bazelbuild#21546)
+ Passthrough HTTP headers to remote downloader service (bazelbuild#21503)
+ [credentialhelper] Support paths relative to `%install_base%` (bazelbuild#21532)
+ Update LibrariesToLinkCollector.java for .dll suffix stripping (bazelbuild#21524)
+ Backport changes for updating default lockfile used in integration tests. (bazelbuild#21547)
+ Fix a flaky test by avoiding leaking the eager capability RPC thread. (bazelbuild#21550)
+ Add a profiler span for the findMissingDigests call associated with an upload. (bazelbuild#21552)
+ Move the disk cache reads and writes into a thread pool. (bazelbuild#21551)
+ Lazily open files to be uploaded to an HTTP cache. (bazelbuild#21549)
+ Exclude convenience symlinks after changing the output base (bazelbuild#21505)
+ StarlarkBaseExternalContext.java: propagate error message when deleting temporary directory failed (bazelbuild#21555)
+ Fix `bazel fetch` by replacing query with cquery for underlying implementation (bazelbuild#21567)
+ Fix watching paths in undefined repos in repo rules (bazelbuild#21575)
+ Implicit dependencies should be visible to rule/aspect definitions in `.bzl` files in the same package (bazelbuild#21577)
+ Disable some tests because of JDK21 (bazelbuild#21595)
+ Expose the ApkInfo provider constructor to Starlark. (bazelbuild#21588)
+ Add multiplex sandboxing support to JavaBuilder (bazelbuild#21598)
+ Attempt to fix cancellation crash in repo fetching w/ worker thread (bazelbuild#21599)
+ Move compile StarlarkMethod back to CcModuleAPI (bazelbuild#21605)
+ Expose AndroidIdeInfo in android_common (bazelbuild#21607)
+ Release 7.1.0 (2024-03-11)
+ Update centos7 platform in build_bazel_binaries.yml (bazelbuild#21644)
+ Fix `bazel mod tidy` failure with no changes (bazelbuild#21662)
+ Update .bazelversion to 7.1.0 (bazelbuild#21664)
+ Let native Turbine image find `ct.sym` with non-hermetic `java_runtime` (bazelbuild#21670)
+ Actually use shouldPublish() to determine whether to publish the execution log to the BEP. (bazelbuild#21671)
+ Also inject a failure for createWritableDirectory when testing that ActionOutputDirectoryHelper propagates exceptions. (bazelbuild#21683)
+ Fix race condition and add more logging for null entry error message (bazelbuild#21692)
+ Allow any canonical repo name to be used with `bazel mod show_repo` (bazelbuild#21694)
+ Fix two `bazel mod tidy` crashes (bazelbuild#21700)
+ Cherry-pick Java execution info improvements (bazelbuild#21703)
+ Disable //src/test/shell/bazel:srcs_test on Intel macOS (bazelbuild#21707)
+ Fix sandbox cleanup crashing after server restart (bazelbuild#21733)
+ Revert "Fix `bazel fetch` by replacing query with cquery for … (bazelbuild#21735)
+ Release 7.1.1 (2024-03-21)
+ Implement RemoteActionFileSystem#statIfFound correctly when the path cannot be canonicalized (bazelbuild#21889)
+ Don't upload remote input to remote cache (bazelbuild#21941)
+ Do not watch `.netrc` in `read_netrc` (bazelbuild#22186)
+ Set public visibility for R8 desugar binary (bazelbuild#22176)

Acknowledgements:

This release contains contributions from many people at Google, as well as Alessandro Patti, Artem V. Navrotskiy, bazel.build machine account, Brentley Jones, Cameron Martin, Chi Wawng, Christian Scott, Cristin Donoso, David Ostrovsky, Ed Schouten, Fabian Meumertzheim, Gunnar Wagenknecht, Jordan Mele, Keith Smiley, lberki, Nikhil Kalige, oquenchil, Patrick Balestra, Rahul Butani, Ryan Beasley, Siddhartha Bagaria, Son Luong Ngoc, Sushain Cherivirala, thesayyn, Tianyu Geng, Viktor Kustov, Xdng Yng, Xùdōng Yáng, Yannic, Yannic Bonenberger.
rdeushane pushed a commit to SibrosTech/bazel that referenced this pull request Jun 20, 2024
Baseline:  d798ebd

Release Notes:

+ Consider MODULE.bazel for workspace detection in bazel.sh (bazelbuild#20594)
+ Auto-create deploy jars for Bazel `java_test` targets if requested (bazelbuild#20602)
+ `java_binary` wrapper should forward `restricted_to` (bazelbuild#20611)
+ Mount user-specified bind mounts before Bazel's own magic. (bazelbuild#20609)
+ Fix bootstrapped Bazel binary (bazelbuild#20612)
+ Modify the error message that occurs when a requested target does not… (bazelbuild#20636)
+ Cherry-pick all presubmit.yml changes (bazelbuild#20736)
+ Accept labels of aliases in config_setting. (bazelbuild#20649)
+ Improve `use_repo_rule` error when not referencing a `repository_rule` (bazelbuild#20732)
+ Attempt to make main repo mapping inverse more efficient (bazelbuild#20633)
+ Retry binding to ipv6 localhost (bazelbuild#20755)
+ Print interactive sandboxed shell command with `--sandbox_debug` (bazelbuild#20769)
+ Fix two issues with --incompatible_sandbox_hermetic_tmp that manifested themselves when the output base was under /tmp (bazelbuild#20766)
+ Optimize prefetchInputs. (bazelbuild#20719)
+ Fix crash on `bazel mod` error containing `%` (bazelbuild#20651)
+ Cover missing cases during module extension label normalization (bazelbuild#20630)
+ Do not print errors when repository rules are interrupted (bazelbuild#20662)
+ Restart at most once when prepopulating repository rule environment (bazelbuild#20643)
+ Add profiles to the call sites of `updateRunfiles` (bazelbuild#20803)
+ Fixes for Bazel's own integration tests fail locally on Linux (bazelbuild#20822)
+ Remove unnecessary `cc_test` coverage handling (bazelbuild#20641)
+ Fix NPE in BzlmodRepoRuleFunction (bazelbuild#20829)
+ Return labels instead of strings from DescribableExecutionUnit methods. (bazelbuild#20788)
+ Introduce a SpawnLogContext interface. (bazelbuild#20842)
+ Fix `common` `.bazelrc` behavior for flag expansions (bazelbuild#20844)
+ Add a profiler span for fetching repositories. (bazelbuild#20852)
+ Make Bazel's RAM estimate container aware (bazelbuild#20644)
+ Auto-create deploy jars for Bazel `java_test` targets if requested (bazelbuild#20762)
+ Ignore read-only errors when updating the `mtime` of the `install_base` (bazelbuild#20648)
+ Add profiling to `remoteActionBuildingSemaphore.acquire()` (bazelbuild#20645)
+ DigestUtils: avoid throwing on invalid digest function name (bazelbuild#20650)
+ Use a larger buffer size for `java.util.zip.*Stream` classes (bazelbuild#20642)
+ Flip flag `--experimental_use_semaphore_for_jobs`. (bazelbuild#20646)
+ RemoteSpawnRunner: record inbetween phases in timing profile (bazelbuild#20647)
+ Add fastutil 7.2.1 dependency (bazelbuild#20854)
+ Allow repo rules to download multiple things in parallel. (bazelbuild#20856)
+ Add support for tmpfs mounts under `/tmp` with hermetic tmp (bazelbuild#20859)
+ Add new flag `--enable_workspace` that allows us to disable WORKSPACE… (bazelbuild#20855)
+ Add support for bind mounts under `/tmp` with hermetic tmp (bazelbuild#20772)
+ Document `--digest_function` startup flag (bazelbuild#20864)
+ Add flag `experimental_throttle_remote_action_building` (bazelbuild#20858)
+ Remove suffix from fastutil alias. (bazelbuild#20872)
+ Move StableSort into the exec package.
+ Test that missing spawn outputs are logged correctly.
+ Simplify computeDigest.
+ Add SpawnBuilder#with{Inputs,Tools} overloads accepting a NestedSet.
+ Correctly log paths for runfiles and filesets.
+ Report empty files in the spawn log.
+ Propagate the tool bit to logged directory inputs.
+ Move some more common logic into SpawnLogContext.
+ Introduce a new compact execution log format.
+ Apply zstd compression to the compact execution log.
+ Avoid unnecessary overhead when determining whether an action input is a directory.
+ Offer Shell completion for top-level packages without subpackages (bazelbuild#20879)
+ Increase maximal length of profile span for repository function calls (bazelbuild#20907)
+ Still generate a WORKSPACE file in repo rules if --enable_workspace is set (bazelbuild#20914)
+ Read authentication information from .netrc (bazelbuild#20915)
+ deps: rules_python 0.4.0 -> 0.22.0 (bazelbuild#20916)
+ Avoid emitting canonical labels into generated repos (bazelbuild#20917)
+ python: make incompatible_python_disallow_native_rules work for top-level external repo targets (bazelbuild#20923)
+ Remove flag guarding for the AndroidIdeInfo provider (bazelbuild#20932)
+ Point _virtual_includes to stable locations so IDE integrations survive builds (bazelbuild#20946)
+ [rfc] Allow repository rules to lazily declare environment variable deps (bazelbuild#20944)
+ Replaced usage of rev with awk in bash runfiles (bazelbuild#20934)
+ Cherry pick a few changes to address flaky tests (bazelbuild#20956)
+ Cherry-pick the change to reduce repository invalidations to Bazel 7.1 (bazelbuild#20949)
+ The label API shakeup & docs cleanup (bazelbuild#20977)
+ New docs for labels, repos, etc (bazelbuild#20978)
+ Add support for arbitrary headers to rctx.download[_and_extract] (bazelbuild#20979)
+ Show a warning message when the credential helper invocation fails (bazelbuild#20992)
+ Fix singlejar resource mapping for external repositories (bazelbuild#20989)
+ Remove user specific path from the lockfile (Fixes bazelbuild#19621) (bazelbuild#21009)
+ Also report cycles involving WORKSPACE from BzlmodRepoCycleReporter (bazelbuild#21013)
+ Fix -fatal_warnings on macOS (bazelbuild#21018)
+ Cherry-picks for module extension repo mapping usage tracking (bazelbuild#21033)
+ bzlmod: support git repos in source.json (bazelbuild#21036)
+ Add `bazel mod dump_repo_mapping` (bazelbuild#21023)
+ Cherry-picks for elimination of repo rule restarts (bazelbuild#21082)
+ Fix inconsistent dep graph stubs in Bzlmod tests (bazelbuild#21085)
+ Distinguish the disk and remote caches in the action progress status. (bazelbuild#21084)
+ Clarify where to find the definition of the --experimental_remote_scrubbing_config configuration format. (bazelbuild#21089)
+ Disable `--legacy_external_runfiles` in Bazel tests (bazelbuild#21086)
+ Follow directory symlink in RemoteActionFileSystem#getDirectoryEntries(). (bazelbuild#21088)
+ Treat the inability to load the Windows filesystem JNI as an error. (bazelbuild#21090)
+ Fix up permissions error in getInputStream, like we already do for getOutputStream. (bazelbuild#21087)
+ Force output checking for incremental run commands without the bytes. (bazelbuild#20988)
+ Remove visionos_x86_64 CPUs (bazelbuild#21022)
+ Close test.err before deleteing it (bazelbuild#21020)
+ Fix linker feature detection being performed on wrong linker (bazelbuild#20990)
+ Add an option to set a minimum size threshold for zstd blob compression. (bazelbuild#21124)
+ Publish RCs to GitHub (bazelbuild#21127)
+ Avoid using `InputStream.available()` to detect EOF while reading delimited protos. (bazelbuild#21143)
+ Starlark: reuse positional array in native calls where possible (bazelbuild#21144)
+ Harmonize BUILD files. (bazelbuild#21145)
+ Add bash completion for external targets (bazelbuild#21149)
+ Make some minor adjustments to the compact execution log format and document it better. (bazelbuild#21146)
+ Optimize the execution log sorter by using reference equality. (bazelbuild#21147)
+ Update to Turbine 0.4.0 (bazelbuild#21161)
+ Split StableSort into a separate target. (bazelbuild#21152)
+ Document that the compact execution log isn't guaranteed to be serialized in increasing ID order. (bazelbuild#21165)
+ Fix the comment for MessageOutputStream#write(). (bazelbuild#21166)
+ Make repo marker files sensitive to repo mapping changes (bazelbuild#21172)
+ Include the digest hash function in the compact execution log. (bazelbuild#21174)
+ Report unresolved symlinks as such in the execution log. (bazelbuild#21177)
+ Correctly handle unresolved symlinks when they appear in the inputs. (bazelbuild#21181)
+ Add missing close(). (bazelbuild#21183)
+ Add a profile span for building the upload manifest. (bazelbuild#21184)
+ Remove obsolete comments and dividers. (bazelbuild#21185)
+ Implement a new execution log conversion tool. (bazelbuild#21187)
+ Implement a new execution log conversion tool. (bazelbuild#21192)
+ Introduce a MessageInputStream abstraction, mirroring MessageOutputStream. (bazelbuild#21207)
+ Upgrade to use Bazel 7.0.2 (bazelbuild#21208)
+ Do not store the repository name in `RepoSpec` (bazelbuild#21209)
+ Make sure we build as well as test //src/tools/execlog/... on CI. (bazelbuild#21216)
+ Teach ExecLogConverter to read the compact format. (bazelbuild#21223)
+ Switch macOS minimum version flag to gcc compatible version (bazelbuild#21219)
+ Update default visionOS CPU to sim_arm64 (bazelbuild#21240)
+ Avoid exception-based control flow in RemoteActionFileSystem#stat. (bazelbuild#21236)
+ Cherry-pick: linker_param_file only added to command line if it starts with "@" (bazelbuild#21235)
+ Fixes for experimental extend rule and subrule functionality (bazelbuild#21237)
+ Fix NPE in ResourceManager when collecting local resource estimation in the profiler. (bazelbuild#21229)
+ Optimize RemoteActionFileSystem#readdir for the tree artifact input case. (bazelbuild#21251)
+ Document --incompatible_disallow_unsound_directory_outputs. (bazelbuild#21252)
+ Also path map transitive header jar paths with direct classpath optimization (bazelbuild#21227)
+ Error on invalid path characters in `.bazelignore` (bazelbuild#21259)
+ Mark gcc-<version> as `gcc` instead of `compiler` in Unix CC toolchain (bazelbuild#21224)
+ Avoid exception-based control flow in RAFS#getDigest and RAFS#getFastDigest. (bazelbuild#21264)
+ Add `add_exports/add_opens` to bazel java_binary deploy jars (bazelbuild#21270)
+ Manipulate the local filesystem directly in the writeLocalFile test helper. (bazelbuild#21272)
+ Improve the documentation for PathFragment methods dealing with segments. (bazelbuild#21275)
+ Canonicalize the parent path in RemoteActionFileSystem#delete. (bazelbuild#21282)
+ Revert "Also path map transitive header jar paths with direct classpath optimization" (bazelbuild#21281)
+ Make it possible to toggle cache key scrubbing by rule kind (bazelbuild#21276)
+ Fix a hanging issue with skymeld & `--combined_report=lcov`. (bazelbuild#21271)
+ Canonicalize the parent path in RemoteActionFileSystem#renameTo. (bazelbuild#21285)
+ Exclude `//src/test/py/bazel:mod_command_test` from RBE tests due to frequent flaky timeouts.
+ Add `bazel mod tidy` (bazelbuild#21265)
+ Don't use worker threads for repo fetching during Skyframe er… (bazelbuild#21305)
+ Fix flakiness in //src/test/shell/bazel:starlark_repository_test (bazelbuild#21309)
+ Document best practice of avoiding extensions directly specifying repository names (bazelbuild#21300)
+ Allow `@repo_name` labels in override attributes (bazelbuild#21313)
+ Reproducible extension (bazelbuild#21306)
+ Omit unique module versions from canonical repo names (bazelbuild#21316)
+ Add `Label.to_display_form()` (bazelbuild#21312)
+ Clarify the purpose and overall behavior of RemoteActionFileSystem. (bazelbuild#21294)
+ Make SpawnLogConvert an abstract class instead of an interface. (bazelbuild#21325)
+ Add support for additional command profiler event types. (bazelbuild#21327)
+ Remove the fileSize parameter from DigestUtils. (bazelbuild#21328)
+ Optimize RemoteActionFileSystem#resolveSymbolicLinks by caching intermediate results in a trie. (bazelbuild#21333)
+ Mark `use_repo_rule` extension as reproducible (bazelbuild#21335)
+ Make SpawnLogContext interruptible. (bazelbuild#21337)
+ Document --cache_computed_file_digests. (bazelbuild#21326)
+ Generate a lockfile for the distribution archive on the fly (bazelbuild#21338)
+ Introduce --local_resources flag (bazelbuild#21331)
+ Cherry-pick recent changes to fix CI flakiness and breakages (bazelbuild#21349)
+ Clear the file digests cache on clean. (bazelbuild#21346)
+ Parallelize TreeArtifactValue.visitTree across files instead of subdirectories. (bazelbuild#21347)
+ Temporarily hardcode rules_java repository name (bazelbuild#21356)
+ Remove unnecessary test assertions to fix flakiness. (bazelbuild#21354)
+ Make it possible to avoid an extra stat() when obtaining a digest from the cache. (bazelbuild#21353)
+ Collect directory contents in parallel in CompactSpawnLogContext. (bazelbuild#21361)
+ Introduce --default_test_resources flag (bazelbuild#21311)
+ python: rules_python 0.22.0 -> 0.22.1 soas to register Python toolchain by default (bazelbuild#21369)
+ Add vendor mode (bazelbuild#21366)
+ Clarify the behavior of --incompatible_remote_symlinks in the presence of a dangling symlink. (bazelbuild#21363)
+ Handle symlinks in a more consistent manner in UploadManifest. (bazelbuild#21371)
+ Set the executable bit on files in output directories uploaded to a disk or remote cache. (bazelbuild#21376)
+ Call out that TreeArtifactVisitor.visit is called in a nondeterministic order. (bazelbuild#21377)
+ Optimize out a stat call. (bazelbuild#21388)
+ Compute output directories in parallel when building the upload manifest. (bazelbuild#21386)
+ Fix rule definition environment for repo rules (bazelbuild#21397)
+ Share classpath `NestedSet` between full and header compile actions (bazelbuild#21389)
+ Emit labels in display form in Java rules (bazelbuild#21395)
+ Fetch refactor and mod command fix (bazelbuild#21385)
+ Implement `describeKey` for more actions (bazelbuild#21421)
+ Let scrubbed actions fall back to local execution when remote execution is enabled. (bazelbuild#21384)
+ Publish the new execution log format to the build event protocol. (bazelbuild#21417)
+ Ensure that the mtime of an AC entry is smaller, not larger, than the CAS blobs it references. (bazelbuild#21416)
+ Pass the name of the classpath manifest jar to JacocoCoverageRunner (bazelbuild#21413)
+ Traverse symlinks to directories while collecting a TreeArtifactValue. (bazelbuild#21418)
+ Correctly handle file inputs/outputs with directory contents in the execution log. (bazelbuild#21427)
+ Upgrade to async-profiler v3.0. (bazelbuild#21428)
+ Avoid a superfluous stat() in DigestUtil. (bazelbuild#21400)
+ [credentialhelper] Respect `expires` field from helper (bazelbuild#21429)
+ Improve performance of --reuse_sandbox_directories (bazelbuild#21433)
+ [credentialhelper] Update flag doc to point to more convenient usage instructions (bazelbuild#21441)
+ Repo file/dir watching API (bazelbuild#21435)
+ Clarify the meaning of Dirent.Type.UNKNOWN. (bazelbuild#21434)
+ Add a native image of turbine to the prebuilt Java tools (bazelbuild#21426)
+ Update java_tools v13.4 / rules_java 7.4.0 (bazelbuild#21359)
+ Automated rollback of commit b11fa7a. (bazelbuild#21448)
+ Remove the restriction that relative symlinks in a tree artifact may not point outside the tree. (bazelbuild#21449)
+ Revert "Add `Label.to_display_form()`" (bazelbuild#21454)
+ Do not record any repo mapping entries in the RepoMappingRecorder for WORKSPACE repo rules (bazelbuild#21457)
+ Reland "Also path map transitive header jar paths with direct classpath optimization" (bazelbuild#21458)
+ Backport CI test configs (bazelbuild#21456)
+ Use execution info instead of hard-coded mnemonics for Java path mapping (bazelbuild#21461)
+ Always decide whether to scrub an input by its effective path. (bazelbuild#21472)
+ Set RC branch when creating GitHub releases (bazelbuild#21477)
+ Fix vendor existing repo (bazelbuild#21487)
+ [test][windows] Export BAZEL_TEST=1 on windows (bazelbuild#21494)
+ Enable aar_import JNI libs to work with --android_platforms. (bazelbuild#21502)
+ Fix stale trash dir not cleaned up on worker creation (bazelbuild#21510)
+ Fix genrule autostamping in bazel (bazelbuild#21512)
+ Remove --host_jvm_args=-Djava.net.preferIPv6Addresses=true (bazelbuild#21546)
+ Passthrough HTTP headers to remote downloader service (bazelbuild#21503)
+ [credentialhelper] Support paths relative to `%install_base%` (bazelbuild#21532)
+ Update LibrariesToLinkCollector.java for .dll suffix stripping (bazelbuild#21524)
+ Backport changes for updating default lockfile used in integration tests. (bazelbuild#21547)
+ Fix a flaky test by avoiding leaking the eager capability RPC thread. (bazelbuild#21550)
+ Add a profiler span for the findMissingDigests call associated with an upload. (bazelbuild#21552)
+ Move the disk cache reads and writes into a thread pool. (bazelbuild#21551)
+ Lazily open files to be uploaded to an HTTP cache. (bazelbuild#21549)
+ Exclude convenience symlinks after changing the output base (bazelbuild#21505)
+ StarlarkBaseExternalContext.java: propagate error message when deleting temporary directory failed (bazelbuild#21555)
+ Fix `bazel fetch` by replacing query with cquery for underlying implementation (bazelbuild#21567)
+ Fix watching paths in undefined repos in repo rules (bazelbuild#21575)
+ Implicit dependencies should be visible to rule/aspect definitions in `.bzl` files in the same package (bazelbuild#21577)
+ Disable some tests because of JDK21 (bazelbuild#21595)
+ Expose the ApkInfo provider constructor to Starlark. (bazelbuild#21588)
+ Add multiplex sandboxing support to JavaBuilder (bazelbuild#21598)
+ Attempt to fix cancellation crash in repo fetching w/ worker thread (bazelbuild#21599)
+ Move compile StarlarkMethod back to CcModuleAPI (bazelbuild#21605)
+ Expose AndroidIdeInfo in android_common (bazelbuild#21607)
+ Release 7.1.0 (2024-03-11)
+ Update centos7 platform in build_bazel_binaries.yml (bazelbuild#21644)
+ Fix `bazel mod tidy` failure with no changes (bazelbuild#21662)
+ Update .bazelversion to 7.1.0 (bazelbuild#21664)
+ Let native Turbine image find `ct.sym` with non-hermetic `java_runtime` (bazelbuild#21670)
+ Actually use shouldPublish() to determine whether to publish the execution log to the BEP. (bazelbuild#21671)
+ Also inject a failure for createWritableDirectory when testing that ActionOutputDirectoryHelper propagates exceptions. (bazelbuild#21683)
+ Fix race condition and add more logging for null entry error message (bazelbuild#21692)
+ Allow any canonical repo name to be used with `bazel mod show_repo` (bazelbuild#21694)
+ Fix two `bazel mod tidy` crashes (bazelbuild#21700)
+ Cherry-pick Java execution info improvements (bazelbuild#21703)
+ Disable //src/test/shell/bazel:srcs_test on Intel macOS (bazelbuild#21707)
+ Fix sandbox cleanup crashing after server restart (bazelbuild#21733)
+ Revert "Fix `bazel fetch` by replacing query with cquery for … (bazelbuild#21735)
+ Release 7.1.1 (2024-03-21)
+ Implement RemoteActionFileSystem#statIfFound correctly when the path cannot be canonicalized (bazelbuild#21889)
+ Don't upload remote input to remote cache (bazelbuild#21941)
+ Do not watch `.netrc` in `read_netrc` (bazelbuild#22186)
+ Set public visibility for R8 desugar binary (bazelbuild#22176)

Acknowledgements:

This release contains contributions from many people at Google, as well as Alessandro Patti, Artem V. Navrotskiy, bazel.build machine account, Brentley Jones, Cameron Martin, Chi Wawng, Christian Scott, Cristin Donoso, David Ostrovsky, Ed Schouten, Fabian Meumertzheim, Gunnar Wagenknecht, Jordan Mele, Keith Smiley, lberki, Nikhil Kalige, oquenchil, Patrick Balestra, Rahul Butani, Ryan Beasley, Siddhartha Bagaria, Son Luong Ngoc, Sushain Cherivirala, thesayyn, Tianyu Geng, Viktor Kustov, Xdng Yng, Xùdōng Yáng, Yannic, Yannic Bonenberger.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants