Undefined @@rules_java_builtin repository with --noenable_workspace option #22754

lbcjbb · 2024-06-14T17:47:52Z

Description of the bug:

It is sometimes necessary to have a WORSPACE file inside a test data directory. This is particularly the case for gazelle tests using the rule gazelle_generation_test.
But when we use a pure bzlmod Bazel project, with --enable_bzlmod and --noenable_workspace options, and without a file WORKSPACE.bzlmod, Bazel fails to visit this tests data directory containing a WORKSPACE file:

ERROR: Failed to load package, for testdata, skipping: error loading package '': Unable to find package for @@rules_java_builtin//toolchains:local_java_repository.bzl: The repository '@@rules_java_builtin' could not be resolved: Repository '@@rules_java_builtin' is not defined.

No problem with --enable_workspace option.
No problem with an empty WORKSPACE.bzlmod
No problem with an empty MODULE.bazel file inside the test data directory (gazelle_generation_test should support this solution).

Which category does this issue belong to?

External Dependency

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

.
├── .bazelrc
├── .bazelversion
├── BUILD.bazel
├── MODULE.bazel
└── testdata
    └── WORKSPACE

.bazelrc

common --enable_bzlmod
common --noenable_workspace
common --lockfile_mode=off

.bazelversion

7.2.0

BUILD.bazel

testfiles = glob(["testdata/**"])
print(testfiles)

MODULE.bazel

module(name = "foo")

$ bazel build :all
WARNING: Target pattern parsing failed.
ERROR: Skipping ':all': while parsing ':all': error loading package '': Unable to find package for @@rules_java_builtin//toolchains:local_java_repository.bzl: The repository '@@rules_java_builtin' could not be resolved: Repository '@@rules_java_builtin' is not defined.
ERROR: while parsing ':all': error loading package '': Unable to find package for @@rules_java_builtin//toolchains:local_java_repository.bzl: The repository '@@rules_java_builtin' could not be resolved: Repository '@@rules_java_builtin' is not defined.
INFO: Elapsed time: 0.023s
INFO: 0 processes.
ERROR: Build did NOT complete successfully

Which operating system are you running Bazel on?

Linux / amd64

What is the output of `bazel info release`?

release 7.2.0

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

No response

What's the output of `git remote get-url origin; git rev-parse HEAD` ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

The text was updated successfully, but these errors were encountered:

Wyverald · 2024-06-17T19:00:56Z

Thanks for the report! We missed a case of disabling WORKSPACE logic in LocalRepositoryLookupFunction. I'll send a change.

…le_workspace` This still leaves the question of "what do we do instead?". See issues #22208 and #21515. Fixes #22754.

…le_workspace` This still leaves the question of "what do we do instead?". See issues bazelbuild#22208 and bazelbuild#21515. Fixes bazelbuild#22754. Closes bazelbuild#22774. PiperOrigin-RevId: 645148811 Change-Id: Ib9d07d2ecbc3a79e3341de6739de1c3349124d6b

…--noenable_workspace` (#22837) This still leaves the question of "what do we do instead?". See issues #22208 and #21515. Fixes #22754. Closes #22774. PiperOrigin-RevId: 645148811 Change-Id: Ib9d07d2ecbc3a79e3341de6739de1c3349124d6b Commit 1246ff4 Co-authored-by: Xdng Yng <wyverald@gmail.com>

iancha1992 · 2024-06-21T21:13:43Z

A fix for this issue has been included in Bazel 7.2.1 RC2. Please test out the release candidate and report any issues as soon as possible.
If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=7.2.1rc2. Thanks!

…le_workspace` This still leaves the question of "what do we do instead?". See issues bazelbuild#22208 and bazelbuild#21515. Fixes bazelbuild#22754. Closes bazelbuild#22774. PiperOrigin-RevId: 645148811 Change-Id: Ib9d07d2ecbc3a79e3341de6739de1c3349124d6b

lbcjbb · 2024-06-26T16:08:40Z

Tested with the new 7.2.1 release, and everything works as expected.
Thank you @iancha1992 @Wyverald

…le_workspace` This still leaves the question of "what do we do instead?". See issues bazelbuild#22208 and bazelbuild#21515. Fixes bazelbuild#22754. Closes bazelbuild#22774. PiperOrigin-RevId: 645148811 Change-Id: Ib9d07d2ecbc3a79e3341de6739de1c3349124d6b

…--noenable_workspace` (#22911) This still leaves the question of "what do we do instead?". See issues #22208 and #21515. Fixes #22754. Closes #22774. PiperOrigin-RevId: 645148811 Change-Id: Ib9d07d2ecbc3a79e3341de6739de1c3349124d6b Commit 1246ff4 Co-authored-by: Xdng Yng <wyverald@gmail.com>

…--noenable_workspace` (bazelbuild#22837) This still leaves the question of "what do we do instead?". See issues bazelbuild#22208 and bazelbuild#21515. Fixes bazelbuild#22754. Closes bazelbuild#22774. PiperOrigin-RevId: 645148811 Change-Id: Ib9d07d2ecbc3a79e3341de6739de1c3349124d6b Commit bazelbuild@1246ff4 Co-authored-by: Xdng Yng <wyverald@gmail.com>

This begins the Bzlmod compatibility migration by updating Bazel to version 7.3.2 and adding initial `MODULE.bazel` and `WORKSPACE.bzlmod` files. Part of: bazelbuild#1482 Though Bzlmod remains disabled, updating to Bazel 7.3.2 requred updating or adding the following packages to maintain `WORKSPACE` compatibility. In `rules_scala_setup()`: - bazel_skylib: 1.4.1 => 1.7.1 - rules_cc: 0.0.6 => 0.0.10 - rules_java: 5.4.1 => 7.9.0 - rules_proto: 5.3.0-21.7 => 6.0.2 Dev dependencies in `WORKSPACE`: - com_google_protobuf: 28.2 - rules_pkg: 1.0.1 - rules_jvm_external: 6.4 - com_google_absl: abseil-cpp-20240722.0 - zlib: 1.3.1 Of all of the new, explicit dev dependencies, only `com_google_protobuf` will be necessary to include in `MODULE.bazel`. The Bzlmod mechanism will discover these other transitive dev dependencies automatically. Also removed the `rules_java_extra` repo from `WORKSPACE`, which appeared unused. --- Though the current `rules_java` version is 7.12.1, and largely works with this repo, it requires a few temporary workarounds. Rather than commit the workarounds, upgrading only to 7.9.0 now seems less crufty. What follows is a very detailed explanation of what happens with 7.12.1 with Bazel 7.3.2, just to have it on the record. --- The workaround is to change a few toolchain and macro file targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. This isn't a terribly bad or invasive workaround, but `@bazel_tools//tools/jdk:` is clearly the canonical path. Best to keep it that way, lest we build up technical debt. Without the workaround, these targets would fail: - //test/src/main/resources/java_sources:CompiledWithJava11 - //test/src/main/resources/java_sources:CompiledWithJava8 - //test/toolchains:java21_toolchain - //test:JunitRuntimePlatform - //test:JunitRuntimePlatform_test_runner - //test:scala_binary_jdk_11 with this error: ```txt ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. ``` This appears to be a consequence of both upgrading the Bazel version from 6.3.0 to 7.3.2 and updating `rules_java` to 7.12.1. The `rules_java_builtin` repo is part of the `WORKSPACE` prefix that adds implicit dependencies: - https://bazel.build/external/migration#builtin-default-deps This repo was added to 7.0.0-pre.20231011.2 in the following change, mapped to `@rules_java` within the scope of the `@bazel_tools` repo: - bazelbuild/bazel: Add rules_java_builtin to the users of Java modules bazelbuild/bazel@ff1abb2 This change tried to ensure `rules_java` remained compatible with earlier Bazel versions. However, it changed all instances of `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` to `//toolchains:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java: Make rules_java backwards compatible with Bazel 6.3.0 bazelbuild/rules_java@30ecf3f Bazel has bumped `rules_java` in its `workspace_deps.bzl` from 7.9.0 to 7.11.0, but it's only available as of 8.0.0-pre.20240911.1. - bazelbuild/bazel: Update rules_java 7.11.1 / java_tools 13.8 bazelbuild/bazel#23571 bazelbuild/bazel@f92124a --- What I believe is happening is, under Bazel 7.3.2 and `rules_java` 7.12.1: - Bazel creates `rules_java` 7.9.0 as `@rules_java_builtin` in the `WORKSPACE` prefix. - `@bazel_tools` has `@rules_java` mapped to `@rules_java_builtin` when initialized during the `WORKSPACE` prefix, during which `@bazel_tools//tools/jdk` registers `alias()` targets to `@rules_java` toolchain targets. These aliased toolchains specify `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` in their `toolchains` attribute. - `WORKSPACE` loads `@rules_java` 7.12.1 and registers all its toolchains with type `@rules_java//toolchains:bootstrap_runtime_toolchain_type`. - Some `@rules_java` rules explicitly specifying toolchains from `@bazel_tools//tools/jdk` can't find them, because the `@bazel_tools//tools/jdk` toolchain aliases expect toolchains of type `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`. This has broken other projects in the same way: - bazelbuild/bazel: [Bazel CI] Downstream project broken by rules_java upgrade #23619 bazelbuild/bazel#23619 These problems don't appear under Bzlmod, and `@rules_java_builtin` was never required. This is because `WORKSPACE` executes its statements sequentially, while Bzlmod builds the module dependency graph _before_ instantiating repositories (within module extensions). It seems a fix is on the way that removes `@rules_java_builtin` from the `WORKSPACE` prefix, and adds `@rules_java` to the suffix. At this moment, though, it's not even in a prerelase: - bazelbuild/bazel: Remove rules_java_builtin in WORKSPACE prefix bazelbuild/bazel@7506690 --- Note that the error message matches that from the following resolved issue, but that issue was for non-Bzlmod child repos when `WORKSPACE` was disabled. - bazelbuild/bazel: Undefined @@rules_java_builtin repository with --noenable_workspace option bazelbuild/bazel#22754

lbcjbb added type: bug untriaged labels Jun 14, 2024

lbcjbb assigned iancha1992, satyanandak and sgowroji Jun 14, 2024

github-actions bot added the team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. label Jun 14, 2024

sgowroji unassigned sgowroji, iancha1992 and satyanandak Jun 15, 2024

Wyverald added P1 I'll work on this now. (Assignee required) and removed untriaged labels Jun 17, 2024

Wyverald self-assigned this Jun 17, 2024

Wyverald mentioned this issue Jun 17, 2024

Loading top-level targets in local_path_override modules in child directory breaks the build #22208

Open

Wyverald added a commit that referenced this issue Jun 18, 2024

Do not eval WORKSPACE in LocalRepositoryLookupFunction when `--noenab…

253ceac

…le_workspace` This still leaves the question of "what do we do instead?". See issues #22208 and #21515. Fixes #22754.

Wyverald mentioned this issue Jun 18, 2024

Do not eval WORKSPACE in LocalRepositoryLookupFunction when --noenable_workspace #22774

Closed

copybara-service bot closed this as completed in 1246ff4 Jun 20, 2024

bazel-io mentioned this issue Jun 20, 2024

[7.2.1] Do not eval WORKSPACE in LocalRepositoryLookupFunction when --noenable_workspace #22837

Merged

iancha1992 mentioned this issue Jun 21, 2024

[7.3.0] Do not eval WORKSPACE in LocalRepositoryLookupFunction when `--noenab… #22855

Closed

bazel-io mentioned this issue Jun 27, 2024

[7.3.0] Do not eval WORKSPACE in LocalRepositoryLookupFunction when --noenable_workspace #22911

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Undefined @@rules_java_builtin repository with --noenable_workspace option #22754

Undefined @@rules_java_builtin repository with --noenable_workspace option #22754

lbcjbb commented Jun 14, 2024

Wyverald commented Jun 17, 2024

iancha1992 commented Jun 21, 2024

lbcjbb commented Jun 26, 2024 •

edited

Loading

Undefined @@rules_java_builtin repository with --noenable_workspace option #22754

Undefined @@rules_java_builtin repository with --noenable_workspace option #22754

Comments

lbcjbb commented Jun 14, 2024

Description of the bug:

Which category does this issue belong to?

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Which operating system are you running Bazel on?

What is the output of bazel info release?

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

What's the output of git remote get-url origin; git rev-parse HEAD ?

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

Have you found anything relevant by searching the web?

Any other information, logs, or outputs that you want to share?

Wyverald commented Jun 17, 2024

iancha1992 commented Jun 21, 2024

lbcjbb commented Jun 26, 2024 • edited Loading

What is the output of `bazel info release`?

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

What's the output of `git remote get-url origin; git rev-parse HEAD` ?

lbcjbb commented Jun 26, 2024 •

edited

Loading