-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bazel 6.X.X - runfiles output-directories does not respect workspace-name anymore #18128
Comments
This should only be the case with If you use the runfiles libraries (e.g. @Wyverald Should we update the docs? |
@fmeum Thanks for clarification. The flag |
This is a fairly severe breaking change. It becomes impossible to write some code that is backwards compatible to bazel versions before bzlmod. The basic difference is where files are based. Let's say I have a WORKSPACE named "my_project". Under it I have tests/BUILD, which produces the target //tests/data.txt, which is consumed by //tests/data_gen_test. Under the old scheme, that file would be in Problem 1: python runfiles@fmeum pointed out that you need to use the runfiles module for python. While that is true, it is not clear if you should use BUILD
data_gen_test.py
This worked until I tried Problem 2: rules_python runfiles might not be doing the trick to find _main.The previous correct usage to find a file path was Problem 3: What happens when you vendor in modules?OK, this might be a google only problem, we take my_package and vendor it in. runfiles will be wrong when we do that. That is understandable. But let's make it easy to fix on import. One can do that with a copybara transform of something like "rules_pkg/" => "third_party/rules/rules_pkg/". That is harder when we don't have the package name to be the key of the regex replacement. |
@rickeylev This is what has been bitting me all this week. I would love to hear your opinion of what Python runfiles one should use. |
I would also be interested in @rickeylev's opinion. I would personally say we should:
|
The rules_python runfiles library should be used. I'm actually not sure if the one in bazel_tools is as functional; I don't exactly remember when the code was copied over or what changes were put into only rules_python and not bazel_tools. In any case, the one in bazel_tools shouldn't be used.
Either should work. The runfiles library takes the first component and looks it up in the repo mapping file (
This "namespacing" was always a mistake. Implicitly adding the build system's concept of the "location" of a library to the language's runtime identity of a library is a bad thing. The equivalent in java would be e.g. Repo names are not stable. Under workspace, they are user selected. Under bzlmod they are munged, plus there is the repo-mapping, plus multi-version-override stuff.
Yes, that'd be one option, though that also has issues -- it relies on the repo root directory being added to sys.path, which can become exceedingly long (hence why we will eventually disable that by default). The suggested layout in the larger Python community is to have a top-level "src" directory, with everything below that, e.g. |
Sorry, forgot to mention, you can use multiple imports if you need to support multiple versions of bazel and/or rules_python. Something like:
If you want to use the published pypi package, you can do that, too: |
Please see bazelbuild/bazel#18128 Workspace names are going away.
Please see bazelbuild/bazel#18128 Workspace names are going away.
Major change: Bump Bazel to the new LTS at 7.X * Included a default Python toolchain 3.10 There is a problem at the moment running lit using Python311 where it cannot find the module. Our CI system already enforces Python310 to get around it but we should set the toolchain in Bazel itself to avoid this problem. I had already filed llvm/llvm-project#75963 * I moved skylib to MODULE.bazel to test the MODULE portion of Bazel is working. A remaining action item is to move LLVM itself to MODULE.bazel which would preclude needing to define skylib since its a dependency of LLVM. * Workspace names are no longer named in Bazel 7+ I had to change stablehlo to _main to fix the runfiles directory. Please see bazelbuild/bazel#18128 We should consider using https://github.com/bazelbuild/rules_python/blob/main/python/runfiles/runfiles.py#L262 instead to get the runfiles directory for future proofing. Fixes #1878 --------- Co-authored-by: mlevesquedion <mlevesquedion@google.com>
Description of the bug:
Starting with Bazel 6, the output directory for runfiles does not use the worskpace-name for the sub-directory anymore.
From the documentation:
With Bazel 5.X.X:
foo/bar/baz.runfiles/<workspace-name>
With Bazel 6.X.X:
foo/bar/baz.runfiles/_main
What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
No response
Which operating system are you running Bazel on?
Linux
What is the output of
bazel info release
?release 6.1.0
If
bazel info release
returnsdevelopment version
or(@non-git)
, tell us how you built Bazel.No response
What's the output of
git remote get-url origin; git rev-parse master; git rev-parse HEAD
?No response
Have you found anything relevant by searching the web?
No response
Any other information, logs, or outputs that you want to share?
No response
The text was updated successfully, but these errors were encountered: