Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(gazelle): ensure that gazelle helper modules are on PYTHONPATH #1590

Merged
merged 10 commits into from
Dec 3, 2023

Conversation

aignas
Copy link
Collaborator

@aignas aignas commented Dec 2, 2023

Before this change there was a bug in how the parsing helpers were being
used in case we were using Python 3.11 toolchain, which is using a more
strict version of the entrypoint template. This change adds imports = ["."]
to ensure that the gazelle helper components are on PYTHONPATH and updates
the non-bzlmod tests to run under 3.11.

We also:

  • Change .bazelrc to use explicit __init__.py definition to avoid
    non-reproducible errors in the future.
  • Add a dedicated gazelle_binary that uses DEFAULT_LANGUAGES and
    //python.

Fixes #1589

Before this change there was a bug in how the parsing helpers were being
used in case we were using Python 3.11 toolchain, which is using a more
strict version of the entrypoint template. This change moves the python
code to a different location to ensure that the top level package is
something more unique than just "python" and updates the non-bzlmod
tests to run under 3.11.

We also change ".bazelrc" to use explicit "__init__.py" definition to
avoid non-reproducible errors in the future.

Fixes #1589
This means that we are replicating the setup that would be
used by our users more closely. Note, that there is a seemingly benign
error message:
```console
$ bazel run //:gazelle
...
INFO: Running command line: bazel-bin/gazelle
gazelle: .../rules_python/gazelle/python/lifecycle.go:26:3: pattern helper.zip: matched no files
```
@aignas
Copy link
Collaborator Author

aignas commented Dec 2, 2023

@rickeylev, what do you think about cherry-picking this into a 0.27.1 release?

@aignas aignas force-pushed the fix/gazelle/include-init-py branch from 7d67276 to 0ba7c9c Compare December 2, 2023 16:11
Copy link
Collaborator

@rickeylev rickeylev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cherry-picking this into a 0.27.1 release?

+1, I agree

gazelle/WORKSPACE Outdated Show resolved Hide resolved
gazelle/.bazelrc Show resolved Hide resolved
import sys

# NOTE @aignas 2023-12-02: Use absolute imports with respect to WORKSPACE root.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to not rely on the repo root being added to sys.path -- that's a behavior I plan to disable once the Starlark impl can be enabled.

But, I can understand this is just a bugfix CL, so this is fine for now. Is Gazelle able to under PyInfo.imports, or have some equivalent?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This made me realize that I just needed to use imports = ["."] on the py_binary that is being used here to fix the error and no moving of files is actually needed. Thanks!

@aignas aignas changed the title fix(gazelle): use absolute imports in gazelle helper fix(gazelle): ensure that gazelle helper modules are on PYTHONPATH Dec 3, 2023
@aignas aignas enabled auto-merge December 3, 2023 00:23
@aignas
Copy link
Collaborator Author

aignas commented Dec 3, 2023

The build is failing on Windows with:

(00:26:41) ERROR: C:/b/bk-windows-g69g/bazel/rules-python-python/examples/BUILD.bazel:51:23: Building Python zip: //examples:py_proto_library_example_bzlmod failed: error reading file '@@python_3_11_6_x86_64-pc-windows-msvc//:Lib/__pycache__/compileall.cpython-311.pyc': C:\b\p4ogcjpt\external\python_3_11_6_x86_64-pc-windows-msvc\Lib\__pycache__\compileall.cpython-311.pyc (The process cannot access the file because it is being used by another process)

@aignas aignas added this pull request to the merge queue Dec 3, 2023
Merged via the queue into main with commit bc5a0b0 Dec 3, 2023
6 checks passed
aignas added a commit that referenced this pull request Dec 4, 2023
…1590)

Before this change there was a bug in how the parsing helpers were being
used in case we were using Python 3.11 toolchain, which is using a more
strict version of the entrypoint template. This change adds `imports =
["."]`
to ensure that the gazelle helper components are on PYTHONPATH and
updates
the non-bzlmod tests to run under 3.11.

We also:
* Change `.bazelrc` to use explicit `__init__.py` definition to avoid
  non-reproducible errors in the future.
* Add a dedicated `gazelle_binary` that uses `DEFAULT_LANGUAGES` *and*
  `//python`.

Fixes #1589
philsc pushed a commit to philsc/rules_python that referenced this pull request Dec 5, 2023
…bazelbuild#1527)

Bazel at head enables bzlmod by default, but the requirements.bzl
entry_point functions aren't supported under bzlmod. Until workspace
support is entirely dropped, explicitly disable bzlmod for the
pip_repository_entry_points test.

Work towards bazelbuild#1590
@aignas aignas deleted the fix/gazelle/include-init-py branch January 14, 2024 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

gazelle plugin is not working with Python 3.11 in 0.27.0 release
2 participants