Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README to say that using requirement() is optional #594

Merged
merged 2 commits into from
Jan 3, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 43 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ Bazel will only fetch/build wheels for the requirements in the subgraph of your

There are API differences between `pip_parse` and `pip_install`:
1. `pip_parse` requires a fully resolved lock file of your python dependencies. You can generate this by using the `compile_pip_requirements` rule,
running `pip-compile` directly, or using virtualenv and `pip freeze`. `pip_parse` uses a label argument called `requirements_lock` instead of
running `pip-compile` directly, or using virtualenv and `pip freeze`. `pip_parse` uses a label argument called `requirements_lock` instead of
`requirements` to make this distinction clear.
2. `pip_parse` translates your requirements into a starlark macro called `install_deps`. You must call this macro in your WORKSPACE to
declare your dependencies.
Expand All @@ -148,12 +148,11 @@ install_deps()

### Consuming `pip` dependencies

Each extracted wheel repo contains a `py_library` target representing the
wheel's contents. Rather than depend on this target's label directly -- which
would require hardcoding the wheel repo's mangled name into your BUILD files --
you should instead use the `requirement()` function defined in the central
repo's `//:requirements.bzl` file. This function maps a pip package name to a
label.
Each extracted wheel repo contains a `py_library` target representing
the wheel's contents. There are two ways to access this library. The
first is using the `requirement()` function defined in the central
repo's `//:requirements.bzl` file. This function maps a pip package
name to a label:

```python
load("@my_deps//:requirements.bzl", "requirement")
Expand All @@ -169,12 +168,37 @@ py_library(
)
```

The reason `requirement()` exists is that the pattern for the labels,
while not expected to change frequently, is not guaranteed to be
stable. Using `requirement()` ensures that you do not have to refactor
your `BUILD` files if the pattern changes.

For reference, the wheel repos are canonically named following the pattern:
`@{central_repo_name}_pypi__{distribution}_{version}`. Characters in the
distribution and version that are illegal in Bazel label names (e.g. `-`, `.`)
are replaced with `_`. While this naming pattern doesn't change often, it is
not guaranted to remain stable, so use of the `requirement()` function is recommended.
On the other hand, using `requirement()` has several drawbacks; see
[this issue][requirements-drawbacks] for an enumeration. If you don't
want to use `requirement()` then you can instead use the library
labels directly. For `pip_parse` the labels are of the form

```
@{name}_{package}//:pkg
```

Here `name` is the `name` attribute that was passed to `pip_parse` and
`package` is the pip package name with characters that are illegal in
Bazel label names (e.g. `-`, `.`) replaced with `_`. If you need to
update `name` from "old" to "new", then you can run the following
buildozer command:

```
buildozer 'substitute deps @old_([^/]+)//:pkg @new_${1}//:pkg' //...
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not work as the target list needs to be specified as //...:* for all of the targets.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point @aignas

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Derp, I ran this locally to test but copy pasted wrong apparently...

```

For `pip_install` the labels are instead of the form

```
@{name}//pypi__{package}
```

[requirements-drawbacks]: https://github.com/bazelbuild/rules_python/issues/414

#### 'Extras' requirement consumption

Expand All @@ -196,15 +220,15 @@ you'd just put `requirement("useful_dep")`.

### Consuming Wheel Dists Directly

If you need to depend on the wheel dists themselves, for instance to pass them
If you need to depend on the wheel dists themselves, for instance to pass them
to some other packaging tool, you can get a handle to them with the `whl_requirement` macro. For example:

```python
filegroup(
name = "whl_files",
data = [
whl_requirement("boto3"),
]
filegroup(
name = "whl_files",
data = [
whl_requirement("boto3"),
]
)
```

Expand Down