Skip to content

Commit

Permalink
Merge branch 'main' into builtin_goals_in_plugins
Browse files Browse the repository at this point in the history
  • Loading branch information
@tdyas
tdyas committed Jun 26, 2024
2 parents 392e1bc + ecf0a7f commit 46c626a
Show file tree
Hide file tree
Showing 48 changed files with 2,285 additions and 831 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/release.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ jobs:
with:
key: macOS10-15-x86_64-rustup-${{ hashFiles('src/rust/engine/rust-toolchain')
}}-v2
path: '~/.rustup/toolchains/1.78.0-*
path: '~/.rustup/toolchains/1.79.0-*
~/.rustup/update-hashes
Expand Down Expand Up @@ -276,7 +276,7 @@ jobs:
with:
key: macOS11-ARM64-rustup-${{ hashFiles('src/rust/engine/rust-toolchain')
}}-v2
path: '~/.rustup/toolchains/1.78.0-*
path: '~/.rustup/toolchains/1.79.0-*
~/.rustup/update-hashes
Expand Down Expand Up @@ -367,7 +367,7 @@ jobs:
uses: actions/cache@v3
with:
key: Linux-x86_64-rustup-${{ hashFiles('src/rust/engine/rust-toolchain') }}-v2
path: '~/.rustup/toolchains/1.78.0-*
path: '~/.rustup/toolchains/1.79.0-*
~/.rustup/update-hashes
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/test.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ jobs:
uses: actions/cache@v3
with:
key: Linux-ARM64-rustup-${{ hashFiles('src/rust/engine/rust-toolchain') }}-v2
path: '~/.rustup/toolchains/1.78.0-*
path: '~/.rustup/toolchains/1.79.0-*
~/.rustup/update-hashes
Expand Down Expand Up @@ -130,7 +130,7 @@ jobs:
uses: actions/cache@v3
with:
key: Linux-x86_64-rustup-${{ hashFiles('src/rust/engine/rust-toolchain') }}-v2
path: '~/.rustup/toolchains/1.78.0-*
path: '~/.rustup/toolchains/1.79.0-*
~/.rustup/update-hashes
Expand Down Expand Up @@ -232,7 +232,7 @@ jobs:
uses: actions/cache@v3
with:
key: macOS12-x86_64-rustup-${{ hashFiles('src/rust/engine/rust-toolchain') }}-v2
path: '~/.rustup/toolchains/1.78.0-*
path: '~/.rustup/toolchains/1.79.0-*
~/.rustup/update-hashes
Expand Down Expand Up @@ -441,7 +441,7 @@ jobs:
uses: actions/cache@v3
with:
key: macOS10-15-x86_64-rustup-${{ hashFiles('src/rust/engine/rust-toolchain') }}-v2
path: '~/.rustup/toolchains/1.78.0-*
path: '~/.rustup/toolchains/1.79.0-*
~/.rustup/update-hashes
Expand Down Expand Up @@ -508,7 +508,7 @@ jobs:
uses: actions/cache@v3
with:
key: macOS11-ARM64-rustup-${{ hashFiles('src/rust/engine/rust-toolchain') }}-v2
path: '~/.rustup/toolchains/1.78.0-*
path: '~/.rustup/toolchains/1.79.0-*
~/.rustup/update-hashes
Expand Down
1 change: 1 addition & 0 deletions 3rdparty/python/requirements.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ chevron==0.14.0
fasteners==0.16.3
freezegun==1.2.1
ijson==3.2.3
libcst==1.4.0
packaging==21.3
pex==2.3.3
psutil==5.9.8
Expand Down
106 changes: 84 additions & 22 deletions 3rdparty/python/user_reqs.lock
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
// "fasteners==0.16.3",
// "freezegun==1.2.1",
// "ijson==3.2.3",
// "libcst==1.4.0",
// "mypy-typing-asserts==0.1.1",
// "node-semver==0.9.0",
// "packaging==21.3",
Expand Down Expand Up @@ -939,6 +940,66 @@
"requires_python": ">=3.7",
"version": "2.0.0"
},
{
"artifacts": [
{
"algorithm": "sha256",
"hash": "d024f44059a853b4b852cfc04fec33e346659d851371e46fc8e7c19de24d3da9",
"url": "https://files.pythonhosted.org/packages/71/da/16307f14b47f761235050076e1d2954fc7de9346f1410ba8c67a54a9f40e/libcst-1.4.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl"
},
{
"algorithm": "sha256",
"hash": "b8ecdba8934632b4dadacb666cd3816627a6ead831b806336972ccc4ba7ca0e9",
"url": "https://files.pythonhosted.org/packages/7b/b1/8476fe4fa1061062855459d519ffe2115a891638c230ee3465c69fdbfd7a/libcst-1.4.0-cp39-cp39-macosx_10_9_x86_64.whl"
},
{
"algorithm": "sha256",
"hash": "8e54c777b8d27339b70f304d16fc8bc8674ef1bd34ed05ea874bf4921eb5a313",
"url": "https://files.pythonhosted.org/packages/7e/0d/89516795ff2a11be10c060c539895b3781793d46cb7c9b0b7b3c4fa3fbc1/libcst-1.4.0-cp39-cp39-macosx_11_0_arm64.whl"
},
{
"algorithm": "sha256",
"hash": "bb0abf627ee14903d05d0ad9b2c6865f1b21eb4081e2c7bea1033f85db2b8bae",
"url": "https://files.pythonhosted.org/packages/95/cf/a2be91d53e4068d4def8b5cc475f20e1c1a7d32c85634ee7d6b3ea2e3c9b/libcst-1.4.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
},
{
"algorithm": "sha256",
"hash": "061d6855ef30efe38b8a292b7e5d57c8e820e71fc9ec9846678b60a934b53bbb",
"url": "https://files.pythonhosted.org/packages/c0/c8/15ca337e5f5604aabed899609ba08abbc0e7815ffdfca37802da52d4d0bf/libcst-1.4.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"
},
{
"algorithm": "sha256",
"hash": "449e0b16604f054fa7f27c3ffe86ea7ef6c409836fe68fe4e752a1894175db00",
"url": "https://files.pythonhosted.org/packages/e4/bd/ff41d7a8efc4f60a61d903c3f9823565006f44f2b8b11c99701f552b0851/libcst-1.4.0.tar.gz"
}
],
"project_name": "libcst",
"requires_dists": [
"Sphinx>=5.1.1; extra == \"dev\"",
"black==23.12.1; extra == \"dev\"",
"build>=0.10.0; extra == \"dev\"",
"coverage>=4.5.4; extra == \"dev\"",
"fixit==2.1.0; extra == \"dev\"",
"flake8==7.0.0; extra == \"dev\"",
"hypothesis>=4.36.0; extra == \"dev\"",
"hypothesmith>=0.0.4; extra == \"dev\"",
"jinja2==3.1.4; extra == \"dev\"",
"jupyter>=1.0.0; extra == \"dev\"",
"maturin<1.6,>=0.8.3; extra == \"dev\"",
"nbsphinx>=0.4.2; extra == \"dev\"",
"prompt-toolkit>=2.0.9; extra == \"dev\"",
"pyre-check==0.9.18; platform_system != \"Windows\" and extra == \"dev\"",
"pyyaml>=5.2",
"setuptools-rust>=1.5.2; extra == \"dev\"",
"setuptools-scm>=6.0.1; extra == \"dev\"",
"slotscheck>=0.7.1; extra == \"dev\"",
"sphinx-rtd-theme>=0.4.3; extra == \"dev\"",
"ufmt==2.6.0; extra == \"dev\"",
"usort==1.0.8.post1; extra == \"dev\""
],
"requires_python": ">=3.9",
"version": "1.4.0"
},
{
"artifacts": [
{
Expand Down Expand Up @@ -1119,43 +1180,43 @@
"artifacts": [
{
"algorithm": "sha256",
"hash": "28e552a060ba2740d0d2aabe35162652c1459a0b9069fe0db7f4ee0e18e74d58",
"url": "https://files.pythonhosted.org/packages/27/17/a9872f20809e37ad03c523994ef3e0b7420c6508fe4553b7c0d8476ee03a/pydantic-1.10.15-py3-none-any.whl"
"hash": "aa2774ba5412fd1c5cb890d08e8b0a3bb5765898913ba1f61a65a4810f03cf29",
"url": "https://files.pythonhosted.org/packages/b6/85/f23404c7c82ddaa60e83ee0b1ce9c1692a04ed2861c7306ba5d06410608d/pydantic-1.10.16-py3-none-any.whl"
},
{
"algorithm": "sha256",
"hash": "92229f73400b80c13afcd050687f4d7e88de9234d74b27e6728aa689abcf58cc",
"url": "https://files.pythonhosted.org/packages/4f/fa/da8c9468b6c7eb974721af57abbbfd33537b8c29a9a2c0c788a3dc6aea1b/pydantic-1.10.15-cp39-cp39-musllinux_1_1_i686.whl"
"hash": "8bb388f6244809af69ee384900b10b677a69f1980fdc655ea419710cffcb5610",
"url": "https://files.pythonhosted.org/packages/11/af/3fbad4c7f9a1c4410b500cd10ba4a9c4fee38ff341428f17d3e6b01636f9/pydantic-1.10.16.tar.gz"
},
{
"algorithm": "sha256",
"hash": "67f1a1fb467d3f49e1708a3f632b11c69fccb4e748a325d5a491ddc7b5d22383",
"url": "https://files.pythonhosted.org/packages/53/c8/b417d5016955033a7d8b57fb0ec40666cae9a36ce248b875b8d8f8164feb/pydantic-1.10.15-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
"hash": "d8d3c71d14c8bd26d2350c081908dbf59d5a6a8f9596d9ef2b09cc1e61c8662b",
"url": "https://files.pythonhosted.org/packages/1f/18/c83803430a0f4caa9acae30c862899f53750798dbf4de0b95023d1aea037/pydantic-1.10.16-cp39-cp39-macosx_11_0_arm64.whl"
},
{
"algorithm": "sha256",
"hash": "676ed48f2c5bbad835f1a8ed8a6d44c1cd5a21121116d2ac40bd1cd3619746ed",
"url": "https://files.pythonhosted.org/packages/54/b3/3e806647d3ee420ab4e4e0f9c9e215b87ffd3a654ea893ab32da0726c9e5/pydantic-1.10.15-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl"
"hash": "3895ddb26f22bdddee7e49741486aa7b389258c6f6771943e87fc00eabd79134",
"url": "https://files.pythonhosted.org/packages/2a/6a/13542c4ffa7b30dc8fc3a147970d9d31961060fe062228f967d8584df5ce/pydantic-1.10.16-cp39-cp39-musllinux_1_1_i686.whl"
},
{
"algorithm": "sha256",
"hash": "a980a77c52723b0dc56640ced396b73a024d4b74f02bcb2d21dbbac1debbe9d0",
"url": "https://files.pythonhosted.org/packages/58/3c/0606b7c07a531f27da504d547623a31e05146ea28c011fa316d331dfad8e/pydantic-1.10.15-cp39-cp39-macosx_11_0_arm64.whl"
"hash": "9d91f6866fd3e303c632207813ef6bc4d86055e21c5e5a0a311983a9ac5f0192",
"url": "https://files.pythonhosted.org/packages/58/68/1a0798d394116983a732868592ab6810a369a0165c101200b21b5a451f9b/pydantic-1.10.16-cp39-cp39-macosx_10_9_x86_64.whl"
},
{
"algorithm": "sha256",
"hash": "ca832e124eda231a60a041da4f013e3ff24949d94a01154b137fc2f2a43c3ffb",
"url": "https://files.pythonhosted.org/packages/6f/3a/1a057845b787469aa8c4ff2c126d5110d616223c82e74561b8ae56b46ec7/pydantic-1.10.15.tar.gz"
"hash": "b73e6386b439b4881d79244e9fc1e32d1e31e8d784673f5d58a000550c94a6c0",
"url": "https://files.pythonhosted.org/packages/a1/96/3cd54412c7f423571735224bd7e81e1d40e027bef022055daf0a4f5c8e71/pydantic-1.10.16-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
},
{
"algorithm": "sha256",
"hash": "51d405b42f1b86703555797270e4970a9f9bd7953f3990142e69d1037f9d9e51",
"url": "https://files.pythonhosted.org/packages/ac/93/0dc45c885fb3f1dcccc5e2c498f95007defbdad122998286769384afc7ba/pydantic-1.10.15-cp39-cp39-macosx_10_9_x86_64.whl"
"hash": "5f039881fb2ef86f6de6eacce6e71701b47500355738367413ccc1550b2a69cf",
"url": "https://files.pythonhosted.org/packages/d1/f6/a9bac5c07e14e82f7ee525ad45bb9d0038efab91729f408c940ed023d54d/pydantic-1.10.16-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl"
},
{
"algorithm": "sha256",
"hash": "2746189100c646682eff0bce95efa7d2e203420d8e1c613dc0c6b4c1d9c1fde4",
"url": "https://files.pythonhosted.org/packages/b0/23/87acb9a7e70a18730aa38f0f3d12e855581411b4c843531c61f384c904c5/pydantic-1.10.15-cp39-cp39-musllinux_1_1_x86_64.whl"
"hash": "55b945da2756b5cef93d792521ad0d457fdf2f69fd5a2d10a27513f5281717dd",
"url": "https://files.pythonhosted.org/packages/fc/e3/cb85dfd9619f382558f7284c7a93bd669f84782b2c0b810d30c683163223/pydantic-1.10.16-cp39-cp39-musllinux_1_1_x86_64.whl"
}
],
"project_name": "pydantic",
Expand All @@ -1165,7 +1226,7 @@
"typing-extensions>=4.2.0"
],
"requires_python": ">=3.7",
"version": "1.10.15"
"version": "1.10.16"
},
{
"artifacts": [
Expand Down Expand Up @@ -2042,13 +2103,13 @@
"artifacts": [
{
"algorithm": "sha256",
"hash": "450b20ec296a467077128bff42b73080516e71b56ff59a60a02bef2232c4fa9d",
"url": "https://files.pythonhosted.org/packages/a2/73/a68704750a7679d0b6d3ad7aa8d4da8e14e151ae82e6fee774e6e0d05ec8/urllib3-2.2.1-py3-none-any.whl"
"hash": "a448b2f64d686155468037e1ace9f2d2199776e17f0a46610480d311f73e3472",
"url": "https://files.pythonhosted.org/packages/ca/1c/89ffc63a9605b583d5df2be791a27bc1a42b7c32bab68d3c8f2f73a98cd4/urllib3-2.2.2-py3-none-any.whl"
},
{
"algorithm": "sha256",
"hash": "d0570876c61ab9e520d776c38acbbb5b05a776d3f9ff98a5c8fd5162a444cf19",
"url": "https://files.pythonhosted.org/packages/7a/50/7fd50a27caa0652cd4caf224aa87741ea41d3265ad13f010886167cfcc79/urllib3-2.2.1.tar.gz"
"hash": "dd505485549a7a552833da5e6063639d0d177c04f23bc3864e41e5dc5f612168",
"url": "https://files.pythonhosted.org/packages/43/6d/fa469ae21497ddc8bc93e5877702dca7cb8f911e337aca7452b5724f1bb6/urllib3-2.2.2.tar.gz"
}
],
"project_name": "urllib3",
Expand All @@ -2060,7 +2121,7 @@
"zstandard>=0.18.0; extra == \"zstd\""
],
"requires_python": ">=3.8",
"version": "2.2.1"
"version": "2.2.2"
},
{
"artifacts": [
Expand Down Expand Up @@ -2329,6 +2390,7 @@
"fasteners==0.16.3",
"freezegun==1.2.1",
"ijson==3.2.3",
"libcst==1.4.0",
"mypy-typing-asserts==0.1.1",
"node-semver==0.9.0",
"packaging==21.3",
Expand Down
8 changes: 0 additions & 8 deletions docs/docs/contributions/development/maintenance-tasks-and-scripts.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,14 +62,6 @@ Some tools use `NodeJSToolBase` to install executable npm packages. To update th

Example: [#21007](https://github.com/pantsbuild/pants/pull/21007).

## Generate a new JSON schema file

Some editors can use JSON Schema for better completions (etc.) when editing TOML files like `pants.toml`. To generate such a schema:

1. Run `pants help-all > all-help.json`
2. Run `pants run build-support/bin/generate_json_schema.py -- --all-help-file=all-help.json`
3. For a new release, upload the resulting file to https://www.schemastore.org/json/

## Update or create FaaS complete platforms files

The function-as-a-service (FaaS) subsystems provide some built-in PEX complete platforms JSON files, for specific runtimes. To update or create these:
Expand Down
24 changes: 24 additions & 0 deletions docs/docs/contributions/releases/release-process.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,30 @@ Once the workflow finishes, look through any failures and determine if there's a

Alternatively, after starting the workflow, post the link to the in-progress run in `#development` in Slack, so that someone can come back to it when it does finish.

## Step 5: Publish a schema in JSON Schema Store

Some editors can use JSON Schema for better completions when editing TOML files like `pants.toml`.
Pants configuration file schema is published at https://www.schemastore.org/. For every stable `2.x.0` release,
a new schema needs to be generated and uploaded by submitting a PR against https://github.com/SchemaStore/schemastore.
This is an example pull request for reference: https://github.com/SchemaStore/schemastore/pull/3880.

To produce `pantsbuild-<version>.json` schema file, run:

```bash
pants help-all > all-help.json
pants run build-support/bin/generate_json_schema.py -- --all-help-file=all-help.json
```

It may be helpful to compare the last schema file with the newly produced one to make sure there are no discrepancies
(e.g. the config values have a sensible type and the help strings are rendered adequately). You can download the
schemas of previous releases from the store website; the JSON files are available at
`https://json.schemastore.org/pantsbuild-<version>.json`.

Watch out for any configuration parameters that may rely on your local environment as certain default config values
will be expanded using local runtime environment which is undesirable. The script handles those known config values
by keeping a list of them, however, it may need to be extended as more options with environment specific default
values are added.

## When Things Go Wrong

From time to time, a release will fail. It's a complex process. The first thing to do after you've
Expand Down
15 changes: 15 additions & 0 deletions docs/docs/using-pants/key-concepts/targets-and-build-files.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,21 @@ python_distribution(
)
```

### Multiple BUILD files in a directory

Typically, there would be one BUILD file in every directory containing source code and any other resources you
may want to use as part of your builds. Most likely, having just one BUILD file in a directory is also what you would want.
However, you can have multiple BUILD files in a single directory, if desired. When running a Pants goal, the contents
of the BUILD files will be merged making it possible to better group your targets.

Storing targets in multiple BUILD files also makes it possible to dynamically include or exclude targets from your
builds. For example, you could include some experimental targets when running a Pants goal from a command line by
extending the list of recognized BUILD file patterns:

```bash
$ pants --build-patterns="+['BUILD.experimental']" package project:app
```

## Target addresses

A target is identified by its unique address, in the form `path/to/dir:name`. The above example has the addresses `helloworld/greet:tests` and `helloworld/greet:bin`.
Expand Down
19 changes: 19 additions & 0 deletions docs/notes/2.23.x.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,15 @@ The default version of the pex tool has been updated from 2.3.1 to 2.3.3.

Fix running python source files that have dashes in them (bug introduced in 2.20). For example: `pants run path/to/some-executable.py`

A new `entry_point_dependencies` field is now available for `python_tests` and `python_test` targets. This allows tests
to depend on a subset (or all) of the `entry_points` defined on `python_distribution` targets. A dependency defined in
`entry_point_dependencies` emulates an editable install of those `python_distribution` targets. Instead of including
all of the `python_distribution`'s sources, only the specified entry points are made available. The entry_points metadata
is also installed in the pytest sandbox so that tests (or the code under test) can load that metadata via `pkg_resources`.
To use this, enable the `pants.backend.experimental.python` backend.

Exported virtualenvs can use Pants-provided Python if a `PythonProvider` backend is enabled (like `pants.backend.python.providers.experimental.pyenv`). Before Pants 2.23, virtualenv exports could only use pre-installed python binaries.

#### Terraform

The `tfsec` linter now works on all supported platforms without extra config.
Expand All @@ -101,6 +110,14 @@ Nodejs processes configured with `extra_env_vars`, e.g.
now supports extending the `PATH` variable of such processes. Passing `extra_env_vars=["PATH=/usr/bin"]` was previously
silently ignored.

Two issues with pants `corepack` integration has been resolved:
1. The `"packageManager"` package.json field is now respected for other package.json than the one at the build root.
Previously, if for example a nodejs tool was configured with a resolve based off of such a package.json, the bug caused
pants to invoke `corepack`s default versions of the package managers instead.
2. The pants.toml option `[nodejs].package_manager` can now be assigned any of the supported package managers
(npm, pnpm, yarn) without providing a corresponding `[nodejs].package_managers` version setting. The version is then
entirely handled by `corepack`. Previously this mode caused pants to fail.

#### Shell

The `tailor` goal now has independent options for tailoring `shell_sources` and `shunit2_tests` targets. The option was split from `tailor` into [`tailor_sources`](https://www.pantsbuild.org/2.22/reference/subsystems/shell-setup#tailor_sources) and [`tailor_shunit2_tests`](https://www.pantsbuild.org/2.22/reference/subsystems/shell-setup#tailor_shunit2_tests).
Expand Down Expand Up @@ -131,6 +148,8 @@ Plugins may now provide "daemon" goals by implememting the `daemon_goals` functi

The "Provided by" information in the documentation now correctly reflects the proper backend to enable to activate a certain feature.

Metadata for paths in the repository can now be requested via the `PathMetadataRequest` and `PathMetadataResult` types. This API is intended for rules which need access to the "full" metadata for a path.

### New call-by-name syntax for @rules

Pants has a new mechanism for `@rule` invocation in backends. In this release the following backends were migrated to use this new mechanism. There should not be any user-visible effects, but please be on the lookout for any unusual bugs or error messages.
Expand Down
2 changes: 1 addition & 1 deletion src/python/pants/VERSION
View file
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.23.0.dev1
2.23.0.dev2
Loading

0 comments on commit 46c626a

Please sign in to comment.