Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs site for version 2.24.0.dev0 #265

Merged
merged 1 commit into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,10 @@ Example: [#21004](https://github.com/pantsbuild/pants/pull/21004).

### Google Cloud Functions

Complete platforms are not provided for GCF yet: [#18195](https://github.com/pantsbuild/pants/issues/18195).
1. Adjust `PythonGoogleCloudFunctionRuntime.known_runtimes` as required
2. Run `build-support/bin/generate_faas_complete_platforms.py` to create any new files and update the existing ones, using GCF's published docker images

Example: [#21248](https://github.com/pantsbuild/pants/pull/21248).

## Cherry-pick a pull request to an older version

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
---
title: GitHub Actions aarch64 runners
sidebar_position: 3
---

---

GitHub Actions does not have affordable hosted aarch64 runners.

For a while we enjoyed hardware donated by the Works on ARM program, but that has now been terminated,
so we instead rely on self-hosted EC2 instances.

We use [Runs On](https://runs-on.com/) to simplify the management of these instances. Runs On monitors
requests for runners, launches EC2 instances on the fly to satify those requests, and terminates them
when the workflows complete.

## The Custom AMI

We configure Runs On to launch these instances with a custom AMI that pre-installs the Python interpreters
needed to bootstrap and test Pants on aarch64. This custom AMI is built on top of the standard Runs On
[Ubuntu 22 ARM64](runs-on-v2.2-ubuntu22-full-arm64-*) AMI.

It may occasionally be necessary to update this AMI, for example to pick up updates to the underlying standard
AMI. To do so manually:

### Create a temporary EC2 instance on the standard AMI

In the AWS web console, initiate instance creation, and when prompted for an AMI, search for the
latest standard AMI, as described in the Runs On [images repo](https://github.com/runs-on/runner-images-for-aws).
In practice this means using `runs-on-v2.2-ubuntu22-full-arm64` as the search term, and checking the results
in the Community AMIs tab.

After selecting the AMI, open the Advanced section and set the instance's User Data to:

```bash
#!/bin/bash
systemctl start ssh
```

to ensure that its SSH daemon runs on startup. Then choose "pantsbuild.org.bastion" as the
instance's SSH keypair.

### SSH into the temporary EC2 instance

Once the instance is running, click on "Connect" to get SSH instructions. You will need the
`pantsbuild.org.bastion` private key, which is in our 1Password account.

### Install Pythons on the instance

Once you are in a bash prompt on the instance, run the following:

```bash
sudo apt install -y software-properties-common
sudo add-apt-repository -y ppa:deadsnakes/ppa
sudo apt update
sudo apt install -y \
python3.7 python3.7-dev python3.7-venv \
python3.8 python3.8-dev python3.8-venv \
python3.9 python3.9-dev python3.9-venv
```

to install the necessary Pythons.

### Create an AMI from the instance

From the instance's actions menu in the web console select "Images" and then "Create image".
The image name doesn't strictly matter, but `ubuntu22-full-arm64-python3.7-python3.8-python3.9`
is a good name for consistency (multiple AMIs may have the same name).

We hope to [automate the above via Packer](https://runs-on.com/guides/building-custom-ami-with-packer/)
at some point.

### Update the Runs On config

Edit `.github/runs-on.yml` and update the `ami` field. Note that the logical image name in this file
happens to be the same as the AMI name, but doesn't strictly have to be. This is the name that we
target in the `runs-on` stanza of a CI job in `generate_github_workflows.py`.

Note that the new AMI will only be used in PR CI jobs after the config change is merged into the
target branch, so the CI job for the update itself will still use the old AMI.

### Deregister the old AMI

After the config update has been merged into `main` and any release branches it needs to be in,
we can deregister the old AMI in the AWS web console, to avoid confusion.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: GitHub Actions macOS ARM64 runners
title: GitHub Actions macOS runners
sidebar_position: 2
---

Expand Down Expand Up @@ -203,3 +203,9 @@ should be able to pick up any job with this setting:
- macOS11
- ARM64
```

## Self-hosted macOS X86_64 Runners

As of August 2024 we are also running a self-hosted macOS 10.15 x86_64 runner on MacStadium,
as GitHub Actions does not offer hosted runners for this old OS version. It is set up similarly
to the instructions above, with appropriate changes to the `runs-on` labels and env vars.
2 changes: 2 additions & 0 deletions docs/docs/contributions/releases/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,5 @@

- [Release strategy](./release-strategy.mdx)
- [Release process](./release-process.mdx)
- [GitHub Actions macOS runners](./github-actions-macos-arm64-runners.mdx)
- [GitHub Actions aarch64 runners](./github-actions-macos-aarch64-runners.mdx)
2 changes: 1 addition & 1 deletion docs/docs/contributions/releases/release-process.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ The release commit is the commit that bumps the VERSION string. For `dev`/`a0` r

### Bump the VERSION

From the `main` branch, run `pants --no-watch-filesystem --no-pantsd run src/python/pants_release/start_release.py -- --new 2.99.0.dev1 --release-manager your_github_username --publish` with the relevant version and your own GitHub username.
From the `main` branch, run `pants run src/python/pants_release/start_release.py -- --new 2.9.0.dev1 --release-manager your_github_username --publish` with the relevant version and your own GitHub username.

This will create a pull request that:

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/docker/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ ARG VAR3=default

### Target build stage

When your `Dockerfile` is a multi-stage build file, you may specify which stage to build with the [`--docker-build-target-stage`](../../reference/subsystems/docker.mdx#section-build-target-stage) for all images, or provide a per image setting with the `docker_image` field [`target_stage`](../../reference/targets/docker_image.mdx#target_stage).
When your `Dockerfile` is a multi-stage build file, you may specify which stage to build with the [`--docker-build-target-stage`](../../reference/subsystems/docker.mdx#build_target_stage) for all images, or provide a per image setting with the `docker_image` field [`target_stage`](../../reference/targets/docker_image.mdx#target_stage).

```dockerfile
FROM python:3.8 AS base
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/getting-started/incremental-adoption.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ We would love to help you with adopting Pants. Please reach out through [Slack](

### 1. A basic `pants.toml`

Follow the [Getting Started](./index.mdx) guide to install Pants and [set up an initial `pants.toml`](./initial-configuration.mdx). Validate that running `pants count-loc ::` works properly. If you want to exclude a specific folder at first, you can use the [`pants_ignore`](../../reference/global-options.mdx#section-pants-ignore) option.
Follow the [Getting Started](./index.mdx) guide to install Pants and [set up an initial `pants.toml`](./initial-configuration.mdx). Validate that running `pants count-loc ::` works properly. If you want to exclude a specific folder at first, you can use the [`pants_ignore`](../../reference/global-options.mdx#pants_ignore) option.

Add the [relevant backends](../using-pants/key-concepts/backends.mdx) to `[GLOBAL].backend_packages`.

Expand Down Expand Up @@ -91,7 +91,7 @@ If you're migrating from another system that already uses the name `BUILD`, such

First, by default Pants recognizes `BUILD.extension` for any `extension` as a valid BUILD file. So you can use a name like `BUILD.pants` without changing configuration.

Second, you can [configure](../../reference/global-options.mdx#section-build-patterns) Pants to use a different set of file names entirely:
Second, you can [configure](../../reference/global-options.mdx#build_patterns) Pants to use a different set of file names entirely:

```toml title="pants.toml"
[GLOBAL]
Expand Down
4 changes: 4 additions & 0 deletions docs/docs/go/private-modules/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"label": "Private Modules",
"position": 2
}
46 changes: 46 additions & 0 deletions docs/docs/go/private-modules/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
title: Private Modules
sidebar_position: 1
---

Use Golang modules from private repositories

---
# Using Private Modules in Golang

Pants can build go binaries that use modules from private git repositories.
To do this specify the private repo(s) in `GOPRIVATE` and provide credentials for the git repo in your `$HOME/.netrc`.

Define the `GOPRIVATE` variable in the `subprocess_env_vars` section of your `pants.toml`. The example below shows the `.netrc` file so that git can authentcate. A simple `go.mod` shows the inclusion of the private module, nothing special here.


```toml tab={"label":"pants.toml"}
[GLOBAL]
backend_packages.add = [
"pants.backend.experimental.go",
]


[golang]
subprocess_env_vars = [
'GOPRIVATE=github.com/your-user/your-module/*',
'HOME',
]
extra_tools = [
'git',
]
```

```go tab={"label":"go.mod"}
module consumer

go 1.22

require github.com/your-user/your-repo/your-module v0.0.1
```

``` tab={"label":".netrc"}
machine github.com
login your-user
password your-token
```
2 changes: 1 addition & 1 deletion docs/docs/introduction/welcome-to-pants.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -68,5 +68,5 @@ Pants is making engineering teams productive and happy at a range of companies a
Pants is an open-source software project, developed at [github.com/pantsbuild/pants](https://github.com/pantsbuild/pants). Pants is released under the [Apache License 2.0](https://github.com/pantsbuild/pants/blob/master/LICENSE).

:::note Pants v2 vs. v1
This documentation is for Pants v2, which is a new system built from the ground up, based on lessons from past work on Pants v1, as well valued feedback from the user community. See [\[https://v1.pantsbuild.org](https://v1.pantsbuild.org)](https://v1.pantsbuild.org/) for Pants v1 documentation.
This documentation is for Pants v2, which is a new system built from the ground up, based on lessons from past work on Pants v1, as well valued feedback from the user community. See [https://v1.pantsbuild.org](https://v1.pantsbuild.org/) for Pants v1 documentation.
:::
6 changes: 3 additions & 3 deletions docs/docs/jvm/java-and-scala.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,13 @@ Pants `2.11.x` adds support for choosing JDK and Scala versions per target in yo

#### JDK

JDKs used by Pants are automatically fetched using [Coursier](https://get-coursier.io/), and are chosen using the [`[jvm].jdk` setting](../../reference/subsystems/jvm.mdx#section-jdk) to set a repository-wide default.
JDKs used by Pants are automatically fetched using [Coursier](https://get-coursier.io/), and are chosen using the [`[jvm].jdk` setting](../../reference/subsystems/jvm.mdx#jdk) to set a repository-wide default.

To override the default on a particular target, you can use the [`jdk=` field](../../reference/targets/java_source.mdx#jdk). It can be useful to use the [`parametrize` builtin](../using-pants/key-concepts/targets-and-build-files.mdx#parametrizing-targets) with the `jdk=` field, particularly to run test targets under multiple JDKs.

#### Scala version

The Scala version to use is configured on a resolve-by-resolve basis (see the "Third-party dependencies" section below) using the [`[scala].version_for_resolve` option](../../reference/subsystems/scala.mdx#section-version_for_resolve). The default Scala version for your repository will thus be whichever Scala version is configured for the "default" resolve, which is configured by the [`[jvm].default_resolve` option](../../reference/subsystems/jvm#default_resolve).
The Scala version to use is configured on a resolve-by-resolve basis (see the "Third-party dependencies" section below) using the [`[scala].version_for_resolve` option](../../reference/subsystems/scala.mdx#version_for_resolve). The default Scala version for your repository will thus be whichever Scala version is configured for the "default" resolve, which is configured by the [`[jvm].default_resolve` option](../../reference/subsystems/jvm#default_resolve).

To use multiple Scala versions in a repository, you would define multiple resolves, and then adjust the [`resolve` field](../../reference/targets/scalatest_test.mdx#resolve) of any targets which should be used with the non-`default_resolve` resolve.

Expand Down Expand Up @@ -109,7 +109,7 @@ scala_artifact(

Pants will use the right artifact for the Scala version corresponding for the resolve specified (or the default one).

Pants requires use of a lockfile for thirdparty dependencies. After adding or editing `jvm_artifact` targets, you will need to update affected lockfiles by running `pants generate-lockfiles`. The default lockfile is located at `3rdparty/jvm/default.lock`, but it can be relocated (as well as additional resolves declared) via the [`[jvm].resolves` option](../../reference/subsystems/jvm.mdx#section-resolves).
Pants requires use of a lockfile for thirdparty dependencies. After adding or editing `jvm_artifact` targets, you will need to update affected lockfiles by running `pants generate-lockfiles`. The default lockfile is located at `3rdparty/jvm/default.lock`, but it can be relocated (as well as additional resolves declared) via the [`[jvm].resolves` option](../../reference/subsystems/jvm.mdx#resolves).

:::note Thirdparty symbols and the `packages` argument
To efficiently determine which symbols are provided by thirdparty code (i.e., without hitting the network in order to compute dependencies in the common case), Pants relies on a static mapping of which artifacts provide which symbols, and defaults to treating each `jvm_artifact` as providing symbols within its `group`.
Expand Down
6 changes: 3 additions & 3 deletions docs/docs/jvm/kotlin.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -68,13 +68,13 @@ Pants supports choosing the JDK and Kotlin versions per target in your repositor

#### JDK

JDKs used by Pants are automatically fetched using [Coursier](https://get-coursier.io/), and are chosen using the [`[jvm].jdk` option](../../reference/subsystems/jvm.mdx#section-jdk) to set a repository-wide default.
JDKs used by Pants are automatically fetched using [Coursier](https://get-coursier.io/), and are chosen using the [`[jvm].jdk` option](../../reference/subsystems/jvm.mdx#jdk) to set a repository-wide default.

To override the default on a particular target, you can use the [`jdk=` field](../../reference/targets/kotlin_source.mdx#jdk). It can be useful to use the [`parametrize` builtin](../using-pants/key-concepts/targets-and-build-files.mdx#parametrizing-targets) with the `jdk=` field, particularly to run test targets under multiple JDKs.

#### Kotlin version

The Kotlin version to use is configured on a resolve-by-resolve basis (see the "Third-party dependencies" section below) using the [`[kotlin].version_for_resolve` option](../../reference/subsystems/kotlin.mdx#section-version_for_resolve). The default Kotlin version for your repository will thus be whichever Kotlin version is configured for the "default" resolve, which is configured by the [`[jvm].default_resolve` option](../../reference/subsystems/jvm#default_resolve).
The Kotlin version to use is configured on a resolve-by-resolve basis (see the "Third-party dependencies" section below) using the [`[kotlin].version_for_resolve` option](../../reference/subsystems/kotlin.mdx#version_for_resolve). The default Kotlin version for your repository will thus be whichever Kotlin version is configured for the "default" resolve, which is configured by the [`[jvm].default_resolve` option](../../reference/subsystems/jvm#default_resolve).

Each resolve must contain the following jars for the Kotlin runtime with the version matching the version specified for the resolve in the `[kotlin].version_for_resolve` option:

Expand Down Expand Up @@ -110,7 +110,7 @@ jvm_artifact(
)
```

Pants requires use of a lockfile for third-party dependencies. After adding or editing `jvm_artifact` targets, you will need to update affected lockfiles by running `pants generate-lockfiles`. The default lockfile is located at `3rdparty/jvm/default.lock`, but it can be relocated (as well as additional resolves declared) via the [`[jvm].resolves` option](../../reference/subsystems/jvm.mdx#section-resolves).
Pants requires use of a lockfile for third-party dependencies. After adding or editing `jvm_artifact` targets, you will need to update affected lockfiles by running `pants generate-lockfiles`. The default lockfile is located at `3rdparty/jvm/default.lock`, but it can be relocated (as well as additional resolves declared) via the [`[jvm].resolves` option](../../reference/subsystems/jvm.mdx#resolves).

:::note Thirdparty symbols and the `packages` argument
To efficiently determine which symbols are provided by third-party code (i.e., without hitting the network in order to compute dependencies in the common case), Pants relies on a static mapping of which artifacts provide which symbols, and defaults to treating each `jvm_artifact` as providing symbols within its `group`.
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/python/goals/check.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ When you run `pants check ::`, Pants will skip any files belonging to skipped ta
:::caution MyPy may still try to check the skipped files!
The `skip_mypy` field only tells Pants not to provide the skipped files as direct input to MyPy. But MyPy, by default, will still try to check files that are [dependencies of the direct inputs](https://mypy.readthedocs.io/en/stable/running_mypy.html#following-imports). So if your skipped files are dependencies of unskipped files, they may still be checked.

To change this behavior, use MyPy's [`--follow-imports` option](https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-follow-imports), typically by setting it to `silent`. You can do so either by adding it to the [`args` option](../../../reference/subsystems/mypy.mdx#section-args) in the `[mypy]` section of your Pants config file, or by setting it in [`mypy.ini`](https://mypy.readthedocs.io/en/stable/config_file.html).
To change this behavior, use MyPy's [`--follow-imports` option](https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-follow-imports), typically by setting it to `silent`. You can do so either by adding it to the [`args` option](../../../reference/subsystems/mypy.mdx#args) in the `[mypy]` section of your Pants config file, or by setting it in [`mypy.ini`](https://mypy.readthedocs.io/en/stable/config_file.html).
:::

### First-party type stubs (`.pyi` files)
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/python/goals/test.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -523,7 +523,7 @@ ignore_errors = true

`ignore_errors = true` means that those files will simply be left off of the final coverage report.

(Pants should autodiscover the config file `.coveragerc`. See [coverage-py](../../../reference/subsystems/coverage-py.mdx#section-config-discovery).)
(Pants should autodiscover the config file `.coveragerc`. See [coverage-py](../../../reference/subsystems/coverage-py.mdx#config-discovery).)

There's a proposal for Pants to fix this by generating multiple reports when necessary: [https://github.com/pantsbuild/pants/issues/11137](https://github.com/pantsbuild/pants/issues/11137). We'd appreciate your feedback.
:::
Expand Down
Loading
Loading