Skip to content

Commit

Permalink
Add remote_provider = "..." option, replacing scheme-look-ups (#20240)
Browse files Browse the repository at this point in the history
This replaces the over-complicated address URI scheme-based look-up from
#19827 for deciding on which provider to use with just a normal option
`[GLOBAL].remote_provider`. This defaults to `reapi`, to preserve the
current (stable) behaviour.

That is, now, one can set:

```toml
[GLOBAL]
remote_provider = "experimental-file"
remote_store_address = "file:///..."
```

Or similar.

Another item in #19694 

The commits are somewhat usefully broken up for review.
  • Loading branch information
huonw authored Jan 15, 2024
1 parent 672b1b5 commit c39fbd6
Show file tree
Hide file tree
Showing 22 changed files with 353 additions and 382 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ The values of the `ACTIONS_CACHE_URL` and `ACTIONS_RUNTIME_TOKEN` environment va
uses: actions/github-script@v6
with:
script: |
core.exportVariable('PANTS_REMOTE_STORE_ADDRESS', 'experimental:github-actions-cache+' + (process.env.ACTIONS_CACHE_URL || ''));
core.exportVariable('PANTS_REMOTE_STORE_ADDRESS', process.env.ACTIONS_CACHE_URL);
core.exportVariable('PANTS_REMOTE_OAUTH_BEARER_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN);
```
Expand All @@ -65,6 +65,7 @@ Once the GitHub values are configured, Pants will read the environment variables
```toml
[GLOBAL]
# GitHub Actions cache URL and token are set via environment variables
remote_provider = "experimental-github-actions-cache"
remote_cache_read = true
remote_cache_write = true
```
Expand All @@ -85,7 +86,8 @@ To read and write the cache to `/path/to/cache`, you will need to configure `pan

```toml
[GLOBAL]
remote_store_address = "experimental:file:///path/to/cache"
remote_store_provider = "experimental-file"
remote_store_address = "file:///path/to/cache"
remote_cache_read = true
remote_cache_write = true
```
Expand Down
6 changes: 3 additions & 3 deletions src/python/pants/engine/internals/scheduler.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,6 @@
LOCAL_STORE_LEASE_TIME_SECS,
ExecutionOptions,
LocalStoreOptions,
normalize_remote_address,
)
from pants.util.contextutil import temporary_file_path
from pants.util.logging import LogLevel
Expand Down Expand Up @@ -179,6 +178,7 @@ def __init__(
parsed_javascript_deps_result=NativeParsedJavascriptDependencies,
)
remoting_options = PyRemotingOptions(
provider=execution_options.remote_provider.value,
execution_enable=execution_options.remote_execution,
store_headers=execution_options.remote_store_headers,
store_chunk_bytes=execution_options.remote_store_chunk_bytes,
Expand All @@ -193,8 +193,8 @@ def __init__(
execution_headers=execution_options.remote_execution_headers,
execution_overall_deadline_secs=execution_options.remote_execution_overall_deadline_secs,
execution_rpc_concurrency=execution_options.remote_execution_rpc_concurrency,
store_address=normalize_remote_address(execution_options.remote_store_address),
execution_address=normalize_remote_address(execution_options.remote_execution_address),
store_address=execution_options.remote_store_address,
execution_address=execution_options.remote_execution_address,
execution_process_cache_namespace=execution_options.process_execution_cache_namespace,
instance_name=execution_options.remote_instance_name,
root_ca_certs_path=execution_options.remote_ca_certs_path,
Expand Down
Loading

0 comments on commit c39fbd6

Please sign in to comment.