Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow overriding the hostname and instance name in bytestream:// URIs #13085

Conversation

EdSchouten
Copy link
Contributor

In cases where full network transparency doesn't exist, people may run
Bazel with custom values of --remote_executor and --remote_instance_name
to proxy gRPC traffic. Such proxies may do caching, tunneling and
authentication.

What's problematic about this is that the values of these command line
flags aren't just used to establish a gRPC connection to a remote
execution service. They also get logged by Bazel in build event streams
in the form of bytestream:// URIs. This means that a build event stream
generated on system A may contain bytestream:// URIs that are not valid
on system B.

Let's introduce a command line flag, --remote_bytestream_uri_prefix,
that can be used to force generation of bytestream:// URIs that are
canonical.

@google-cla google-cla bot added the cla: yes label Feb 22, 2021
@EdSchouten EdSchouten force-pushed the eschouten/20210222-bytestream-url branch from 7804106 to 6c29268 Compare February 22, 2021 07:48
@EdSchouten
Copy link
Contributor Author

EdSchouten commented Feb 22, 2021

CI failure is caused by #13087.

EDIT: Rebased!

@EdSchouten EdSchouten force-pushed the eschouten/20210222-bytestream-url branch from 6c29268 to c96c80a Compare February 22, 2021 11:12
In cases where full network transparency doesn't exist, people may run
Bazel with custom values of --remote_executor and --remote_instance_name
to proxy gRPC traffic. Such proxies may do caching, tunneling and
authentication.

What's problematic about this is that the values of these command line
flags aren't just used to establish a gRPC connection to a remote
execution service. They also get logged by Bazel in build event streams
in the form of bytestream:// URIs. This means that a build event stream
generated on system A may contain bytestream:// URIs that are not valid
on system B.

Let's introduce a command line flag, --remote_bytestream_uri_prefix,
that can be used to force generation of bytestream:// URIs that are
canonical.
@EdSchouten EdSchouten force-pushed the eschouten/20210222-bytestream-url branch from c96c80a to 0a95da8 Compare February 22, 2021 14:51
@aiuto aiuto added the team-Remote-Exec Issues and PRs for the Execution (Remote) team label Feb 22, 2021
@EdSchouten EdSchouten mentioned this pull request Feb 26, 2021
9 tasks
@bazel-io bazel-io closed this in 8dbbde0 Mar 12, 2021
philwo pushed a commit that referenced this pull request Mar 15, 2021
In cases where full network transparency doesn't exist, people may run
Bazel with custom values of --remote_executor and --remote_instance_name
to proxy gRPC traffic. Such proxies may do caching, tunneling and
authentication.

What's problematic about this is that the values of these command line
flags aren't just used to establish a gRPC connection to a remote
execution service. They also get logged by Bazel in build event streams
in the form of bytestream:// URIs. This means that a build event stream
generated on system A may contain bytestream:// URIs that are not valid
on system B.

Let's introduce a command line flag, --remote_bytestream_uri_prefix,
that can be used to force generation of bytestream:// URIs that are
canonical.

Closes #13085.

PiperOrigin-RevId: 362508252
philwo pushed a commit that referenced this pull request Mar 15, 2021
In cases where full network transparency doesn't exist, people may run
Bazel with custom values of --remote_executor and --remote_instance_name
to proxy gRPC traffic. Such proxies may do caching, tunneling and
authentication.

What's problematic about this is that the values of these command line
flags aren't just used to establish a gRPC connection to a remote
execution service. They also get logged by Bazel in build event streams
in the form of bytestream:// URIs. This means that a build event stream
generated on system A may contain bytestream:// URIs that are not valid
on system B.

Let's introduce a command line flag, --remote_bytestream_uri_prefix,
that can be used to force generation of bytestream:// URIs that are
canonical.

Closes #13085.

PiperOrigin-RevId: 362508252
@EdSchouten EdSchouten deleted the eschouten/20210222-bytestream-url branch May 7, 2021 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes team-Remote-Exec Issues and PRs for the Execution (Remote) team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants