Skip to content

Commit

Permalink
Add remote execution configuration.
Browse files Browse the repository at this point in the history
Enabling remote execution requires passing `--config=cfc-remote`, which
will allow us to roll out remote execution slowly -- and roll it back if
needed.

Several targets and tests are incompatible with remote execution had
have been tagged as such; these targets will be addressed in subsequent
changes:

  * `oci_image` doesn't work with remote execution, though a newer
    version may.
    bazel-contrib/rules_oci#477
  * `pip_parse` builds source dependencies based on the host platform,
    which (a) introduces non-determinism and (b) can result in binaries
    that cannot run on the remote host (e.g., due to incompatible glibc
    versions). We may be able to use annotations to substitute in
    versions of dependencies built by bazel, not pip:
    bazelbuild/rules_python#571.
    But we also might be able to remove the Python targets...

Bug: 321291571
Change-Id: Id4b2ece75f7945736e0e3c6bba056c85842e9618
  • Loading branch information
bmclarnon committed Mar 18, 2024
1 parent 9b64006 commit 1994c11
Show file tree
Hide file tree
Showing 5 changed files with 29 additions and 0 deletions.
12 changes: 12 additions & 0 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,15 @@ build:ubsan --linkopt=-fsanitize=undefined
# work around linker errors.
build:ubsan --linkopt=-fsanitize-link-c++-runtime
build:ubsan --action_env UBSAN_OPTIONS=halt_on_error=1:print_stacktrace=1

# Settings for remote build execution.
build:cfc-remote --experimental_allow_tags_propagation
build:cfc-remote --define=EXECUTOR=remote
build:cfc-remote --remote_executor=grpcs://us-west1-remotebuildexecution.googleapis.com
build:cfc-remote --remote_instance_name=projects/560162532548/instances/rbe_instance
build:cfc-remote --noremote_upload_local_results
build:cfc-remote --google_default_credentials
build:cfc-remote --extra_execution_platforms=//:remote_platform
build:cfc-remote --host_platform=//:remote_platform
build:cfc-remote --platforms=//:remote_platform
build:cfc-remote --jobs=50
13 changes: 13 additions & 0 deletions BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,16 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

platform(
name = "remote_platform",
parents = ["@local_config_platform//:host"],
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:x86_64",
],
exec_properties = {
"container-image": "docker://gcr.io/fcp-infra/fcp-build@sha256:4a33c4b91e3d146ed654a731ee4827629d8f2ebbcad21998082d7555c79a3f42",
"OSFamily": "Linux",
},
)
1 change: 1 addition & 0 deletions containers/sql_server/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ oci_image(
name = "oci_image",
base = "@distroless_cc_debian12",
entrypoint = ["/main"],
tags = ["no-remote-exec"], # https://github.com/bazel-contrib/rules_oci/issues/477
tars = [":tar"],
)

Expand Down
1 change: 1 addition & 0 deletions containers/test_concat/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ oci_image(
name = "oci_image",
base = "@distroless_cc_debian12",
entrypoint = ["/main"],
tags = ["no-remote-exec"], # https://github.com/bazel-contrib/rules_oci/issues/477
tars = [":tar"],
)

Expand Down
2 changes: 2 additions & 0 deletions tff_worker/server/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ filegroup(
py_test(
name = "pipeline_transform_server_test",
srcs = ["pipeline_transform_server_test.py"],
tags = ["no-remote-exec"], # https://github.com/bazelbuild/rules_python/issues/571
deps = [
":pipeline_transform_server",
":tff_transforms",
Expand All @@ -69,6 +70,7 @@ py_library(
py_test(
name = "tff_transforms_test",
srcs = ["tff_transforms_test.py"],
tags = ["no-remote-exec"], # https://github.com/bazelbuild/rules_python/issues/571
deps = [
":tff_transforms",
"//tff_worker/server/testing:checkpoint_test_utils",
Expand Down

0 comments on commit 1994c11

Please sign in to comment.