Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

--bes_upload_mode=fully_async is ignored in Bazel 5 #14620

Closed
lukaciko opened this issue Jan 21, 2022 · 1 comment
Closed

--bes_upload_mode=fully_async is ignored in Bazel 5 #14620

lukaciko opened this issue Jan 21, 2022 · 1 comment
Assignees
@coeuvre
Labels
P1 I'll work on this now. (Assignee required) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: bug

Comments

@lukaciko
Copy link

Description of the problem / feature request:

Running a build with the --bes_upload_mode=fully_async doesn't have the expected effect. The build events are still uploaded synchronously after the command is run. This seems to happen when remote cache is used.

Bugs: what's the simplest, easiest way to reproduce this bug?

git clone git@github.com:bazelbuild/rules_java.git && cd rules_java

And then run at least twice so the Bazel server is running and the cache is populated:

env USE_BAZEL_VERSION=5.0.0 bazelisk build --remote_cache=grpcs://remote.buildbuddy.io --bes_backend=grpcs://remote.buildbuddy.io //java:srcs --bes_upload_mode=fully_async

After the build is complete it still takes a bit of time to do something, which I believe is uploading the BEP events. Changing USE_BAZEL_VERSION to 4.2.0 makes it snappier. This upload time is more noticeable in short builds that run more targets.

What operating system are you running Bazel on?

macOS 12.0.1

What's the output of bazel info release?

release 5.0.0

Have you found anything relevant by searching the web?

This seems similar to #14576.
@brentleyjones brentleyjones mentioned this issue Jan 21, 2022
Closed
@aiuto aiuto added team-Remote-Exec Issues and PRs for the Execution (Remote) team untriaged labels Jan 23, 2022
@coeuvre coeuvre self-assigned this Jan 24, 2022
@coeuvre coeuvre added P1 I'll work on this now. (Assignee required) type: bug and removed untriaged labels Jan 24, 2022
@Wyverald Wyverald added this to the 5.1 release blockers milestone Jan 24, 2022
@coeuvre coeuvre mentioned this issue Jan 24, 2022
Closed
@Wyverald
Copy link
Member

Wyverald commented Feb 3, 2022

@bazel-io fork 5.1

@Wyverald Wyverald removed this from the 5.1 release blockers milestone Feb 3, 2022
@bazel-io bazel-io mentioned this issue Feb 3, 2022
Closed
brentleyjones pushed a commit to brentleyjones/bazel that referenced this issue Feb 8, 2022
@coeuvre @brentleyjones
Remote: Only waits for background tasks from remote execution.
f46d98f 
We added the block waiting behaviour after each command in remote module to wait for background uploads when introducing async upload. However, not all background uploads should be waited, e.g. uploads from BES module but with flag `--bes_upload_mode=fully_async`.

This PR updates remote module so that only uploads initiated by remote module are waited after the command. This also enable us to implement something like `--remote_upload_mode=fully_async` in the future.

Fixes bazelbuild#14620.

Closes bazelbuild#14634.

PiperOrigin-RevId: 424296966
(cherry picked from commit 3836ad0)
@brentleyjones brentleyjones mentioned this issue Feb 8, 2022
Merged
Wyverald pushed a commit that referenced this issue Feb 9, 2022
@brentleyjones @coeuvre
Remote: Only waits for background tasks from remote execution. (#14752)
a5f2813 
We added the block waiting behaviour after each command in remote module to wait for background uploads when introducing async upload. However, not all background uploads should be waited, e.g. uploads from BES module but with flag `--bes_upload_mode=fully_async`.

This PR updates remote module so that only uploads initiated by remote module are waited after the command. This also enable us to implement something like `--remote_upload_mode=fully_async` in the future.

Fixes #14620.

Closes #14634.

PiperOrigin-RevId: 424296966
(cherry picked from commit 3836ad0)

Co-authored-by: Chi Wang <chiwang@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@coeuvre coeuvre

Labels
P1 I'll work on this now. (Assignee required) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remote: Only waits for background tasks from remote execution. coeuvre/bazel
4 participants
@Wyverald @aiuto @lukaciko @coeuvre