Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Silent failure when credentials file is malformed #18755

Closed
stgpetrovic opened this issue Jun 23, 2023 · 3 comments
Closed

Silent failure when credentials file is malformed #18755

stgpetrovic opened this issue Jun 23, 2023 · 3 comments
Assignees
Labels
P1 I'll work on this now. (Assignee required) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: bug untriaged

Comments

@stgpetrovic
Copy link

Description of the bug:

When using remote build/remote caching, bazel can authenticate using a --google_credentials=path/to/creds.json flag to point to a JSON file containing credentials to authenticate with gcloud. If this flag points to a file that exists but is not valid json (e.g. empty), the bazel server crashes IllegalArgumentException and the client silently exists with nothing to report.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Please see the following repository: https://github.com/stgpetrovic/bazel-creds-errors

You can find reproduce.sh there, with a full explanation in its README.md.

For having everything in one place, though, here's a quick way to reproduce it. I have a suggested fix there, but did not want to spend much time on it (e.g. adding test) before it's validated from your side. Let me know if you'd like me to contribute that fix, or a similar one.

Thanks!

B_TMP_DIR="/tmp/test_bazel_xxxxxxxxxxxxxx12"
rm -rf $B_TMP_DIR; mkdir $B_TMP_DIR; pushd $B_TMP_DIR
touch WORKSPACE
echo -e "cc_binary(name='main', srcs='main.cc')" >> BUILD
echo -e "void main() {}" >> main.cc
touch /tmp/mycredsfile.23213211.json
bazel build //:main --google_credentials=/tmp/mycredsfile.23213211.json --remote_cache=grpcs://remotebuildexecution.googleapis.com
echo $?
rm /tmp/mycredsfile.23213211.json
popd
rm -rf $B_TMP_DIR

Note that if the file does not exist (the IOException path) is OK since that gets caught and reported.

B_TMP_DIR="/tmp/test_bazel_xxxxxxxxxxxxxx12"
rm -rf $B_TMP_DIR; mkdir $B_TMP_DIR; pushd $B_TMP_DIR
touch WORKSPACE
echo -e "cc_binary(name='main', srcs='main.cc')" >> BUILD
echo -e "void main() {}" >> main.cc
rm /tmp/mycredsfile.23213211.json
bazel build //:main --google_credentials=/tmp/mycredsfile.23213211.json --remote_cache=grpcs://remotebuildexecution.googleapis.com
echo $?
rm /tmp/mycredsfile.23213211.json
popd
rm -rf $B_TMP_DIR

Which operating system are you running Bazel on?

Linux 6.1.25-1rodete1-amd64

What is the output of bazel info release?

release 6.2.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

@meteorcloudy meteorcloudy added the team-Remote-Exec Issues and PRs for the Execution (Remote) team label Jun 23, 2023
@meteorcloudy meteorcloudy added the P1 I'll work on this now. (Assignee required) label Jun 23, 2023
@meteorcloudy
Copy link
Member

@stgpetrovic Thanks for reporting and debugging this problem!

@coeuvre Since this is a crashing issue in Bazel that is very hard to debug, I'm assigning P1 to this issue. We should probably cherry pick the fix back to 6.3.0 when it's fixed.

@meteorcloudy
Copy link
Member

@bazel-io fork 6.3.0

@coeuvre
Copy link
Member

coeuvre commented Jun 26, 2023

@meteorcloudy SG, i am looking into this now.

iancha1992 pushed a commit to iancha1992/bazel that referenced this issue Jun 26, 2023
to prevent Bazel sever from crashing.

The functions used to construct credentials could throw runtime exception which will crash Bazel server in which case the Bazel client will just exit silently.

Fixes bazelbuild#18755.

Closes bazelbuild#18770.

PiperOrigin-RevId: 543413376
Change-Id: I71238345f350caaac51f9cc9f654661a90cf6737
iancha1992 added a commit that referenced this issue Jun 27, 2023
to prevent Bazel sever from crashing.

The functions used to construct credentials could throw runtime exception which will crash Bazel server in which case the Bazel client will just exit silently.

Fixes #18755.

Closes #18770.

PiperOrigin-RevId: 543413376
Change-Id: I71238345f350caaac51f9cc9f654661a90cf6737

Co-authored-by: Chi Wang <chiwang@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 I'll work on this now. (Assignee required) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: bug untriaged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants