Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrap kubectl calls to properly output an error #74

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

artem-zinnatullin
Copy link
Contributor

We've started experiencing issues with K8S cluster (unrelated to k8s-buildkite-plugin) and they slip in as kubectl errors in the middle of our build logs on Buildkite like this:

Uploading: /tmp/tmp.N7WoODSLLy/dist/symbols/armeabi-v7a.objdump.gz for arch: armeabi-v7a with sharedObjectName:gl.so to https://upload.bugsnag.com/so-symbol
--
  | error: You must be logged in to the server (Unauthorized)
  | error: You must be logged in to the server (Unauthorized)
  | error: You must be logged in to the server (Unauthorized)
  | error: You must be logged in to the server (Unauthorized)
  | NDK mapping NdkMappings(sharedObjectName=x.so, arch=arm64-v8a, ndkMappingFilePath=/tmp/tmp.N7WoODSLLy/capture_symbols/arm64-v8a.objdump.gz), NdkMappings(sharedObjectName=x.so, arch=armeabi-v7a, ndkMappingFilePath=/tmp/tmp.N7WoODSLLy/capture_symbols/armeabi-v7a.objdump.gz) has been uploaded successfully!
  | ProGuard mapping /tmp/tmp.X9Eev60Pxz has been uploaded successfully!
  | The operation took 27.154 sec.

This is due to k8s-buildkite-plugin outputting logs in parallel to the while true control loops where we check job/pod status, etc and ANY kubectl command can fail and print an error that doesn't at all indicate that it is coming from kubectl 🙃

PR wraps all kubectl calls, captures stderr output and prints it with the kubectl command parameters so that it is very clear

  1. kubectl failed
  2. What kubectl command failed (I've checked the invocations, parameter values don't seem to include any sensitive info like tokens)

@artem-zinnatullin
Copy link
Contributor Author

Friendly ping @tgolsson 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant