Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix k8s_wait_for_job_completed #873

Merged
merged 2 commits into from
Dec 10, 2018
Merged

Conversation

andresmgot
Copy link
Contributor

@andresmgot andresmgot commented Dec 6, 2018

Fix #869

k8s_wait_for_job_completed was exiting prematurely. Check the comment below.

Tested on https://circleci.com/gh/kubeapps/kubeapps/4881

@@ -58,12 +58,11 @@ k8s_wait_for_job_completed() {
while [ "$retryTimeSeconds" -gt 0 ]; do
# Avoid to exit the function if the job is not completed yet
set +e
kubectl get jobs -n $namespace -l $labelSelector \
-o jsonpath='{.items[*].status.conditions[?(@.type=="Complete")].status}' | grep "True"
kubectl get jobs -n $namespace -l $labelSelector -o jsonpath='{.items[*].status.conditions[?(@.type=="Complete")].status}' | grep "True"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the line split since I found that adding a trailing space \ make it fail. It's not the cause of the error here but it's safer to have it in just one line.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it? Why? I mean we use line split everywhere. What error were you seeing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean that while I was copying this code I accidentally added a trailing space after the \. The code in the repo was fine but while I was debugging I found that somehow I introduced the trailing space. To avoid some issue like that in the future I removed the splitting of the command. It's not that important though, we can keep using \ if the command is too long.

res=$?
set -e
# There is a job that finished
if [[ $res ]]; then
if [[ "$res" -eq "0" ]]; then
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the error:

▶ a=1; if [[ $a ]]; then echo foo; fi
foo

▶ a=0; if [[ $a ]]; then echo foo; fi
foo

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @andresmgot, that makes sense.

I believe that the underlying issue is that we overrode the e option around kubectl.... | grep true so the script did not fail right? In any case we needed the set +e for some reason I do not remember.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case we needed the set +e for some reason I do not remember.

The command kubectl get jobs may fail because the job has not been created yet. That's why we need to "catch" errors there so we can retry.

Copy link
Contributor

@prydonius prydonius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for fixing this!

Copy link
Contributor

@migmartri migmartri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We found this one!

Thanks!

@@ -58,12 +58,11 @@ k8s_wait_for_job_completed() {
while [ "$retryTimeSeconds" -gt 0 ]; do
# Avoid to exit the function if the job is not completed yet
set +e
kubectl get jobs -n $namespace -l $labelSelector \
-o jsonpath='{.items[*].status.conditions[?(@.type=="Complete")].status}' | grep "True"
kubectl get jobs -n $namespace -l $labelSelector -o jsonpath='{.items[*].status.conditions[?(@.type=="Complete")].status}' | grep "True"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it? Why? I mean we use line split everywhere. What error were you seeing?

res=$?
set -e
# There is a job that finished
if [[ $res ]]; then
if [[ "$res" -eq "0" ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @andresmgot, that makes sense.

I believe that the underlying issue is that we overrode the e option around kubectl.... | grep true so the script did not fail right? In any case we needed the set +e for some reason I do not remember.

@andresmgot andresmgot merged commit b047f3e into vmware-tanzu:master Dec 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants