kubectl wait for un-existed resource.

[kvokka]

What happened:

I can not avoid exit with error for uncreated resource, which is misleading by the command name:

kubectl wait --selector=foo=bar --for=condition=complete jobs
kubectl wait --for=condition=complete jobs/foo

Even if this behavior if intentional the user should have an option to continue waiting instead of exit with error code.

In the contrast, if the resource already exists everything works as it should.

What you expected to happen:

Kubectl should at least to have the ability (option) to wait un-existed resource.

Anything else we need to know?:

Connect kubernetes/kubernetes#75227

Environment:

• Kubernetes version (use kubectl version): 1.15.2
• OS (e.g: cat /etc/os-release): mac Os 10.14.6
• Kernel (e.g. uname -a): Darwin Kernel Version 18.7.0

[kvokka]

/sig cli

[rikatz]

Can take a look into this.

/assign @rikatz

[rikatz]

@kvokka just a question: Deleted conditions are also something you're looking for?

I imagine a situation where you wan't a deleted condition for an object that wasn't even created. This seems pretty strange to me :) but let me know if this is also a scenario.

Tks

[kvokka]

@rikatz Thank you for your response!

For me simple wait until timeout is more than enough. If the developer want to control the object persistence/deletion just let him do it. Sounds reasonable?

The example scenario of the expected behaviour is described in this article.

[rikatz]

Right.

It might be pretty trickier than I thought it was, but I'm already taking a look.

The biggest problem is that the function used to 'visit' an object expects it to exists (ResourceFinder.Do().Visit) so I'm taking a look to check if it's possible to 'bypass/loop' into it ;)

[rikatz]

I've made an initial and dirty PR just to see if this is the path to follow :D

[kvokka]

Thank you for the contribution! Will hope the code will be reviewed/merged soon! :)

[rikatz]

OK, so I need some review :/ The dumbest way is what I did...A sleep with 1s. Not sure why ResourceFinder is used here and if something more "flexible" could be used, so need someone with more experience in Kubernetes Code to review that.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[rikatz]

/remove-lifecycle stale

Got some time to resolve some other stuff, but this is still a thing

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[jemc]

/remove-lifecycle stale

[lopezator]

Had the same problem, I am using an script to create all my k8s objects, then wait for a particular pod to be ready.

I have a race condition, when the wait executes the object apparently is not yet created.

This issue force me to sleep for a few seconds before issuing the wait.

kubectl apply -f foo.yaml
sleep 5 # Just to avoid the wait err below
kubectl -n ns wait pod --for=condition=ready -l name=pod --timeout=120s

[remidebette]

/remove-lifecycle stale

[lbruun]

Here is my current workaround using Bash. It differs from the other workarounds published here in that it has a max-wait-time. It uses the return code from kubectl describe to figure out if the resource exist or not.

# Wait for Pods with a certain name prefix to exist.
# The wait is over when at least one Pod exist or the max wait time is reached.
#
#   $1 : namespace
#   $2 : pod name prefix (wildcards not allowed)
#   $3 : the maximum time to wait in seconds
#
# The command is useful in combination with 'kubectl wait' which can wait for a certain condition,
# but cannot wait for existence.
wait_for_pods_to_exist() {
  local ns=$1
  local pod_name_prefix=$2
  local max_wait_secs=$3
  local interval_secs=2
  local start_time=$(date +%s)
  while true; do

    current_time=$(date +%s)
    if (( (current_time - start_time) > max_wait_secs )); then
      echo "Waited for pods in namespace \"$ns\" with name prefix \"$pod_name_prefix\" to exist for $max_wait_secs seconds without luck. Returning with error."
      return 1
    fi

    if kubectl -n $ns describe pod $pod_name_prefix --request-timeout "5s"  &> /dev/null; then
      break
    else
      sleep $interval_secs
    fi
  done
}

.. and use like this in combination with kubectl wait:

# wait up 20 secs for Pod to exist:
wait_for_pods_to_exist "mynamespace" "mypodname" 20
#  ...and then wait for state 'Ready' for up to 30 secs:
kubectl --namespace "mynamespace" wait --for=condition=ready pod/mypodname

(I haven't found a need to wait for any other resource type than pods, but the above can easily be generalized)

I truly believe that 'wait-for-existence' should be something kubectl should be capable of instead of all of us inventing this on our own. To make it clear, people seem to have needs such as:

• Wait for at least 1 to exist (my requirement)
• Wait for exactly X to exist (including zero, for example as described here)

Programmatically, it is easier done in kubectl, i.e. in Go code, than in say Bash. But the main argument for putting it in kubectl is that it seems to be a recurring theme among users.

[mrclrchtr]

@lbruun thx a lot for the script.

A little better could be to use the following instead of waiting for ready state:

kubectl rollout status --namespace mynamespace deployment/mydeploymentname --timeout=600s

This waits for the pod to be completely ready. With waiting only for the ready state, I had problems because it was not completely deployed.

[ardaguclu]

It seems that there is a high demand for that feature and I'd like to try my chance to propose a viable solution. For the backwards compatibility, it would be better to introduce a new flag for this functionality and keep the current behavior without it;

/assign

[ardaguclu]

/triage accepted
/priority backlog
/transfer kubectl

[djmcgreal-cc]

@ardaguclu - any update? Thanks :)

[ardaguclu]

Thanks for the reminder because I forgot this issue :). I'll prioritize this.

[seastco]

this would be great to have. just ran into this and had to cook up some ugly bash scripting.

[Skaronator]

@seastco as you can see this issue has been already fixed and closed. Its available in: v1.32.0-alpha.0 v1.31.1 v1.31.0 v1.31.0-rc.1 v1.31.0-rc.0 v1.31.0-beta.0 v1.31.0-alpha.3

as you can see in commit kubernetes/kubernetes@b95fce1.

[ein-stein-chen]

Note that kubernetes/kubernetes@e24b9a0 (the actual commit implementing this feature) was reverted with kubernetes/kubernetes#125630 and the follow-up attempt (kubernetes/kubernetes#125632) was closed with the plan to instead expand the --for argument to also cover this use case.

So this issue probably should be reopened.

[ardaguclu]

This kubernetes/kubernetes#125868 was the final decision and can be used.