Zookeeper Operator upgrade fails because of readiness check in deploy file

[pbelgundi]

Tried upgrading zookeeper-operator from version 0.2.4 to 0.2.5-rc0 and upgrade got stuck when this readiness check for specified in the operator manifest:

 readinessProbe:
            exec:
              command:
                - stat
                - /tmp/operator-sdk-ready
            initialDelaySeconds: 4
            periodSeconds: 10
            failureThreshold: 1

On removing this check, initial deployment and upgrade both completed successfully.
This check needs to be removed/corrected in operator deployment manifest

[anishakj]

In the case of rolling update of zookeeper-operator, a deadlock occurs that prevents the new pod to become the leader.
Rolling update works as follows:

a. A new Pod is created
b. A new Pod tries to be the leader with call leader.Become function.
c. But the new Pod keeps waiting, as the old Pod is now the leader
d. If a new Pod is not the leader, file /tmp/operator-sdk-ready is will be not created
e. ReadinessProbe will not succeed without file /tmp/operator-sdk-ready
f. This will create a deadlock and rolling updates get stopped.

This can be fixed by removing the readiness probe check from the operator. More details about this can be found at operator-framework/operator-sdk#932