FATA[0120] Failed to run bundle: install plan is not available for the subscription memcached-operator-v0-0-1-sub: timed out waiting for the condition

[essie-chiang]

Bug Report

What did you do?

follow the doc https://sdk.operatorframework.io/docs/building-operators/golang/quickstart/
to quick start operator.

run
operator-sdk run bundle /memcached-operator-bundle:v0.0.1

What did you expect to see?

it will create run success.

What did you see instead? Under which circumstances?

operator-sdk run bundle quay.io/jysjiangwork/memcached-operator-bundle:v0.0.1

FATA[0004] Failed to run bundle: create catalog: error creating catalog source: namespaces "memcached-operator-system" not found
root@cam-jumphost1:/essie_dir/memcached-operator# oc get nodes
NAME STATUS ROLES AGE VERSION
10.87.119.157 Ready master,worker 113d v1.19.0+4c3480d
10.87.119.187 Ready master,worker 113d v1.19.0+4c3480d
root@cam-jumphost1:/essie_dir/memcached-operator# oc create ns memcached-operator-system
namespace/memcached-operator-system created
root@cam-jumphost1:~/essie_dir/memcached-operator# operator-sdk run bundle quay.io/jysjiangwork/memcached-operator-bundle:v0.0.1
INFO[0016] Successfully created registry pod: quay-io-jysjiangwork-memcached-operator-bundle-v0-0-1
INFO[0016] Created CatalogSource: memcached-operator-catalog
INFO[0016] OperatorGroup "operator-sdk-og" created
INFO[0017] Created Subscription: memcached-operator-v0-0-1-sub
FATA[0120] Failed to run bundle: install plan is not available for the subscription memcached-operator-v0-0-1-sub: timed out waiting for the condition

Environment

Operator type:
/language go

Kubernetes cluster type:

"OpenShift"

$ operator-sdk version

operator-sdk version

operator-sdk version: "v1.15.0", commit: "f6326e832a8a5e5453d0ad25e86714a0de2c0fc8", kubernetes version: "1.21", go version: "go1.16.10", GOOS: "linux", GOARCH: "amd64"

$ go version (if language is Go)

go version go1.16.4 linux/amd64

$ kubectl version

kubectl version

Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0", GitCommit:"2b525e8d2647a41e686bc7da5b7430667a13953e", GitTreeState:"clean", BuildDate:"2021-08-20T22:06:30Z", GoVersion:"go1.15.14", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0+d5ed12c", GitCommit:"d5ed12c2580a64bf7125514c06435b4144a77e31", GitTreeState:"clean", BuildDate:"2021-09-17T23:43:57Z", GoVersion:"go1.15.14", Compiler:"gc", Platform:"linux/amd64"}

Possible Solution

Additional context

oc get po

NAME READY STATUS RESTARTS AGE
quay-io-jysjiangwork-memcached-operator-bundle-v0-0-1 0/1 CrashLoopBackOff 6 7m48s
root@cam-jumphost1:~/essie_dir/memcached-operator# oc describe po quay-io-jysjiangwork-memcached-operator-bundle-v0-0-1
Name: quay-io-jysjiangwork-memcached-operator-bundle-v0-0-1
Namespace: memcached-operator-system
Priority: 0
Node: 10.87.119.187/10.87.119.187
Start Time: Thu, 25 Nov 2021 01:03:58 -0800
Labels:
Annotations: cni.projectcalico.org/podIP: 172.30.32.183/32
cni.projectcalico.org/podIPs: 172.30.32.183/32
k8s.v1.cni.cncf.io/network-status:
[{
"name": "",
"ips": [
"172.30.32.183"
],
"default": true,
"dns": {}
}]
k8s.v1.cni.cncf.io/networks-status:
[{
"name": "",
"ips": [
"172.30.32.183"
],
"default": true,
"dns": {}
}]
openshift.io/scc: anyuid
Status: Running
IP: 172.30.32.183
IPs:
IP: 172.30.32.183
Containers:
registry-grpc:
Container ID: cri-o://b26708b09d2523c4cdf45743340270a245a36c0d479e16086ae26f819b9ea76f
Image: quay.io/operator-framework/opm:latest
Image ID: quay.io/operator-framework/opm@sha256:d31c6ea5c50be93d6eb94d2b508f0208e84a308c011c6454ebf291d48b37df19
Port: 50051/TCP
Host Port: 0/TCP
Command:
sh
-c
mkdir -p /database &&
opm registry add -d /database/index.db -b quay.io/jysjiangwork/memcached-operator-bundle:v0.0.1 --mode=semver --skip-tls=false &&
opm registry serve -d /database/index.db -p 50051

State:          Waiting
  Reason:       CrashLoopBackOff
Last State:     Terminated
  Reason:       Error
  Exit Code:    1
  Started:      Thu, 25 Nov 2021 01:10:35 -0800
  Finished:     Thu, 25 Nov 2021 01:10:39 -0800
Ready:          False
Restart Count:  6
Environment:    <none>
Mounts:
  /var/run/secrets/kubernetes.io/serviceaccount from default-token-hcwm4 (ro)

Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-hcwm4:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-hcwm4
Optional: false
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message

---

Normal Scheduled 8m22s default-scheduler Successfully assigned memcached-operator-system/quay-io-jysjiangwork-memcached-operator-bundle-v0-0-1 to 10.87.119.187
Normal AddedInterface 8m22s multus Add eth0 [172.30.32.183/32]
Normal Pulled 8m15s kubelet Successfully pulled image "quay.io/operator-framework/opm:latest" in 6.926835539s
Normal Pulled 8m10s kubelet Successfully pulled image "quay.io/operator-framework/opm:latest" in 1.325435684s
Normal Pulled 7m44s kubelet Successfully pulled image "quay.io/operator-framework/opm:latest" in 1.309290047s
Normal Created 7m15s (x4 over 8m14s) kubelet Created container registry-grpc
Normal Pulled 7m15s kubelet Successfully pulled image "quay.io/operator-framework/opm:latest" in 1.29886608s
Normal Started 7m14s (x4 over 8m14s) kubelet Started container registry-grpc
Normal Pulling 6m20s (x5 over 8m22s) kubelet Pulling image "quay.io/operator-framework/opm:latest"
Normal Pulled 6m19s kubelet Successfully pulled image "quay.io/operator-framework/opm:latest" in 1.333396829s
Warning BackOff 3m14s (x21 over 8m1s) kubelet Back-off restarting failed container

[varshaprasad96]

@essie-chiang can we get the output of oc get deployments. Looks like atleast one of the component in the installplan is still not running. Also, can you try running the command by increasing the timeout?

[varshaprasad96]

@essie-chiang Also, is the bundle image pullable from the remote repository into the cluster?

[essie-chiang]

@varshaprasad96

docker pull image success:

# docker pull quay.io/jysjiangwork/memcached-operator-bundle:v0.0.2
v0.0.2: Pulling from jysjiangwork/memcached-operator-bundle
Digest: sha256:4a4cbfae46a4d222f3540fd1539e2fe13f7e0716eebf799b1e3eaa2a32e9cdb3
Status: Image is up to date for quay.io/jysjiangwork/memcached-operator-bundle:v0.0.2
quay.io/jysjiangwork/memcached-operator-bundle:v0.0.2

No deploymen in memcached-operator-system namespace

root@cam-jumphost1:~/essie_dir/memcached-operator# oc get deployments
No resources found in memcached-operator-system namespace.

increase timeout to 200m
# operator-sdk run bundle quay.io/jysjiangwork/memcached-operator-bundle:v0.0.2 --timeout 200m ... FATA[1200] Failed to run bundle: install plan is not available for the subscription memcached-operator-v0-0-1-sub: timed out waiting for the condition

[essie-chiang]

The pod finally failed for the image could not pull success, I searched and find it is a RFE in 4.8, is that makes ocp 4.6 or 4.7 the doc could not work, right?

https://bugzilla.redhat.com/show_bug.cgi?id=1883198

[J0zi]

We have the same issue with operator-sdk run bundle quay.io/operatorhubio/hive-operator:v2.5.3508-6cb94c6

[camilamacedo86]

This issue shows to be the scenario clarified and tracked via: #5773. To avoid duplication and centralize the info, it seems like we can close this one in favour of #5773.

Note that some workarounds were also proposed in the issue: #5773. Please, ensure that you check if the proposed workarounds can help you out. If you check that your problem is not the same scenario, we would like to ask for you re-open this issue.

Thank you for your attention and collaboration.