etcd operator tests fails on 4.4 cluster periodic job

[amitkrout]

/kind bug

What versions of software are you using?

Operating System:

Output of odo version:

How did you run odo exactly?

Actual behavior

https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-odo-master-v4.4-integration-e2e-periodic/1294514240794660864#1:build-log.txt%3A714

Expected behavior

Any logs, error output, etc?

 + make test-operator-hub
ginkgo  -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -focus="odo service command tests" tests/integration/operatorhub/
Running Suite: Operatorhub Suite
================================
Random Seed: 1597479383 - Will randomize all specs
Will run 15 of 15 specs
Running odo with args [odo preference set Experimental true]
[odo] I0815 08:16:25.448705   23075 preference.go:209] The path for preference file is /tmp/artifacts/.odo/preference.yaml
[odo] Global preference was successfully updated
Creating a new project: kovwmvgpzl
Running odo with args [odo project create kovwmvgpzl -w -v4]
[odo]  •  Waiting for project to come up  ...
[odo] I0815 08:16:26.769400   23096 occlient.go:542] Status of creation of project kovwmvgpzl is Active
[odo] I0815 08:16:26.769488   23096 occlient.go:547] Project kovwmvgpzl now exists
[odo] I0815 08:16:26.784519   23096 namespace.go:181] Status of creation of service account &ServiceAccount{ObjectMeta:{default  kovwmvgpzl /api/v1/namespaces/kovwmvgpzl/serviceaccounts/default 63a25ce0-bd2a-49a9-8a12-b2d0f41e89ce 129087 0 2020-08-15 08:16:25 +0000 UTC <nil> <nil> map[] map[] [] []  []},Secrets:[]ObjectReference{ObjectReference{Kind:,Namespace:,Name:default-token-l9qmw,UID:,APIVersion:,ResourceVersion:,FieldPath:,},ObjectReference{Kind:,Namespace:,Name:default-dockercfg-wv9c9,UID:,APIVersion:,ResourceVersion:,FieldPath:,},},ImagePullSecrets:[]LocalObjectReference{LocalObjectReference{Name:default-dockercfg-wv9c9,},},AutomountServiceAccountToken:nil,} is ready
[odo] 
 ✓  Waiting for project to come up [1s]
[odo]  ✓  Project 'kovwmvgpzl' is ready for use
[odo]  ✓  New project created and now using project: kovwmvgpzl
Running odo with args [odo project set kovwmvgpzl]
[odo] Already on project : kovwmvgpzl
Running oc with args [oc get csv]
[oc] No resources found in kovwmvgpzl namespace.
Running oc with args [oc get csv]
[oc] No resources found in kovwmvgpzl namespace.
Running oc with args [oc get csv]
[oc] No resources found in kovwmvgpzl namespace.
Running oc with args [oc get csv]
[oc] No resources found in kovwmvgpzl namespace.
Running oc with args [oc get csv]
[oc] NAME                                  DISPLAY                    VERSION     REPLACES   PHASE
[oc] mongodb-enterprise.v1.6.0             MongoDB                    1.6.0                  
[oc] service-binding-operator.v0.1.1-352   Service Binding Operator   0.1.1-352              
Running oc with args [oc get csv]
[oc] NAME                                  DISPLAY                    VERSION     REPLACES   PHASE
[oc] mongodb-enterprise.v1.6.0             MongoDB                    1.6.0                  
[oc] service-binding-operator.v0.1.1-352   Service Binding Operator   0.1.1-352              Succeeded
Deleting project: kovwmvgpzl
Running odo with args [odo project delete kovwmvgpzl -f]
[odo]  ⚠  Warning! Projects are deleted from the cluster asynchronously. Odo does its best to delete the project. Due to multi-tenant clusters, the project may still exist on a different node.
[odo]  ✓  Deleted project : kovwmvgpzl
• Failure in Spec Setup (JustBeforeEach) [67.319 seconds]
odo service command tests for OperatorHub
/go/src/github.com/openshift/odo/tests/integration/operatorhub/cmd_service_test.go:17
  JSON output
  /go/src/github.com/openshift/odo/tests/integration/operatorhub/cmd_service_test.go:299
    listing catalog of services [JustBeforeEach]
    /go/src/github.com/openshift/odo/tests/integration/operatorhub/cmd_service_test.go:309
    Timeout after 1 minutes
    /go/src/github.com/openshift/odo/tests/helper/helper_generic.go:41
------------------------------ 

[amitkrout]

Able to reproduce it locally but this time mongodb stuck instead of etcd

EDIT: I used cluster-bot to spinnup the latest 4.4.0-0.ci

[prietyc123]

@amitkrout I tested it locally on 4.4 cluster but all specs passed for me.

$ oc version
Client Version: 4.4.0-0.ci-2020-02-26-215902
Kubernetes Version: v1.17.1

$ make test-operator-hub
ginkgo  -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -focus="odo service command tests" tests/integration/operatorhub/
Running Suite: Operatorhub Suite
================================
Random Seed: 1597523248 - Will randomize all specs
Will run 15 of 15 specs

••••••••••••

•••
Ran 15 of 15 Specs in 1012.916 seconds
SUCCESS! -- 15 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS

Ginkgo ran 1 suite in 17m2.170847324s
Test Suite Passed

[amitkrout]

@amitkrout I tested it locally on 4.4 cluster but all specs passed for me.

@prietyc123 Which cluster server version you used ? The config file we use in CI for cluster version is an ephemeral OpenShift Release, for example 4.4.0-0.ci builds.

May be none of the operator mongodb and etcd are stable in the latest openshift 4.4.0-0.ci build.

[amitkrout]

etcd operator is broken on 4.3 periodic job - https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-odo-master-v4.3-integration-e2e-periodic/1294786046050963456#1:build-log.txt%3A769

[amitkrout]

I have verified all the three operators against the latest stable release version of 4.2, 4.3, 4.4 and 4.5 without any issue. I have created a pr openshift/release#11013 to run odo test on the latest stable released version now onward.

Moreover we should run our test on the sable server version instead of latest ci or nightly build. Ofcourse for unreleased server version like 4.6 we have to relay on ephemeral ci builds