It is not possible to test automatically che-operator updates

[tolusha]

Describe the bug

To check updates we use the test script [1]
It fails with such an error

./testUpdate.sh openshift stable
 PACKAGE_VERSION=7.11.0 
 CSV=eclipse-che-preview-openshift.v7.11.0 
 Channel=stable 
 Namespace=eclipse-che-preview-test 
Installing test pre-requisistes
namespace/eclipse-che-preview-test unchanged
Apply che-operator source
--- Use default eclipse che application registry ---
apiVersion: operators.coreos.com/v1
kind: OperatorSource
metadata:
  name: eclipse-che-preview-openshift
  namespace: openshift-marketplace
spec:
  authorizationToken: {}
  endpoint: 'https://quay.io/cnr'
  registryNamespace: eclipse-che-operator-openshift
  type: appregistry
error: unable to recognize "/home/tolusha/gocode/src/github.com/eclipse/che-operator/olm/eclipse-che-preview-openshift/operator-source.yaml": no matches for kind "OperatorSource" in version "operators.coreos.com/v1"
Subscribing to version: eclipse-che-preview-openshift.v7.11.0
unable to recognize "STDIN": no matches for kind "OperatorGroup" in version "operators.coreos.com/v1"
unable to recognize "STDIN": no matches for kind "Subscription" in version "operators.coreos.com/v1alpha1"
error: the server doesn't have a resource type "subscription"
error: the server doesn't have a resource type "subscription"
Subscription failed to install the operator

The reason being is that operator-source is removed from latest version of olm.

[1] https://github.com/eclipse/che-operator/blob/master/olm/testUpdate.sh

Environment

• OpenShift 4.4
• Minikube 1.8.2

[flacatus]

Launched on OCP4.4 and I've obtain the same errors. Hower launching the same scripts on OCP4.3 seems is working without any problem.

Installing in OCP 4.4 also I received this event:

Events:                    <none>
error: timed out waiting for the condition on subscriptions/eclipse-che-preview-openshift
Subscription failed to install the operator

[nickboldt]

do we know if the problem is 4.4 itself (different behaviour) or network issues?

[flacatus]

I want to mention I test update from 7.11.0 to 7.12.0 on minikube(1.8.2 version) and seems is working. I change OLM version from 0.12.0 to 0.14.0.

[ibuziuk]

@tolusha setting P1, but please update the labels based on the team priorities

[dmytro-ndp]

An error hasn't been reproduced on latest OCP 4.4-rc.11 cluster.

[nickboldt]

If this is resolved, please update #16690 (comment) when you have news about the PRs being done.

[tolusha]

Fixed olm version to test operator updates on minikube.
Also I was able to test updates on OpenShift 4.5.