Handle hosted mode overlaps

[JustinKuli]

Mostly https://issues.redhat.com/browse/ACM-12032, but also a potential fix for some continues ConstraintsNotSatisfiable problems in the tests.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JustinKuli, mprahl

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [JustinKuli,mprahl]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[JustinKuli]

Issue during first run (made it all the way to the hosted mode test though)

[FAIL] Testing OperatorPolicy Test CRD deletion delayed because of a finalizer [It] Should become compliant after the finalizer is removed [supports-hosted]
/home/runner/work/config-policy-controller/config-policy-controller/test/e2e/case38_install_operator_test.go:2724

• [FAILED] [73.221 seconds]
Testing OperatorPolicy Test CRD deletion delayed because of a finalizer [It] Should become compliant after the finalizer is removed [supports-hosted]
/home/runner/work/config-policy-controller/config-policy-controller/test/e2e/case38_install_operator_test.go:2720

  Timeline >>
  [FAILED] in [It] - /home/runner/work/config-policy-controller/config-policy-controller/test/e2e/case38_install_operator_test.go:2724 @ 06/10/24 16:24:30.51
  Debug info for failure.
  policy JSON: {
    "apiVersion": "policy.open-cluster-management.io/v1beta1",
    "kind": "OperatorPolicy",
    "metadata": {
      "annotations": {
        "policy.open-cluster-management.io/parent-policy-compliance-db-id": "124",
        "policy.open-cluster-management.io/policy-compliance-db-id": "64"
      },
      "creationTimestamp": "2024-06-10T16:23:06Z",
      "generation": 3,
      "labels": {
        "policy.open-cluster-management.io/cluster-name": "managed",
        "policy.open-cluster-management.io/cluster-namespace": "managed"
      },
      "name": "oppol-mustnothave",
      "namespace": "managed",
      "ownerReferences": [
        {
          "apiVersion": "policy.open-cluster-management.io/v1",
          "kind": "Policy",
          "name": "parent-policy",
          "uid": "65e40d37-0746-4224-96ab-26907f501c2b"
        }
      ],
      "resourceVersion": "15156",
      "uid": "8e29a727-55be-4d8c-a215-522b639752ae"
    },
    "spec": {
      "complianceConfig": {
        "catalogSourceUnhealthy": "Compliant",
        "deploymentsUnavailable": "NonCompliant",
        "upgradesAvailable": "Compliant"
      },
      "complianceType": "mustnothave",
      "remediationAction": "enforce",
      "removalBehavior": {
        "clusterServiceVersions": "Delete",
        "customResourceDefinitions": "Delete",
        "operatorGroups": "DeleteIfUnused",
        "subscriptions": "Delete"
      },
      "severity": "medium",
      "subscription": {
        "channel": "stable-3.10",
        "name": "project-quay",
        "namespace": "operator-policy-testns",
        "source": "operatorhubio-catalog",
        "sourceNamespace": "olm"
      },
      "upgradeApproval": "Automatic"
    },
    "status": {
      "compliant": "NonCompliant",
      "conditions": [
        {
          "lastTransitionTime": "2024-06-10T16:23:22Z",
          "message": "MustNotHave policies ignore kind CatalogSource",
          "reason": "CatalogSourceNotApplicable",
          "status": "False",
          "type": "CatalogSourcesUnhealthy"
        },
        {
          "lastTransitionTime": "2024-06-10T16:23:22Z",
          "message": "the ClusterServiceVersion is not present",
          "reason": "ClusterServiceVersionNotPresent",
          "status": "True",
          "type": "ClusterServiceVersionCompliant"
        },
        {
          "lastTransitionTime": "2024-06-10T16:23:22Z",
          "message": "NonCompliant; the policy spec is valid, the OperatorGroup is not present, the Subscription is not present, there are no relevant InstallPlans in the namespace, the ClusterServiceVersion is not present, the CustomResourceDefinition has a deletion timestamp, MustNotHave policies ignore kind Deployment, MustNotHave policies ignore kind CatalogSource",
          "reason": "NonCompliant",
          "status": "False",
          "type": "Compliant"
        },
        {
          "lastTransitionTime": "2024-06-10T16:23:22Z",
          "message": "the CustomResourceDefinition has a deletion timestamp",
          "reason": "CustomResourceDefinitionDeleting",
          "status": "False",
          "type": "CustomResourceDefinitionCompliant"
        },
        {
          "lastTransitionTime": "2024-06-10T16:23:22Z",
          "message": "MustNotHave policies ignore kind Deployment",
          "reason": "DeploymentNotApplicable",
          "status": "True",
          "type": "DeploymentCompliant"
        },
        {
          "lastTransitionTime": "2024-06-10T16:23:22Z",
          "message": "there are no relevant InstallPlans in the namespace",
          "reason": "NoInstallPlansFound",
          "status": "True",
          "type": "InstallPlanCompliant"
        },
        {
          "lastTransitionTime": "2024-06-10T16:23:22Z",
          "message": "the OperatorGroup is not present",
          "reason": "OperatorGroupNotPresent",
          "status": "True",
          "type": "OperatorGroupCompliant"
        },
        {
          "lastTransitionTime": "2024-06-10T16:23:22Z",
          "message": "the Subscription is not present",
          "reason": "SubscriptionNotPresent",
          "status": "True",
          "type": "SubscriptionCompliant"
        },
        {
          "lastTransitionTime": "2024-06-10T16:23:06Z",
          "message": "the policy spec is valid",
          "reason": "PolicyValidated",
          "status": "True",
          "type": "ValidPolicySpec"
        }
      ],
      "relatedObjects": [
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "CatalogSource",
            "metadata": {
              "name": "operatorhubio-catalog",
              "namespace": "olm"
            }
          },
          "properties": {
            "uid": "3cff6adc-14d2-413a-9222-dc1222cb59cb"
          },
          "reason": "Resource found but will not be handled in mustnothave mode"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "ClusterServiceVersion",
            "metadata": {
              "name": "-",
              "namespace": "operator-policy-testns"
            }
          },
          "reason": "Resource not found as expected"
        },
        {
          "compliant": "NonCompliant",
          "object": {
            "apiVersion": "apiextensions.k8s.io/v1",
            "kind": "CustomResourceDefinition",
            "metadata": {
              "name": "quayregistries.quay.redhat.com"
            }
          },
          "reason": "The object is being deleted but has not been removed yet"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "apps/v1",
            "kind": "Deployment",
            "metadata": {
              "name": "quay-operator-tng",
              "namespace": "operator-policy-testns"
            }
          },
          "reason": "Resource not found as expected"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "InstallPlan",
            "metadata": {
              "name": "-",
              "namespace": "operator-policy-testns"
            }
          },
          "reason": "There are no relevant InstallPlans in this namespace"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1",
            "kind": "OperatorGroup",
            "metadata": {
              "name": "-",
              "namespace": "operator-policy-testns"
            }
          },
          "reason": "Resource not found as expected"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "Subscription",
            "metadata": {
              "name": "project-quay",
              "namespace": "operator-policy-testns"
            }
          },
          "reason": "Resource not found as expected"
        }
      ],
      "resolvedSubscriptionLabel": "project-quay.operator-policy-testns"
    }
  }
  wanted related objects: [{Properties:<nil> Object:{Metadata:{Name:- Namespace:} Kind:CustomResourceDefinition APIVersion:apiextensions.k8s.io/v1} Compliant:Compliant Reason:No relevant CustomResourceDefinitions found}]
  wanted condition: {Type:CustomResourceDefinitionCompliant Status:True ObservedGeneration:0 LastTransitionTime:0001-01-01 00:00:00 +0000 UTC Reason:RelevantCRDNotFound Message:no CRDs were found for the operator}

  << Timeline

  [FAILED] Timed out after 60.000s.
  The function passed to Eventually failed at /home/runner/work/config-policy-controller/config-policy-controller/test/e2e/case38_install_operator_test.go:141 with:
  Expected
      <bool>: false
  to be true
  In [It] at: /home/runner/work/config-policy-controller/config-policy-controller/test/e2e/case38_install_operator_test.go:2724 @ 06/10/24 16:24:30.51