Restrict reported overlaps to enforced policies #269

JustinKuli · 2024-06-18T17:20:58Z

Previously, informed OperatorPolicies could report that they were overlapping, but in those cases it is not necessary to prevent the policies from operating normally. Now, only overlapping enforced policies will be considered invalid.

Refs:

https://issues.redhat.com/browse/ACM-12207

Previously, informed OperatorPolicies could report that they were overlapping, but in those cases it is not necessary to prevent the policies from operating normally. Now, only overlapping *enforced* policies will be considered invalid. Refs: - https://issues.redhat.com/browse/ACM-12207 Signed-off-by: Justin Kulikauskas <jkulikau@redhat.com>

JustinKuli · 2024-06-18T17:59:43Z

KinD tests (latest) Attempt 1

  [FAIL] Testing OperatorPolicy Test CRD deletion delayed because of a finalizer [It] Initially behaves correctly as musthave [supports-hosted]
  /home/runner/work/config-policy-controller/config-policy-controller/test/e2e/case38_install_operator_test.go:2673

• [FAILED] [18.070 seconds]
Testing OperatorPolicy Test CRD deletion delayed because of a finalizer [It] Initially behaves correctly as musthave [supports-hosted]
/home/runner/work/config-policy-controller/config-policy-controller/test/e2e/case38_install_operator_test.go:2657

  Timeline >>
  STEP: Creating the parent object @ 06/18/24 17:40:28.261
  STEP: Creating the child object with the owner reference @ 06/18/24 17:40:28.385
  STEP: Verifying the child object exists @ 06/18/24 17:40:28.39
  STEP: Waiting for a CRD to appear, which should indicate the operator is installing @ 06/18/24 17:40:28.458
  [FAILED] in [It] - /home/runner/work/config-policy-controller/config-policy-controller/test/e2e/case38_install_operator_test.go:2673 @ 06/18/24 17:40:34.525
  [FAILED] in [AfterAll] - /home/runner/work/config-policy-controller/config-policy-controller/test/e2e/e2e_suite_test.go:198 @ 06/18/24 17:40:34.59
  Debug info for failure.
  policy JSON: {
    "apiVersion": "policy.open-cluster-management.io/v1beta1",
    "kind": "OperatorPolicy",
    "metadata": {
      "annotations": {
        "policy.open-cluster-management.io/parent-policy-compliance-db-id": "124",
        "policy.open-cluster-management.io/policy-compliance-db-id": "64"
      },
      "creationTimestamp": "2024-06-18T17:40:28Z",
      "generation": 2,
      "labels": {
        "policy.open-cluster-management.io/cluster-name": "managed",
        "policy.open-cluster-management.io/cluster-namespace": "managed"
      },
      "name": "oppol-mustnothave",
      "namespace": "managed",
      "ownerReferences": [
        {
          "apiVersion": "policy.open-cluster-management.io/v1",
          "kind": "Policy",
          "name": "parent-policy",
          "uid": "a9ab7e5c-74f6-4267-aafa-7901a0709af2"
        }
      ],
      "resourceVersion": "8885",
      "uid": "d4baf786-3a4e-4a8e-8b92-9bb8da163215"
    },
    "spec": {
      "complianceConfig": {
        "catalogSourceUnhealthy": "Compliant",
        "deploymentsUnavailable": "NonCompliant",
        "upgradesAvailable": "Compliant"
      },
      "complianceType": "musthave",
      "remediationAction": "enforce",
      "removalBehavior": {
        "clusterServiceVersions": "Delete",
        "customResourceDefinitions": "Delete",
        "operatorGroups": "DeleteIfUnused",
        "subscriptions": "Delete"
      },
      "severity": "medium",
      "subscription": {
        "channel": "stable-3.10",
        "name": "project-quay",
        "namespace": "operator-policy-testns",
        "source": "operatorhubio-catalog",
        "sourceNamespace": "olm"
      },
      "upgradeApproval": "Automatic"
    },
    "status": {
      "compliant": "Compliant",
      "conditions": [
        {
          "lastTransitionTime": "2024-06-18T17:40:28Z",
          "message": "CatalogSource was found",
          "reason": "CatalogSourcesFound",
          "status": "False",
          "type": "CatalogSourcesUnhealthy"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:32Z",
          "message": "ClusterServiceVersion (quay-operator.v3.10.6) - install strategy completed with no errors",
          "reason": "InstallSucceeded",
          "status": "True",
          "type": "ClusterServiceVersionCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:33Z",
          "message": "Compliant; the policy spec is valid, the OperatorGroup matches what is required by the policy, the Subscription matches what is required by the policy, no InstallPlans requiring approval were found, ClusterServiceVersion (quay-operator.v3.10.6) - install strategy completed with no errors, no CRDs were found for the operator, all operator Deployments have their minimum availability, CatalogSource was found",
          "reason": "Compliant",
          "status": "True",
          "type": "Compliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:33Z",
          "message": "no CRDs were found for the operator",
          "reason": "RelevantCRDNotFound",
          "status": "True",
          "type": "CustomResourceDefinitionCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:32Z",
          "message": "all operator Deployments have their minimum availability",
          "reason": "DeploymentsAvailable",
          "status": "True",
          "type": "DeploymentCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:33Z",
          "message": "no InstallPlans requiring approval were found",
          "reason": "NoInstallPlansRequiringApproval",
          "status": "True",
          "type": "InstallPlanCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:28Z",
          "message": "the OperatorGroup matches what is required by the policy",
          "reason": "OperatorGroupMatches",
          "status": "True",
          "type": "OperatorGroupCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:28Z",
          "message": "the Subscription matches what is required by the policy",
          "reason": "SubscriptionMatches",
          "status": "True",
          "type": "SubscriptionCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:28Z",
          "message": "the policy spec is valid",
          "reason": "PolicyValidated",
          "status": "True",
          "type": "ValidPolicySpec"
        }
      ],
      "relatedObjects": [
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "CatalogSource",
            "metadata": {
              "name": "operatorhubio-catalog",
              "namespace": "olm"
            }
          },
          "reason": "Resource found as expected"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "ClusterServiceVersion",
            "metadata": {
              "name": "quay-operator.v3.10.6",
              "namespace": "operator-policy-testns"
            }
          },
          "properties": {
            "uid": "06e9a002-3319-4048-863a-343cfc9fb749"
          },
          "reason": "InstallSucceeded"
        },
        {
          "compliant": "UnknownCompliancy",
          "object": {
            "apiVersion": "apiextensions.k8s.io/v1",
            "kind": "CustomResourceDefinition",
            "metadata": {
              "name": "-"
            }
          },
          "reason": "No relevant CustomResourceDefinitions found"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "apps/v1",
            "kind": "Deployment",
            "metadata": {
              "name": "quay-operator-tng",
              "namespace": "operator-policy-testns"
            }
          },
          "properties": {
            "uid": "343f6ceb-fe82-411f-8338-f4ecdae88dfb"
          },
          "reason": "Deployment Available"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "InstallPlan",
            "metadata": {
              "name": "install-9rdpk",
              "namespace": "operator-policy-testns"
            }
          },
          "properties": {
            "uid": "5c983ad4-7432-4609-80be-44f5e8ceec86"
          },
          "reason": "The InstallPlan is Complete"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1",
            "kind": "OperatorGroup",
            "metadata": {
              "name": "operator-policy-testns-vcd8w",
              "namespace": "operator-policy-testns"
            }
          },
          "properties": {
            "createdByPolicy": true,
            "uid": "0707f5eb-3d45-4317-98ef-19937c986d42"
          },
          "reason": "Resource found as expected"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "Subscription",
            "metadata": {
              "name": "project-quay",
              "namespace": "operator-policy-testns"
            }
          },
          "properties": {
            "createdByPolicy": true,
            "uid": "0af4d88e-62ca-4907-b34d-9c613988602b"
          },
          "reason": "Resource found as expected"
        }
      ],
      "resolvedSubscriptionLabel": "project-quay.operator-policy-testns"
    }
  }
  wanted related objects: [{Properties:<nil> Object:{Metadata:{Name:quayregistries.quay.redhat.com Namespace:} Kind:CustomResourceDefinition APIVersion:apiextensions.k8s.io/v1} Compliant:Compliant Reason:Resource found as expected}]
  wanted condition: {Type:CustomResourceDefinitionCompliant Status:True ObservedGeneration:0 LastTransitionTime:0001-01-01 00:00:00 +0000 UTC Reason:RelevantCRDFound Message:there are CRDs present for the operator}

  Debug info for failure.
  policy JSON: {
    "apiVersion": "policy.open-cluster-management.io/v1beta1",
    "kind": "OperatorPolicy",
    "metadata": {
      "annotations": {
        "policy.open-cluster-management.io/parent-policy-compliance-db-id": "124",
        "policy.open-cluster-management.io/policy-compliance-db-id": "64"
      },
      "creationTimestamp": "2024-06-18T17:40:28Z",
      "generation": 2,
      "labels": {
        "policy.open-cluster-management.io/cluster-name": "managed",
        "policy.open-cluster-management.io/cluster-namespace": "managed"
      },
      "name": "oppol-mustnothave",
      "namespace": "managed",
      "ownerReferences": [
        {
          "apiVersion": "policy.open-cluster-management.io/v1",
          "kind": "Policy",
          "name": "parent-policy",
          "uid": "a9ab7e5c-74f6-4267-aafa-7901a0709af2"
        }
      ],
      "resourceVersion": "8873",
      "uid": "d4baf786-3a4e-4a8e-8b92-9bb8da163215"
    },
    "spec": {
      "complianceConfig": {
        "catalogSourceUnhealthy": "Compliant",
        "deploymentsUnavailable": "NonCompliant",
        "upgradesAvailable": "Compliant"
      },
      "complianceType": "musthave",
      "remediationAction": "enforce",
      "removalBehavior": {
        "clusterServiceVersions": "Delete",
        "customResourceDefinitions": "Delete",
        "operatorGroups": "DeleteIfUnused",
        "subscriptions": "Delete"
      },
      "severity": "medium",
      "subscription": {
        "channel": "stable-3.10",
        "name": "project-quay",
        "namespace": "operator-policy-testns",
        "source": "operatorhubio-catalog",
        "sourceNamespace": "olm"
      },
      "upgradeApproval": "Automatic"
    },
    "status": {
      "compliant": "Compliant",
      "conditions": [
        {
          "lastTransitionTime": "2024-06-18T17:40:28Z",
          "message": "CatalogSource was found",
          "reason": "CatalogSourcesFound",
          "status": "False",
          "type": "CatalogSourcesUnhealthy"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:32Z",
          "message": "ClusterServiceVersion (quay-operator.v3.10.6) - install strategy completed with no errors",
          "reason": "InstallSucceeded",
          "status": "True",
          "type": "ClusterServiceVersionCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:33Z",
          "message": "Compliant; the policy spec is valid, the OperatorGroup matches what is required by the policy, the Subscription matches what is required by the policy, no InstallPlans requiring approval were found, ClusterServiceVersion (quay-operator.v3.10.6) - install strategy completed with no errors, there are CRDs present for the operator, all operator Deployments have their minimum availability, CatalogSource was found",
          "reason": "Compliant",
          "status": "True",
          "type": "Compliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:32Z",
          "message": "there are CRDs present for the operator",
          "reason": "RelevantCRDFound",
          "status": "True",
          "type": "CustomResourceDefinitionCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:32Z",
          "message": "all operator Deployments have their minimum availability",
          "reason": "DeploymentsAvailable",
          "status": "True",
          "type": "DeploymentCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:33Z",
          "message": "no InstallPlans requiring approval were found",
          "reason": "NoInstallPlansRequiringApproval",
          "status": "True",
          "type": "InstallPlanCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:28Z",
          "message": "the OperatorGroup matches what is required by the policy",
          "reason": "OperatorGroupMatches",
          "status": "True",
          "type": "OperatorGroupCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:28Z",
          "message": "the Subscription matches what is required by the policy",
          "reason": "SubscriptionMatches",
          "status": "True",
          "type": "SubscriptionCompliant"
        },
        {
          "lastTransitionTime": "2024-06-18T17:40:28Z",
          "message": "the policy spec is valid",
          "reason": "PolicyValidated",
          "status": "True",
          "type": "ValidPolicySpec"
        }
      ],
      "relatedObjects": [
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "CatalogSource",
            "metadata": {
              "name": "operatorhubio-catalog",
              "namespace": "olm"
            }
          },
          "reason": "Resource found as expected"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "ClusterServiceVersion",
            "metadata": {
              "name": "quay-operator.v3.10.6",
              "namespace": "operator-policy-testns"
            }
          },
          "properties": {
            "uid": "06e9a002-3319-4048-863a-343cfc9fb749"
          },
          "reason": "InstallSucceeded"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "apiextensions.k8s.io/v1",
            "kind": "CustomResourceDefinition",
            "metadata": {
              "name": "quayregistries.quay.redhat.com"
            }
          },
          "properties": {
            "uid": "44176670-9905-43b7-8dca-cb6af293a5c9"
          },
          "reason": "Resource found as expected"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "apps/v1",
            "kind": "Deployment",
            "metadata": {
              "name": "quay-operator-tng",
              "namespace": "operator-policy-testns"
            }
          },
          "properties": {
            "uid": "343f6ceb-fe82-411f-[8338](https://github.com/open-cluster-management-io/config-policy-controller/actions/runs/9569421187/job/26381922868?pr=269#step:10:8339)-f4ecdae88dfb"
          },
          "reason": "Deployment Available"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "InstallPlan",
            "metadata": {
              "name": "install-9rdpk",
              "namespace": "operator-policy-testns"
            }
          },
          "properties": {
            "uid": "5c983ad4-7432-4609-80be-44f5e8ceec86"
          },
          "reason": "The InstallPlan is Complete"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1",
            "kind": "OperatorGroup",
            "metadata": {
              "name": "operator-policy-testns-vcd8w",
              "namespace": "operator-policy-testns"
            }
          },
          "properties": {
            "createdByPolicy": true,
            "uid": "0707f5eb-3d45-4317-98ef-19937c986d42"
          },
          "reason": "Resource found as expected"
        },
        {
          "compliant": "Compliant",
          "object": {
            "apiVersion": "operators.coreos.com/v1alpha1",
            "kind": "Subscription",
            "metadata": {
              "name": "project-quay",
              "namespace": "operator-policy-testns"
            }
          },
          "properties": {
            "createdByPolicy": true,
            "uid": "0af4d88e-62ca-4907-b34d-9c613988602b"
          },
          "reason": "Resource found as expected"
        }
      ],
      "resolvedSubscriptionLabel": "project-quay.operator-policy-testns"
    }
  }
  << Timeline

  [FAILED] Failed after 1.028s.
  The function passed to Consistently failed at /home/runner/work/config-policy-controller/config-policy-controller/test/e2e/case38_install_operator_test.go:141 with:
  Expected
      <bool>: false
  to be true
  In [It] at: /home/runner/work/config-policy-controller/config-policy-controller/test/e2e/case38_install_operator_test.go:2673 @ 06/18/24 17:40:34.525```

</details>

openshift-ci · 2024-06-18T19:29:36Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JustinKuli, mprahl

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [JustinKuli,mprahl]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot requested review from gparvin and yiraeChristineKim June 18, 2024 17:21

openshift-ci bot added dco-signoff: yes approved labels Jun 18, 2024

mprahl approved these changes Jun 18, 2024

View reviewed changes

openshift-ci bot assigned mprahl Jun 18, 2024

openshift-ci bot added the lgtm label Jun 18, 2024

openshift-merge-bot bot merged commit 9a2ada5 into open-cluster-management-io:main Jun 18, 2024
9 checks passed

magic-mirror-bot bot mentioned this pull request Jun 18, 2024

🤖 Sync from open-cluster-management-io/config-policy-controller: #269 stolostron/config-policy-controller#912

Merged

JustinKuli deleted the 12207-reduce-oppol-overlaps branch July 25, 2024 13:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Restrict reported overlaps to enforced policies #269

Restrict reported overlaps to enforced policies #269

JustinKuli commented Jun 18, 2024

JustinKuli commented Jun 18, 2024

openshift-ci bot commented Jun 18, 2024

Restrict reported overlaps to enforced policies #269

Restrict reported overlaps to enforced policies #269

Conversation

JustinKuli commented Jun 18, 2024

JustinKuli commented Jun 18, 2024

openshift-ci bot commented Jun 18, 2024