Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCPBUGS-6710: remediations: Normalize remediations' annotations to avoid needlesly marking remediations as Outdated #206

Merged
merged 1 commit into from
Feb 8, 2023

Conversation

jhrozek
Copy link

@jhrozek jhrozek commented Jan 27, 2023

Because the remediation contents we Get() from the cluster after they've
been applied contain an empty Annotations map, but the remediations we
parse from content do not, the comparison always yielded false and the
remediatins have been set to Outdated as a result.

This patch adds an empty annotations map to remediations before
comparing them, so that we compare their normalized version.

note: I have not ran the e2e test addition yet. I did test the PR manually, though.

@openshift-ci-robot
Copy link
Collaborator

@jhrozek: This pull request references Jira Issue OCPBUGS-6710, which is invalid:

  • expected the bug to target the "4.13.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Because the remediation contents we Get() from the cluster after they've
been applied contain an empty Annotations map, but the remediations we
parse from content do not, the comparison always yielded false and the
remediatins have been set to Outdated as a result.

This patch adds an empty annotations map to remediations before
comparing them, so that we compare their normalized version.

note: I have not ran the e2e test addition yet. I did test the PR manually, though.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jhrozek
Copy link
Author

jhrozek commented Jan 31, 2023

/jira refresh

@openshift-ci-robot
Copy link
Collaborator

@jhrozek: This pull request references Jira Issue OCPBUGS-6710, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.13.0) matches configured target version for branch (4.13.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Copy link

@Vincent056 Vincent056 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this issue!
So it was the annotation that is causing the remediation to become outdated?

/lgtm

@jhrozek
Copy link
Author

jhrozek commented Feb 1, 2023

Thanks for fixing this issue! So it was the annotation that is causing the remediation to become outdated?

/lgtm

Yes, as far as I could tell, the remediation we were about to apply (the new version) had annotations: nil and the remediation we compared against (that we read from the server) had annotations: map[string]string

@xiaojiey
Copy link
Collaborator

xiaojiey commented Feb 1, 2023

/hold

@xiaojiey
Copy link
Collaborator

xiaojiey commented Feb 2, 2023

Verification pass with 4.13.0-0.nightly-2023-01-31-174014+ code in the PR:

  1. Deploy compliance operator with the code in the PR:
  2. Create a ss and a ssb:
    $ oc apply -f -<<EOF

apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSetting
metadata:
name: test
namespace: openshift-compliance
rawResultStorage:
nodeSelector:
node-role.kubernetes.io/master: ""
pvAccessModes:

  • ReadWriteOnce
    rotation: 3
    size: 1Gi
    tolerations:
  • effect: NoSchedule
    key: node-role.kubernetes.io/master
    operator: Exists
  • effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  • effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  • effect: NoSchedule
    key: node.kubernetes.io/memory-pressure
    operator: Exists
    roles:
  • worker
    scanTolerations:
  • operator: Exists
    schedule: 0 1 * * *
    showNotApplicable: false
    strictNodeScan: true
    scanLimits: {
    "cpu": "150m",
    "memory": "512Mi"
    }
    debug: true
    autoApplyRemediations: false
    autoUpdateRemediations: false
    EOF
    scansetting.compliance.openshift.io/test created

$ oc apply -f -<<EOF

apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
name: testprofile
namespace: openshift-compliance
spec:
description: test
title: test
enableRules:
- name: ocp4-kubelet-eviction-thresholds-set-hard-imagefs-available
rationale: test
- name: ocp4-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree
rationale: test
- name: ocp4-kubelet-eviction-thresholds-set-hard-memory-available
rationale: test
- name: ocp4-kubelet-eviction-thresholds-set-hard-nodefs-available
rationale: test
- name: ocp4-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree
rationale: test
- name: ocp4-kubelet-eviction-thresholds-set-soft-imagefs-available
rationale: test
- name: ocp4-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree
rationale: test
- name: ocp4-kubelet-eviction-thresholds-set-soft-memory-available
rationale: test
- name: ocp4-kubelet-eviction-thresholds-set-soft-nodefs-available
rationale: test
- name: ocp4-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree
rationale: test
setValues:
- name: ocp4-var-kubelet-evictionhard-imagefs-available
value: "15%"
rationale: test
- name: ocp4-var-kubelet-evictionhard-imagefs-inodesfree
value: "5%"
rationale: test
- name: ocp4-var-kubelet-evictionhard-memory-available
value: "100Mi"
rationale: test
- name: ocp4-var-kubelet-evictionhard-nodefs-available
value: "10%"
rationale: test
- name: ocp4-var-kubelet-evictionhard-nodefs-inodesfree
value: "5%"
rationale: test
- name: ocp4-var-kubelet-evictionsoft-imagefs-available
value: "15%"
rationale: test
- name: ocp4-var-kubelet-evictionsoft-imagefs-inodesfree
value: "5%"
rationale: test
- name: ocp4-var-kubelet-evictionsoft-memory-available
value: "100Mi"
rationale: test
- name: ocp4-var-kubelet-evictionsoft-nodefs-available
value: "10%"
rationale: test
- name: ocp4-var-kubelet-evictionsoft-nodefs-inodesfree
value: "5%"
rationale: test
EOF
tailoredprofile.compliance.openshift.io/testprofile created

$ oc compliance bind -N test -S test tailoredprofile/testprofile
Creating ScanSettingBinding test

$ oc get suite -w
NAME PHASE RESULT
test RUNNING NOT-AVAILABLE
test AGGREGATING NOT-AVAILABLE
test DONE NON-COMPLIANT
test DONE NON-COMPLIANT
$ oc get cr
NAME STATE
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree NotApplied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-1 NotApplied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-2 NotApplied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-3 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-1 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-2 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-3 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-4 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-5 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-1 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-2 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-3 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-4 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-5 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-1 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-2 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-3 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-4 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-5 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-1 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-2 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-3 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-4 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-5 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-1 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-2 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-3 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-4 NotApplied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-5 NotApplied

  1. Edit the cr to apply most of the remediations:
    $ oc get cr
    NAME STATE
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree Applied
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-1 Applied
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-2 Applied
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-3 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-1 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-2 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-3 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-4 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-5 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-1 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-2 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-3 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-4 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-5 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available-1 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available-2 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available-3 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available-4 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available-5 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-1 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-2 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-3 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-4 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-5 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-1 NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-2 NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-3 NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-4 NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-5 NotApplied
    $ oc get mcp -w
    NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE
    master rendered-master-fcaed468c0ac46bfc04edf7acb065df8 False True False 3 0 0 0 70m
    worker rendered-worker-ca09aeeb75afb2ee99378ba6f6531fcf False True False 3 0 0 0 70m
    ...
    worker rendered-worker-465ab54a82e9e673cd58421090546c11 True False False 3 3 3 0 84m
    master rendered-master-2db6488091ea5ffb58fc05ff2407b32f True False False 3 3 3 0 87m
    ####trigger a re-scan:
    $ oc compliance rerun-now scansettingbinding test
    Rerunning scans from 'test': testprofile
    Re-running scan 'openshift-compliance/testprofile'
    $ oc get suite -w
    NAME PHASE RESULT
    test RUNNING NOT-AVAILABLE
    test AGGREGATING NOT-AVAILABLE
    test DONE NON-COMPLIANT
    test DONE NON-COMPLIANT
    ##check result
    $ oc get ccr
    NAME STATUS SEVERITY
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available FAIL medium
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree PASS medium
    testprofile-kubelet-eviction-thresholds-set-hard-memory-available FAIL medium
    testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available FAIL medium
    testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree FAIL medium
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available PASS medium
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree PASS medium
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available PASS medium
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available PASS medium
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree FAIL medium
    ###Recheck the cr status, no cr in outdated status:
    $ oc get cr
    NAME STATE
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available-1 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available-2 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available-3 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree Applied
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-1 Applied
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-2 Applied
    testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-3 Applied
    testprofile-kubelet-eviction-thresholds-set-hard-memory-available NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-memory-available-1 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-memory-available-2 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-memory-available-3 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available-1 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available-2 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available-3 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-1 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-2 NotApplied
    testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-3 NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-1 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-2 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-3 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-4 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-5 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-1 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-2 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-3 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-4 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-5 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available-1 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available-2 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available-3 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available-4 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-memory-available-5 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-1 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-2 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-3 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-4 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-5 Applied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-1 NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-2 NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-3 NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-4 NotApplied
    testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-5 NotApplied

##Manually apply all the remediations,
$ oc get cr
NAME STATE
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-1 Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-2 Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-3 Applied
testprofile-kubelet-eviction-thresholds-set-hard-memory-available Applied
testprofile-kubelet-eviction-thresholds-set-hard-memory-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-hard-memory-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-hard-memory-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-1 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-2 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-4 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-5 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-1 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-2 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-4 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-5 Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-4 Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-5 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-4 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-5 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-1 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-2 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-4 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-5 Applied
$ oc get mcp -w
NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE
master rendered-master-2db6488091ea5ffb58fc05ff2407b32f False True False 3 0 0 0 103m
worker rendered-worker-465ab54a82e9e673cd58421090546c11 False True False 3 0 0 0 103m
...
$ oc get mcp
NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE
master rendered-master-2f6a787e9a8b2a0eb64d8db82ef00fe0 True False False 3 3 3 0 127m
worker rendered-worker-340c4fa4e3601a2c001018891e03405f True False False 3 3 3 0 127m
Step 4: Check Final result:
$ oc compliance rerun-now scansettingbinding test
Rerunning scans from 'test': testprofile
Re-running scan 'openshift-compliance/testprofile'
$ oc get suite -w
NAME PHASE RESULT
test RUNNING NOT-AVAILABLE
test AGGREGATING NOT-AVAILABLE
test DONE COMPLIANT
test DONE COMPLIANT

$ oc get ccr
NAME STATUS SEVERITY
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available PASS medium
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree PASS medium
testprofile-kubelet-eviction-thresholds-set-hard-memory-available PASS medium
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available PASS medium
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree PASS medium
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available PASS medium
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree PASS medium
testprofile-kubelet-eviction-thresholds-set-soft-memory-available PASS medium
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available PASS medium
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree PASS medium
$ oc get cr
NAME STATE
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-1 Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-2 Applied
testprofile-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-3 Applied
testprofile-kubelet-eviction-thresholds-set-hard-memory-available Applied
testprofile-kubelet-eviction-thresholds-set-hard-memory-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-hard-memory-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-hard-memory-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-1 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-2 Applied
testprofile-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-4 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-available-5 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-1 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-2 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-4 Applied
testprofile-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-5 Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-4 Applied
testprofile-kubelet-eviction-thresholds-set-soft-memory-available-5 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-1 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-2 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-4 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-available-5 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-1 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-2 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-3 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-4 Applied
testprofile-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-5 Applied

@openshift-ci openshift-ci bot removed the lgtm label Feb 3, 2023
@jhrozek
Copy link
Author

jhrozek commented Feb 3, 2023

just rebased on recent master

Copy link

@Vincent056 Vincent056 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@jhrozek
Copy link
Author

jhrozek commented Feb 6, 2023

nothing needed from support this is a simple bugfix
@sheriff-rh can you check the changelog entry if it makes sense to you?

@sheriff-rh
Copy link
Collaborator

Thanks @jhrozek .

"On re-running scans, remediations might have been marked as Outdated
despite no changes in the actual remediation content because of
a buggy comparison that did not take into account trivial differences
in remediation metadata"

I am a little confused. "Remediations might have been marked as Outdated" - so remediations start as Outdated, you would run the scan again, and it would still say Outdated because it did not account for differences in metadata? Did it actually run the scan at all?

…oid needlesly marking remediations as Outdated

Because the remediation contents we Get() from the cluster after they've
been applied contain an empty Annotations map, but the remediations we
parse from content do not, the comparison always yielded false and the
remediatins have been set to Outdated as a result.

This patch adds an empty annotations map to remediations before
comparing them, so that we compare their normalized version.
@openshift-ci openshift-ci bot removed the lgtm label Feb 7, 2023
@jhrozek
Copy link
Author

jhrozek commented Feb 7, 2023

Thanks @jhrozek .

"On re-running scans, remediations might have been marked as Outdated despite no changes in the actual remediation content because of a buggy comparison that did not take into account trivial differences in remediation metadata"

I am a little confused. "Remediations might have been marked as Outdated" - so remediations start as Outdated, you would run the scan again, and it would still say Outdated because it did not account for differences in metadata? Did it actually run the scan at all?

Yes, I see how the description was confusing. I reworded it a little, but feel free to suggest even better wording. What happens step-by-step is:

  • the user runs a scan, gets remediations, at first they are NotApplied
  • the user applies the remediations, they are Applied
  • the scan is re-ran either by the user or on schedule
  • even though the remediation did not change at all, the remediation object is now marked as Outdated and the remediation CR contains two remediations, current and outdated. They are the same, which is the crux of the issue - when a remediation is marked as Outdated, the remediation payload should differ and the user should then apply the new one
  • the root cause of the bug was that when CO was comparing the applied remediation payload with what's defined in the DataStream, it would also take into account internal annotations. It should not do that and this is the fix.

Thank you for the review.

Copy link

@rhmdnd rhmdnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor comment on a potential assertion in the test, but we could follow that up.

/lgtm

@openshift-ci
Copy link

openshift-ci bot commented Feb 8, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jhrozek, rhmdnd, Vincent056

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [Vincent056,jhrozek,rhmdnd]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@rhmdnd
Copy link

rhmdnd commented Feb 8, 2023

Removing the hold since the patch was verified by QE.

@rhmdnd
Copy link

rhmdnd commented Feb 8, 2023

/test e2e-aws

Failed due to timeout

@openshift-merge-robot openshift-merge-robot merged commit d1dafd7 into ComplianceAsCode:master Feb 8, 2023
@openshift-ci-robot
Copy link
Collaborator

@jhrozek: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-6710 has been moved to the MODIFIED state.

In response to this:

Because the remediation contents we Get() from the cluster after they've
been applied contain an empty Annotations map, but the remediations we
parse from content do not, the comparison always yielded false and the
remediatins have been set to Outdated as a result.

This patch adds an empty annotations map to remediations before
comparing them, so that we compare their normalized version.

note: I have not ran the e2e test addition yet. I did test the PR manually, though.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants