Skip to content

Commit

Permalink
Add CVE Allowlist expires Test Cases
Browse files Browse the repository at this point in the history 
Fix #18920

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
  • Loading branch information
Yang Jiao committed Jul 12, 2023
1 parent c707106 commit be09540
Show file tree
Hide file tree
Showing 4 changed files with 74 additions and 40 deletions.
7 changes: 7 additions & 0 deletions tests/resources/Harbor-Pages/Configuration.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,13 @@ Delete Top Item In System CVE Allowlist
END
Retry Element Click ${config_security_save_button_xpath}

Set CVE Allowlist Expires
[Arguments] ${expired}
Retry Button Click ${cve_allowlist_expires_btn}
${element}= Set Variable If ${expired} ${cve_allowlist_expires_yesterday} ${cve_allowlist_expires_tomorrow}
Retry Element Click ${element}
Retry Element Click //button[contains(.,'SAVE')]

Get Project Count Quota Text From Project Quotas List
[Arguments] ${project_name}
Switch To Project Quotas
Expand Down
3 changes: 3 additions & 0 deletions tests/resources/Harbor-Pages/Configuration_Elements.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ ${configuration_system_wl_textarea} //*[@id='allowlist-textarea']
${configuration_system_wl_add_confirm_btn} //*[@id='add-to-system']
${configuration_system_wl_delete_a_cve_id_icon} //app-security//form/section//ul/li[1]/a[2]/clr-icon
${configuration_sys_repo_readonly_chb_id} //*[@id='repo_read_only_lbl']
${cve_allowlist_expires_btn} //clr-date-container[.//div[@class='clr-input-group' and not(@hidden)]]//button
${cve_allowlist_expires_yesterday} //td[.//button[@class='day-btn is-today']]/preceding-sibling::td[1]
${cve_allowlist_expires_tomorrow} //td[.//button[@class='day-btn is-today']]/following-sibling::td[1]
${cfg_auth_automatic_onboarding_checkbox} //clr-checkbox-wrapper//label[contains(@for,'oidcAutoOnboard')]
${cfg_auth_user_name_claim_input} //*[@id='oidcUserClaim']

Expand Down
102 changes: 62 additions & 40 deletions tests/resources/TestCaseBody.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -171,93 +171,115 @@ Helm CLI Work Flow
Retry File Should Exist ./${harbor_helm_package}
Helm Registry Logout ${ip}

#Important Note: All CVE IDs in CVE Allowlist cases must unique!
Body Of Verfiy System Level CVE Allowlist
[Arguments] ${image_argument} ${sha256_argument} ${most_cve_list} ${single_cve}
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument}
${sha256}= Set Variable ${sha256_argument}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Go Into Project project${d}
Set Vulnerabilty Serverity 2
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Logout Harbor

Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Check Listed In CVE Allowlist project${d} ${image} ${sha256} ${single_cve} is_in=No
Switch To Configuration Security
Retry Wait Element Visible //li[text()=' None ']
# Add Items To System CVE Allowlist CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528
Add Items To System CVE Allowlist ${most_cve_list}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Add Items To System CVE Allowlist ${most_cve_list}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Add Items To System CVE Allowlist CVE-2021-43519
Add Items To System CVE Allowlist ${single_cve}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Delete Top Item In System CVE Allowlist count=9
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Add Items To System CVE Allowlist ${single_cve}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
# Set System CVE Allowlist expires to expired
Set CVE Allowlist Expires ${True}
Retry Wait Until Page Contains The system CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Set System CVE Allowlist expires to not expired
Set CVE Allowlist Expires ${False}
Retry Wait Until Page Does Not Contains The system CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}

Delete Top Item In System CVE Allowlist count=9
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Check Listed In CVE Allowlist project${d} ${image} ${sha256} ${single_cve}
Close Browser

Body Of Verfiy Project Level CVE Allowlist
[Arguments] ${image_argument} ${sha256_argument} ${most_cve_list} ${single_cve}
[Tags] run-once
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument}
${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument}
${sha256}= Set Variable ${sha256_argument}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Project project${d}
Set Vulnerabilty Serverity 2
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Go Into Project project${d}
Add Items to Project CVE Allowlist ${most_cve_list}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Add Items to Project CVE Allowlist ${single_cve}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Add Items to Project CVE Allowlist ${most_cve_list}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Add Items to Project CVE Allowlist ${single_cve}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
# Set System CVE Allowlist expires to expired
Set CVE Allowlist Expires ${True}
Retry Wait Until Page Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Set System CVE Allowlist expires to not expired
Set CVE Allowlist Expires ${False}
Retry Wait Until Page Does Not Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Delete Top Item In Project CVE Allowlist
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Close Browser

Body Of Verfiy Project Level CVE Allowlist By Quick Way of Add System
[Arguments] ${image_argument} ${sha256_argument} ${cve_list}
[Tags] run-once
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument}
${d}= Get Current Date result_format=%m%s
${image}= Set Variable ${image_argument}
${sha256}= Set Variable ${sha256_argument}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
${signin_user}= Set Variable user025
${signin_pwd}= Set Variable Test1@34
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Configuration Security
Add Items To System CVE Allowlist ${cve_list}
Logout Harbor
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Sign In Harbor ${HARBOR_URL} ${signin_user} ${signin_pwd}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Go Into Project project${d}
Set Vulnerabilty Serverity 2
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Project project${d}
Set Project To Project Level CVE Allowlist
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Add System CVE Allowlist to Project CVE Allowlist By Add System Button Click
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
# Set System CVE Allowlist expires to expired
Set CVE Allowlist Expires ${True}
Retry Wait Until Page Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
# Set System CVE Allowlist expires to not expired
Set CVE Allowlist Expires ${False}
Retry Wait Until Page Does Not Contains The project CVE allowlist has expired. You can enable the allowlist by extending the expiration date.
Close Browser

Body Of Replication Of Push Images to Registry Triggered By Event
Expand Down
2 changes: 2 additions & 0 deletions tests/robot-cases/Group1-Nightly/Trivy.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,11 @@ Test Case - Verfiy System Level CVE Allowlist
Body Of Verfiy System Level CVE Allowlist goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 CVE-2021-43519

Test Case - Verfiy Project Level CVE Allowlist
[Tags] proj_cve
Body Of Verfiy Project Level CVE Allowlist goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 CVE-2021-43519

Test Case - Verfiy Project Level CVE Allowlist By Quick Way of Add System
[Tags] proj_cve_quick_add_sys
Body Of Verfiy Project Level CVE Allowlist By Quick Way of Add System goharbor/harbor-portal 55d776fc7f431cdd008c3d8fc3e090c81c1368ed9ed85335f4664df71f864f0d CVE-2021-36222\nCVE-2021-43527 \nCVE-2021-4044 \nCVE-2021-36084 \nCVE-2021-36085 \nCVE-2021-36086 \nCVE-2021-37750 \nCVE-2021-40528 \nCVE-2021-43519

Test Case - Stop Scan And Stop Scan All
Expand Down

0 comments on commit be09540

Please sign in to comment.