Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2025-02-22 23:01:33 +01:00
Code Issues Projects Releases Wiki Activity

Refactor the keyword in the testcase (#18898)

Browse Source 
Fix #17952

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
This commit is contained in:
Yang Jiao 2023-07-06 13:47:00 +08:00 committed by GitHub
parent c8120d5712
commit fb52fdb3e0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
22 changed files with 199 additions and 369 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
tests/apitests/python/test_proxy_cache.py
View File

@ -58,7 +58,7 @@ class TestProxyCache(unittest.TestCase):
image_for_docker = dict(image = "for_proxy", tag = "1.0")
image_for_ctr = dict(image = "redis", tag = "latest")
index_for_docker = dict(image = "index081597864867", tag = "index_tag081597864867")
index_for_docker = dict(image = "index", tag = "index_tag")
access_key = ""
access_secret = ""

4
tests/resources/Harbor-Pages/Project-Config.robot
View File

@ -7,11 +7,8 @@ Resource ../../resources/Util.robot
*** Keywords ***
Goto Project Config
Sleep 3
Retry Element Click //project-detail//ul/li[contains(.,'Summary')]
Sleep 3
Retry Double Keywords When Error Retry Element Click //project-detail//ul/li[contains(.,'Configuration')] Retry Wait Element //clr-checkbox-wrapper/label[contains(.,'Prevent vulnerable images from running.')]
Sleep 2
Click Project Public
Mouse Down //hbr-project-policy-config//input[@name='public']
@ -36,7 +33,6 @@ Click Auto Scan
Mouse Up //hbr-project-policy-config//input[@name='scan-image-on-push']
Save Project Config
Sleep 1
Retry Element Click //hbr-project-policy-config//button[contains(.,'SAVE')]
Public Should Be Selected

24
tests/resources/Harbor-Pages/Project-Members.robot
View File

@ -19,28 +19,6 @@ Resource ../../resources/Util.robot
*** Variables ***
*** Keywords ***
Go Into Project
[Arguments] ${project} ${has_image}=${true}
Retry Keyword N Times When Error 5 Clear Search Input And Go Into Project ${project} ${has_image}
Clear Search Input And Go Into Project
[Arguments] ${project} ${has_image}
# Close prompt in header
Sleep 2
#go To ${url}
Reload Page
Sleep 2
# Clear Search Input
Retry Element Click xpath=//harbor-app/harbor-shell/clr-main-container/navigator/clr-header/div[1]/a/span
Sleep 2
Retry Text Input ${search_input} ${project}
Sleep 2
# Go Into Project
${out} Run Keyword If ${has_image}==${false} Run Keywords Retry Element Click xpath=//*[@id='project-results']//clr-dg-cell[contains(.,'${project}')]/a AND Wait Until Element Is Visible And Enabled xpath=//clr-dg-placeholder[contains(.,\"We couldn\'t find any repositories!\")]
... ELSE Run Keywords Retry Element Click xpath=//*[@id='project-results']//clr-dg-cell[contains(.,'${project}')]/a AND Wait Until Element Is Visible And Enabled xpath=//project-detail//hbr-repository-gridview//clr-dg-cell[contains(.,'${project}/')]
Sleep 1
Capture Page Screenshot
Add User To Project Admin
[Arguments] ${project} ${user}
# *** this keyword has not been used ***
@ -50,7 +28,6 @@ Add User To Project Admin
Retry Text Input xpath=${project_member_add_username_xpath} ${user}
Retry Element Click xpath=${project_member_add_admin_xpath}
Retry Element Click xpath=${project_member_add_save_button_xpath}
Sleep 4
Search Project Member
[Arguments] ${project} ${user}
@ -112,7 +89,6 @@ Delete Project Member
Retry Double Keywords When Error Retry Element Click ${member_action_xpath} Retry Wait Until Page Contains Element ${delete_action_xpath}
Retry Double Keywords When Error Retry Element Click ${delete_action_xpath} Retry Wait Until Page Contains Element ${repo_delete_on_card_view_btn}
Retry Double Keywords When Error Retry Element Click ${repo_delete_on_card_view_btn} Retry Wait Element xpath=${project_member_xpath}
Sleep 1
User Should Be Owner Of Project
[Arguments] ${user} ${pwd} ${project}

3
tests/resources/Harbor-Pages/Project-P2P-Preheat.robot
View File

@ -150,8 +150,7 @@ Verify Artifact Is Labeled Event
Go Into Project ${project_name}
Switch To Project Label
Create New Labels ${label}
Switch To Project Repo
Go Into Repo ${image}
Go Into Repo ${project_name} ${image}
Add Labels To Tag ${tag} ${label}
Back Project Home ${project_name}
Switch To P2P Preheat

5
tests/resources/Harbor-Pages/Project-Repository.robot
View File

@ -32,8 +32,7 @@ View Scan Error Log
Scan Artifact
[Arguments] ${project} ${repo} ${label_xpath}=//clr-dg-row//label[1]
Go Into Project ${project}
Go Into Repo ${project}/${repo}
Go Into Repo ${project} ${repo}
Retry Element Click ${label_xpath}
Retry Element Click ${scan_artifact_btn}
@ -46,4 +45,4 @@ Check Scan Artifact Job Status Is Stopped
Should Be Equal As Strings '${job_status}' 'Scan stopped'
Refresh Repositories
Retry Element Click ${refresh_repositories_xpath}
Retry Element Click ${refresh_repositories_xpath}

3
tests/resources/Harbor-Pages/Project-Tag-Retention.robot
View File

@ -33,7 +33,6 @@ Retry Add A Tag Immutability Rule
Add A Tag Immutability Rule
[Arguments] ${scope} ${tag}
Reload Page
Sleep 3
Retry Double Keywords When Error Retry Element Click xpath=${project_tag_retention_add_rule_xpath} Retry Wait Until Page Contains Element xpath=${project_tag_immutability_save_add_button_xpath}
Retry Clear Element Text ${project_tag_immutability_scope_input_xpath}
Retry Text Input ${project_tag_immutability_scope_input_xpath} ${scope}
@ -97,9 +96,7 @@ Execute Result Should Be
FOR ${idx} IN RANGE 0 20
${out} Run Keyword And Ignore Error Retry Wait Until Page Contains Element //app-tag-retention-tasks//clr-datagrid//clr-dg-row[contains(., '${image}') and contains(., '${result}')]
Exit For Loop If '${out[0]}'=='PASS'
Sleep 1
Retry Element Click ${project_tag_retention_refresh_xpath}
Sleep 5
Retry Wait Until Page Contains Element xpath=${project_tag_retention_record_yes_xpath}
Retry Element Click ${project_tag_retention_list_expand_icon_xpath}
END

2
tests/resources/Harbor-Pages/Project-Webhooks.robot
View File

@ -6,9 +6,7 @@ Resource ../../resources/Util.robot
*** Keywords ***
Switch To Project Webhooks
#Switch To Project Tab Overflow
Retry Element Click xpath=//project-detail//a[contains(.,'Webhooks')]
Sleep 1
Create A New Webhook
[Arguments] ${webhook_name} ${webhook_endpoint_url} ${event_type}=@{EMPTY}

75
tests/resources/Harbor-Pages/Project.robot
View File

@ -26,7 +26,6 @@ Create An New Project And Go Into Project
${out} Run Keyword And Ignore Error Retry Button Click xpath=${create_project_button_xpath}
Log All Return value is ${out[0]}
Exit For Loop If '${out[0]}'=='PASS'
Sleep 1
END
Log To Console Project Name: ${projectname}
Retry Text Input xpath=${project_name_xpath} ${projectname}
@ -36,7 +35,6 @@ Create An New Project And Go Into Project
Run Keyword If '${storage_quota}'!='${null}' Input Storage Quota ${storage_quota} ${storage_quota_unit}
Run Keyword If '${proxy_cache}' == '${true}' Run Keywords Retry Element Click ${project_proxy_cache_switcher_xpath} AND Retry Element Click ${project_registry_select_id} AND Retry Element Click xpath=//select[@id='registry']//option[contains(.,'${registry}')]
Retry Double Keywords When Error Retry Element Click ${create_project_OK_button_xpath} Retry Wait Until Page Not Contains Element ${create_project_OK_button_xpath}
Sleep 2
Go Into Project ${projectname} has_image=${false}
Create An New Project With New User
@ -45,44 +43,35 @@ Create An New Project With New User
Logout Harbor
Sign In Harbor ${url} ${username} ${newPassword}
Create An New Project And Go Into Project ${projectname} ${public}
Sleep 1
Artifact Exist
[Arguments] ${tag_name}
Retry Wait Until Page Contains Element //artifact-list-tab//clr-datagrid//clr-dg-row[contains(.,'sha256') and contains(.,'${tag_name}')]
#It's the log of project.
Go To Project Log
#Switch To Project Tab Overflow
Retry Element Click xpath=${project_log_xpath}
Sleep 2
Switch To Member
Sleep 3
Retry Element Click xpath=${project_member_xpath}
Sleep 1
Switch To Replication
Retry Element Click xpath=${project_replication_xpath}
Sleep 1
Switch To Project Configuration
Retry Element Click ${project_config_tabsheet}
Sleep 1
Switch To Tag Retention
#Switch To Project Tab Overflow
Retry Element Click xpath=${project_tag_strategy_xpath}
Sleep 1
Switch To Tag Immutability
#Switch To Project Tab Overflow
Retry Double Keywords When Error Retry Element Click xpath=${project_tag_strategy_xpath} Retry Wait Until Page Contains Element ${project_tag_immutability_switch}
Retry Double Keywords When Error Retry Element Click xpath=${project_tag_immutability_switch} Retry Wait Until Page Contains Immutability rules
Sleep 1
Switch To Project Tab Overflow
Retry Element Click xpath=${project_tab_overflow_btn}
Sleep 1
Navigate To Projects
Retry Element Click xpath=${projects_xpath}
@ -98,7 +87,6 @@ Project Should Not Display
Search Private Projects
Retry Element Click xpath=//select
Retry Element Click xpath=//select/option[@value=1]
Sleep 1
Make Project Private
[Arguments] ${projectname}
@ -153,7 +141,6 @@ Delete Repo on CardView
Retry Element Click //hbr-gridview//span[contains(.,'${reponame}')]//clr-dropdown/button
Retry Element Click //hbr-gridview//span[contains(.,'${reponame}')]//clr-dropdown/clr-dropdown-menu/button[contains(.,'Delete')]
Retry Element Click ${repo_delete_on_card_view_btn}
Sleep 2
Delete Project
[Arguments] ${projectname}
@ -162,7 +149,6 @@ Delete Project
Retry Element Click ${project_action_xpath}
Retry Element Click xpath=//*[@id='delete-project']
Retry Element Click //clr-modal//button[contains(.,'DELETE')]
Sleep 1
Project Should Not Be Deleted
[Arguments] ${projname}
@ -202,9 +188,7 @@ Do Log Advanced Search
Retry Element Click xpath=//audit-log//clr-dropdown//a[contains(.,'Others')]
Retry Element Click xpath=//audit-log//hbr-filter//clr-icon
Retry Text Input xpath=//audit-log//hbr-filter//input harbor-jobservice
Sleep 1
${rc} = Get Element Count //audit-log//clr-dg-row
Should Be Equal As Integers ${rc} 1
Retry Wait Until Page Not Contains Element //audit-log//clr-dg-row[2]
Retry Click Repo Name
[Arguments] ${repo_name_element}
@ -220,25 +204,44 @@ Retry Click Repo Name
END
Should Be Equal As Strings '${out[0]}' 'PASS'
Go Into Repo
[Arguments] ${repoName}
Sleep 2
Retry Wait Until Page Not Contains Element ${repo_list_spinner}
${repo_name_element}= Set Variable xpath=//clr-dg-cell[contains(.,'${repoName}')]/a
FOR ${n} IN RANGE 1 3
Retry Element Click ${repo_search_icon}
Retry Clear Element Text ${repo_search_input}
Retry Text Input ${repo_search_input} ${repoName}
${out} Run Keyword And Ignore Error Retry Wait Until Page Contains Element ${repo_name_element}
Go Into Project
[Arguments] ${project} ${has_image}=${true}
FOR ${n} IN RANGE 1 4
${out} Run Keyword And Ignore Error Retry Go Into Project ${project} ${has_image}
Run Keyword If '${out[0]}'=='PASS' Exit For Loop
Reload Page
Sleep 2
Run Keyword If '${out[0]}'=='FAIL' Reload Page
Continue For Loop If '${out[0]}'=='FAIL'
Retry Click Repo Name ${repo_name_element}
Sleep 2
Exit For Loop
END
Run Keyword If '${out[0]}'=='FAIL' Capture Page Screenshot
Should Be Equal As Strings '${out[0]}' 'PASS'
Retry Go Into Project
[Arguments] ${project} ${has_image}
Retry Text Input ${search_input} ${project}
Wait Until Page Contains Element //list-project-ro//a[contains(., '${project}')]
Retry Link Click //list-project-ro//a[contains(., '${project}')]
Wait Until Page Contains Element //project-detail//h1[contains(., '${project}')]
Run Keyword If ${has_image}==${false} Wait Until Element Is Visible And Enabled //clr-dg-placeholder[contains(.,\"We couldn\'t find any repositories!\")]
... ELSE Wait Until Element Is Visible And Enabled //project-detail//hbr-repository-gridview//clr-dg-cell[contains(.,'${project}/')]
Go Into Repo
[Arguments] ${project_name} ${repo_name}
FOR ${n} IN RANGE 1 4
${out} Run Keyword And Ignore Error Retry Go Into Repo ${project_name} ${repo_name}
Run Keyword If '${out[0]}'=='PASS' Exit For Loop
Reload Page
Sleep 2
END
Run Keyword If '${out[0]}'=='FAIL' Capture Page Screenshot
Should Be Equal As Strings '${out[0]}' 'PASS'
Retry Go Into Repo
[Arguments] ${project_name} ${repo_name}
Retry Text Input ${search_input} ${project_name}/${repo_name}
Wait Until Page Contains Element //list-repository-ro//a[contains(., '${project_name}/${repo_name}')]
Retry Link Click //list-repository-ro//a[contains(., '${project_name}/${repo_name}')]
Wait Until Page Contains Element //artifact-list-page//h2[contains(., '${repo_name}')]
Click Index Achieve
[Arguments] ${tag_name}
Retry Element Click //artifact-list-tab//clr-datagrid//clr-dg-row[contains(.,'sha256') and contains(.,'${tag_name}')]//clr-dg-cell[1]//clr-tooltip//a
@ -261,12 +264,10 @@ Go Into Index And Contain Artifacts
Switch To CardView
Retry Element Click xpath=//hbr-repository-gridview//span[@class='card-btn']/clr-icon
Sleep 5
Expand Repo
[Arguments] ${projectname}
Retry Element Click //repository//clr-dg-row[contains(.,'${projectname}')]//button/clr-icon
Sleep 1
Edit Repo Info
Retry Element Click //*[@id='repo-info']
@ -285,11 +286,9 @@ Edit Repo Info
Switch To Project Label
Retry Element Click xpath=//project-detail//a[contains(.,'Labels')]
Sleep 1
Switch To Project Repo
Retry Element Click xpath=//project-detail//a[contains(.,'Repositories')]
Sleep 1
Add Labels To Tag
[Arguments] ${tagName} ${labelName}
@ -312,14 +311,12 @@ Filter Labels In Tags
Retry Element Click xpath=//*[@id='search-btn']
Retry Element Click xpath=//*[@id='filterArea']//div//button[contains(.,'${labelName2}')]
Retry Element Click xpath=//app-artifact-filter//clr-icon[contains(@shape,'search')]
Sleep 2
Retry Wait Until Page Contains Element xpath=//clr-dg-row[contains(.,'${labelName2}')]
Retry Wait Until Page Not Contains Element xpath=//clr-dg-row[contains(.,'${labelName1}')]
Get Statics
[Arguments] ${locator}
Reload Page
Sleep 5
${privaterepo}= Get Text ${locator}
[Return] ${privaterepo}

28
tests/resources/Harbor-Pages/Replication.robot
View File

@ -31,8 +31,6 @@ Filter Replication Rule
Filter Registry
[Arguments] ${registry_name}
${registry_name_element}= Set Variable xpath=//clr-dg-cell[contains(.,'${registry_name}')]
Switch To Replication Manage
Switch To Registries
Retry Element Click ${filter_registry_btn}
Retry Text Input ${filter_registry_input} ${registry_name}
Retry Wait Until Page Contains Element ${registry_name_element}
@ -138,7 +136,6 @@ Create A Rule With Existing Endpoint
Run Keyword If '${bandwidth_unit}' != 'Kbps' Select Bandwidth Unit ${bandwidth_unit}
#click save
Retry Double Keywords When Error Retry Element Click ${rule_save_button} Retry Wait Until Page Not Contains Element ${rule_save_button}
Sleep 2
Endpoint Is Unpingable
Retry Element Click ${ping_test_button}
@ -151,20 +148,16 @@ Endpoint Is Pingable
Disable Certificate Verification
Checkbox Should Be Selected ${destination_insecure_checkbox}
Retry Element Click ${destination_insecure_xpath}
Sleep 1
Enable Certificate Verification
Checkbox Should Not Be Selected ${destination_insecure_checkbox}
Retry Element Click ${destination_insecure_xpath}
Sleep 1
Switch To Registries
Retry Element Click ${nav_to_registries}
Sleep 1
Switch To Replication Manage
Retry Element Click ${nav_to_replications}
Sleep 1
Trigger Replication Manual
[Arguments] ${rule}
@ -176,9 +169,7 @@ Trigger Replication Manual
#change from click to mouse down and up
Mouse Down ${dialog_replicate}
Mouse Up ${dialog_replicate}
Sleep 2
Retry Wait Until Page Contains Element //*[@id='contentAll']//div[contains(.,'${rule}')]/../div/clr-icon[@shape='success-standard']
Sleep 1
Rename Rule
[Arguments] ${rule} ${newname}
@ -192,7 +183,7 @@ Rename Rule
Select Rule
[Arguments] ${rule}
Retry Double Keywords When Error Retry Element Click //clr-dg-row[contains(.,'${rule}')]/div/div[1]/div Retry Wait Element ${replication_rule_exec_id}
Retry Double Keywords When Error Retry Element Click //clr-dg-row[contains(.,'${rule}')]/div/div[1]/div Element Should Be Enabled ${replication_rule_exec_id}
Stop Jobs
Retry Element Click ${stop_jobs_button}
@ -249,10 +240,8 @@ Rename Endpoint
Delete Endpoint
[Arguments] ${name}
Retry Element Click ${endpoint_filter_search}
Retry Text Input ${endpoint_filter_input} ${name}
#click checkbox before target endpoint
Retry Double Keywords When Error Retry Element Click //clr-dg-row[contains(.,'${name}')]//div[contains(@class,'clr-checkbox-wrapper')] Retry Wait Element ${registry_del_btn}
Filter Registry ${name}
Retry Double Keywords When Error Retry Element Click //clr-dg-row[contains(.,'${name}')]//div[contains(@class,'clr-checkbox-wrapper')] Checkbox Should Be Selected //clr-dg-row[contains(.,'${name}')]//input
Retry Element Click ${registry_del_btn}
Wait Until Page Contains Element ${dialog_delete}
Retry Element Click ${dialog_delete}
@ -264,21 +253,20 @@ Select Rule And Replicate
Retry Double Keywords When Error Retry Element Click xpath=${dialog_replicate} Retry Wait Until Page Not Contains Element xpath=${dialog_replicate}
Image Should Be Replicated To Project
[Arguments] ${project} ${image} ${period}=60 ${times}=20 ${tag}=${EMPTY} ${expected_image_size_in_regexp}=${null} ${total_artifact_count}=${null} ${archive_count}=${null}
[Arguments] ${project} ${image} ${times}=6 ${tag}=${EMPTY} ${expected_image_size_in_regexp}=${null} ${total_artifact_count}=${null} ${archive_count}=${null}
FOR ${n} IN RANGE 0 ${times}
Sleep ${period}
Go Into Project ${project}
Go Into Project ${project}
Switch To Project Repo
${out} Run Keyword And Ignore Error Retry Wait Until Page Contains ${project}/${image}
Log To Console Return value is ${out[0]}
Continue For Loop If '${out[0]}'=='FAIL'
Go Into Repo ${project}/${image}
${size}= Run Keyword If '${tag}'!='${EMPTY}' and '${expected_image_size_in_regexp}'!='${null}' Get Text //clr-dg-row[contains(., '${tag}')]//clr-dg-cell[5]/div
Go Into Repo ${project} ${image}
${size}= Run Keyword If '${tag}'!='${EMPTY}' and '${expected_image_size_in_regexp}'!='${null}' Get Text //clr-dg-row[contains(., '${tag}')]//clr-dg-cell[4]/div
Run Keyword If '${tag}'!='${EMPTY}' and '${expected_image_size_in_regexp}'!='${null}' Should Match Regexp '${size}' '${expected_image_size_in_regexp}'
${index_out} Go Into Index And Contain Artifacts ${tag} total_artifact_count=${total_artifact_count} archive_count=${archive_count} return_immediately=${true}
Log All index_out: ${index_out}
Run Keyword If '${index_out}'=='PASS' Exit For Loop
Sleep 30
Sleep 10
END
Verify Artifacts Counts In Archive

1
tests/resources/Harbor-Pages/ToolKit.robot
View File

@ -24,7 +24,6 @@ Delete Success
FOR ${obj} IN @{obj}
Retry Wait Until Page Contains Element //*[@id='contentAll']//div[contains(.,'${obj}')]/../div/clr-icon[@shape='success-standard']
END
Sleep 1
Delete Fail
[Arguments] @{obj}

7
tests/resources/Harbor-Pages/Verify.robot
View File

@ -463,8 +463,7 @@ Verify Artifact Index
FOR ${project} IN @{project}
${name}= Get Value From Json ${json} $.projects[?(@.name=${project})].artifact_index.name
${tag}= Get Value From Json ${json} $.projects[?(@.name=${project})].artifact_index.tag
Go Into Project ${project} has_image=${true}
Go Into Repo ${project}/${name}[0]
Go Into Repo ${project} ${name}[0]
Go Into Index And Contain Artifacts ${tag}[0] total_artifact_count=2
Pull image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project} ${name}[0]:${tag}[0]
Navigate To Projects
@ -474,9 +473,7 @@ Verify Artifact Index
Loop Repo
[Arguments] ${project} @{repos}
FOR ${repo} IN @{repos}
Navigate To Projects
Go Into Project ${project} has_image=${true}
Go Into Repo ${project}/${repo}[0][cache_image_namespace]/${repo}[0][cache_image]
Go Into Repo ${project} ${repo}[0][cache_image_namespace]/${repo}[0][cache_image]
Pull image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project} ${repo}[0][cache_image_namespace]/${repo}[0][cache_image]:${repo}[0][tag]
END

8
tests/resources/Harbor-Pages/Vulnerability.robot
View File

@ -28,7 +28,6 @@ Switch To Vulnerability Page
Retry Wait Element ${scan_now_button}
Set Vulnerabilty Serverity
#0 is critical, 1 is high, 2 is medium, 3 is low, 4 is negligible.
[Arguments] ${level}
Goto Project Config
#enable first
@ -36,7 +35,6 @@ Set Vulnerabilty Serverity
Checkbox Should Be Selected //project-detail//clr-checkbox-wrapper//input[@name='prevent-vulenrability-image-input']
Retry Element Click //project-detail//select
#wait for dropdown popup
Sleep 1
Select From List By Index //project-detail//select ${level}
Retry Element Click ${project_config_save_btn}
@ -44,9 +42,7 @@ Scan Is Disabled
Retry Wait Until Page Contains Element //button[@id='scan-btn' and @disabled='']
Scan Repo
#use fail for image can not scan, otherwise use success
[Arguments] ${tagname} ${status}
#select one tag
Retry Element Click //clr-dg-row[contains(.,'${tagname}')]//label
Retry Element Click //button[@id='scan-btn']
Run Keyword If '${status}' == 'Succeed' Wait Until Element Is Visible //hbr-vulnerability-bar//hbr-result-tip-histogram 300
@ -62,7 +58,6 @@ Enable Scan On Push
Retry Element Click //clr-checkbox-wrapper[@id='scan-image-on-push-wrapper']//label
Checkbox Should Be Selected //clr-checkbox-wrapper[@id='scan-image-on-push-wrapper']//input
Retry Element Click ${project_config_save_btn}
Sleep 10
Vulnerability Not Ready Project Hint
Sleep 2
@ -89,8 +84,7 @@ Set Default Scanner
Check Listed In CVE Allowlist
[Arguments] ${project_name} ${image} ${tag} ${cve_id} ${is_in}=Yes
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image}
Go Into Repo ${project_name} ${image}
Go Into Artifact ${tag}
Scroll Element Into View //clr-dg-row[contains(.,'${cve_id}')]

40
tests/resources/TestCaseBody.robot
View File

@ -62,8 +62,7 @@ Body Of Scan A Tag In The Repo
Sign In Harbor ${HARBOR_URL} user023 Test1@34
Create An New Project And Go Into Project project${d}
Push Image ${ip} user023 Test1@34 project${d} ${image_argument}:${tag_argument}
Go Into Project project${d}
Go Into Repo project${d}/${image_argument}
Go Into Repo project${d} ${image_argument}
Scan Repo ${tag_argument} Succeed
Scan Result Should Display In List Row ${tag_argument} is_no_vulerabilty=${is_no_vulerabilty}
Pull Image ${ip} user023 Test1@34 project${d} ${image_argument} ${tag_argument}
@ -76,8 +75,7 @@ Body Of Scan Image With Empty Vul
${tag}= Set Variable ${tag_argument}
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} library ${image_argument}:${tag_argument}
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Go Into Project library
Go Into Repo ${image_argument}
Go Into Repo library ${image_argument}
Scan Repo ${tag} Succeed
Scan Result Should Display In List Row ${tag} is_no_vulerabilty=${true}
Close Browser
@ -90,9 +88,7 @@ Body Of Manual Scan All
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Vulnerability Page
Trigger Scan Now And Wait Until The Result Appears
Navigate To Projects
Go Into Project library
Go Into Repo redis
Go Into Repo library redis
Scan Result Should Display In List Row ${sha256}
View Repo Scan Details @{vulnerability_levels}
Close Browser
@ -105,8 +101,7 @@ Body Of View Scan Results
Sign In Harbor ${HARBOR_URL} user025 Test1@34
Create An New Project And Go Into Project project${d}
Push Image ${ip} user025 Test1@34 project${d} tomcat
Go Into Project project${d}
Go Into Repo project${d}/tomcat
Go Into Repo project${d} tomcat
Scan Repo latest Succeed
Scan Result Should Display In List Row latest
View Repo Scan Details @{vulnerability_levels}
@ -121,9 +116,7 @@ Body Of Scan Image On Push
Goto Project Config
Enable Scan On Push
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} memcached
Navigate To Projects
Go Into Project project${d}
Go Into Repo memcached
Go Into Repo project${d} memcached
Scan Result Should Display In List Row latest
View Repo Scan Details @{vulnerability_levels}
Close Browser
@ -193,8 +186,7 @@ Body Of Verfiy System Level CVE Allowlist
Go Into Project project${d}
Set Vulnerabilty Serverity 2
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256} err_msg=cannot be pulled due to configured policy
Go Into Project project${d}
Go Into Repo project${d}/${image}
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Logout Harbor
@ -229,8 +221,7 @@ Body Of Verfiy Project Level CVE Allowlist
Go Into Project project${d}
Set Vulnerabilty Serverity 2
Cannot Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Project project${d}
Go Into Repo project${d}/${image}
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Go Into Project project${d}
Add Items to Project CVE Allowlist ${most_cve_list}
@ -259,8 +250,7 @@ Body Of Verfiy Project Level CVE Allowlist By Quick Way of Add System
Push Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} sha256=${sha256}
Go Into Project project${d}
Set Vulnerabilty Serverity 2
Go Into Project project${d}
Go Into Repo project${d}/${image}
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Pull Image ${ip} ${signin_user} ${signin_pwd} project${d} ${image} tag=${sha256}
Go Into Project project${d}
@ -313,6 +303,7 @@ Body Of Replication Of Pull Images from Registry To Self
Switch To Replication Manage
Create A Rule With Existing Endpoint rule${d} pull ${src_project_name} all e${d} ${_des_pro_name} flattening=${flattening}
Select Rule And Replicate rule${d}
Check Latest Replication Job Status Succeeded
Run Keyword If '${verify_verbose}'=='Y' Verify Artifact Display Verbose ${_des_pro_name} @{target_images}
... ELSE Verify Artifact Display ${_des_pro_name} @{target_images}
Close Browser
@ -329,7 +320,7 @@ Verify Artifact Display Verbose
${total_artifact_count}= Get From Dictionary ${item} total_artifact_count
${archive_count}= Get From Dictionary ${item} archive_count
Log To Console Check image ${image}:${tag} replication to Project ${pro_name}
Image Should Be Replicated To Project ${pro_name} ${image} tag=${tag} total_artifact_count=${total_artifact_count} archive_count=${archive_count} times=6
Image Should Be Replicated To Project ${pro_name} ${image} tag=${tag} total_artifact_count=${total_artifact_count} archive_count=${archive_count}
END
Verify Artifact Display
@ -340,7 +331,7 @@ Verify Artifact Display
${item}= Get Substring ${item} 1 -1
${item}= Evaluate ${item}
${image}= Get From Dictionary ${item} image
Image Should Be Replicated To Project ${pro_name} ${image} times=6
Image Should Be Replicated To Project ${pro_name} ${image}
END
Replication With Flattening
@ -375,7 +366,6 @@ Replication With Flattening
Check Harbor Api Page
Retry Link Click //a[contains(.,'Harbor API V2.0')]
Sleep 3
Switch Window locator=NEW
Title Should Be Harbor Swagger
Retry Wait Element xpath=//h2[contains(.,"Harbor API")]
@ -434,8 +424,7 @@ Verify Webhook By Artifact Deleted Event
Switch Window ${webhook_handle}
Delete All Requests
Switch Window ${harbor_handle}
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image}
Go Into Repo ${project_name} ${image}
@{tag_list} Create List ${tag}
Multi-delete Artifact @{tag_list}
Switch Window ${webhook_handle}
@ -447,8 +436,7 @@ Verify Webhook By Scanning Finished Event
Switch Window ${webhook_handle}
Delete All Requests
Switch Window ${harbor_handle}
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image}
Go Into Repo ${project_name} ${image}
Scan Repo ${tag} Succeed
Switch Window ${webhook_handle}
&{scanning_finished_property}= Create Dictionary type=SCANNING_COMPLETED scan_status=Success namespace=${project_name} tag=${tag} name=${image}
@ -487,7 +475,7 @@ Verify Webhook By Replication Status Changed Event
Switch Window ${harbor_handle}
Switch To Replication Manage
Select Rule And Replicate ${replication_rule_name}
Retry Wait Until Page Contains Succeeded
Check Latest Replication Job Status Succeeded
Switch Window ${webhook_handle}
&{replication_finished_property}= Create Dictionary type=REPLICATION operator=MANUAL registry_type=harbor harbor_hostname=${ip}
Verify Request &{replication_finished_property}

3
tests/resources/Util.robot
View File

@ -237,7 +237,6 @@ Wait Unitl Command Success
Log Trying ${cmd}: ${n} ... console=True
${rc} ${output}= Run And Return Rc And Output ${cmd}
Exit For Loop If '${rc}'=='0'
Sleep 2
END
Log Command Result is ${output}
Should Be Equal As Strings '${rc}' '0'
@ -256,7 +255,7 @@ Retry Keyword N Times When Error
${out} Run Keyword And Ignore Error ${keyword} @{elements}
Run Keyword If '${keyword}'=='Make Swagger Client' Exit For Loop If '${out[0]}'=='PASS' and '${out[1]}'=='0'
... ELSE Exit For Loop If '${out[0]}'=='PASS'
Sleep 10
Sleep 5
END
Run Keyword If '${out[0]}'=='FAIL' Capture Page Screenshot
Should Be Equal As Strings '${out[0]}' 'PASS'

2
tests/robot-cases/Group0-Util/bundle.json
View File

@ -1 +1 @@
{"actions":{"io.cnab.status":{}},"definitions":{"port":{"default":"8080","type":"string"},"text":{"default":"Hello, World!","type":"string"}},"description":"Hello, World!","images":{"hello":{"description":"hello","image":"registry/namespace/image1","imageType":"docker","mediaType":"application/vnd.docker.distribution.manifest.v2+json","size":528}},"invocationImages":[{"image":"registry/namespace/image2","imageType":"docker","mediaType":"application/vnd.docker.distribution.manifest.v2+json","size":941}],"maintainers":[{"email":"user@email.com","name":"user"}],"name":"hello-world","parameters":{"fields":{"definition":"","destination":null}},"schemaVersion":"v1.0.0","version":"0.1.0"}
{"actions":{"io.cnab.status":{}},"definitions":{"port":{"default":"8080","type":"string"},"text":{"default":"Hello, World!","type":"string"}},"description":"Hello, World!","images":{"hello":{"description":"hello","image":"registry/namespace/image1","imageType":"docker","mediaType":"application/vnd.docker.distribution.manifest.v2+json","size":528}},"invocationImages":[{"image":"registry/namespace/image2","imageType":"docker","mediaType":"application/vnd.docker.distribution.manifest.v2+json","size":941}],"maintainers":[{"email":"user@email.com","name":"user"}],"name":"hello-world","parameters":{"fields":{"definition":"","destination":null}},"schemaVersion":"v1.0.0","version":"0.1.0"}

151
tests/robot-cases/Group1-Nightly/Common.robot
View File

@ -36,43 +36,15 @@ Test Case - Push ORAS and Display
[Tags] push_oras
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
Sign In Harbor ${HARBOR_URL} user010 Test1@34
Create An New Project And Go Into Project test${d}
${repo_name}= Set Variable hello-oras-artifact
${tag}= Set Variable 1.0.0
Retry Keyword N Times When Error 5 Oras Push ${ip} user010 Test1@34 test${d} ${repo_name} ${tag}
Go Into Project test${d}
Wait Until Page Contains test${d}/${repo_name}
Go Into Repo test${d}/${repo_name}
Go Into Repo test${d} ${repo_name}
Wait Until Page Contains ${tag}
Close Browser
## TODO: uncomment it once #14470 fixed
# Test Case - Push SIF and Display
# [Tags] push_sif
# Init Chrome Driver
# ${d}= Get Current Date result_format=%m%s
# ${user}= Set Variable user010
# ${pwd}= Set Variable Test1@34
# Sign In Harbor ${HARBOR_URL} ${user} ${pwd}
# Create An New Project And Go Into Project test${d}
# ${repo_name}= Set Variable busybox
# ${tag}= Set Variable 1.28
# Retry Keyword N Times When Error 5 Push Singularity To Harbor library: library/default/ ${ip} ${user} ${pwd} test${d} ${repo_name} ${tag}
# Go Into Project test${d}
# Wait Until Page Contains test${d}/${repo_name}
# Go Into Repo test${d}/${repo_name}
# Wait Until Page Contains ${tag}
# Close Browser
Test Case - Push CNAB Bundle and Display
[Tags] push_cnab
Init Chrome Driver
@ -91,12 +63,11 @@ Test Case - Push CNAB Bundle and Display
Go Into Project test${d}
Wait Until Page Contains test${d}/cnab${d}
Go Into Repo test${d}/cnab${d}
Go Into Repo test${d} cnab${d}
Wait Until Page Contains cnab_tag${d}
Go Into Project test${d}
Wait Until Page Contains test${d}/cnab${d}
Go Into Repo test${d}/cnab${d}
Go Into Repo test${d} cnab${d}
Go Into Index And Contain Artifacts cnab_tag${d} total_artifact_count=3 archive_count=2
Close Browser
@ -118,8 +89,7 @@ Test Case - Repo Size
${d}= Get Current Date result_format=%m%s
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} library alpine 2.6 2.6
Go Into Project library
Go Into Repo alpine
Go Into Repo library alpine
Wait Until Page Contains 1.92MiB
Close Browser
@ -229,7 +199,6 @@ Test Case - Update Label
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To System Labels
Create New Labels label_${d}
Sleep 3
${d1}= Get Current Date
Update A Label label_${d}
Close Browser
@ -240,7 +209,6 @@ Test Case - Delete Label
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To System Labels
Create New Labels label_${d}
Sleep 3
Delete A Label label_${d}
Close Browser
@ -286,8 +254,6 @@ Test Case - User View Logs
Delete Repo project${d} ${replication_image}
Delete Repo project${d} ${img}
Sleep 3
Go To Project Log
Advanced Search Should Display
@ -326,7 +292,6 @@ Test Case - Edit Project Creation
Project Creation Should Display
Logout Harbor
Sleep 3
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Set Pro Create Admin Only
Logout Harbor
@ -346,8 +311,7 @@ Test Case - Edit Repo Info
Sign In Harbor ${HARBOR_URL} user011 Test1@34
Create An New Project And Go Into Project project${d}
Push Image ${ip} user011 Test1@34 project${d} hello-world
Go Into Project project${d}
Go Into Repo project${d}/hello-world
Go Into Repo project${d} hello-world
Edit Repo Info
Close Browser
@ -382,7 +346,6 @@ Test Case - Delete Multi Repo
Create An New Project And Go Into Project project${d}
Push Image ${ip} user013 Test1@34 project${d} hello-world
Push Image ${ip} user013 Test1@34 project${d} busybox
Sleep 2
Go Into Project project${d}
@{repo_list} Create List hello-world busybox
Multi-delete Object ${repo_delete_btn} @{repo_list}
@ -397,8 +360,7 @@ Test Case - Delete Multi Artifacts
Create An New Project And Go Into Project project${d}
Push Image With Tag ${ip} user014 Test1@34 project${d} redis 3.2.10-alpine 3.2.10-alpine
Push Image With Tag ${ip} user014 Test1@34 project${d} redis 4.0.7-alpine 4.0.7-alpine
Go Into Project project${d}
Go Into Repo redis
Go Into Repo project${d} redis
@{tag_list} Create List 3.2.10-alpine 4.0.7-alpine
Multi-delete Artifact @{tag_list}
# Verify
@ -436,13 +398,10 @@ Test Case - Project Admin Operate Labels
${d}= Get Current Date result_format=%m%s
Sign In Harbor ${HARBOR_URL} user019 Test1@34
Create An New Project And Go Into Project project${d}
Sleep 2
# Add labels
Switch To Project Label
Create New Labels label_${d}
Sleep 2
Update A Label label_${d}
Sleep 2
Delete A Label label_${d}
Close Browser
@ -454,14 +413,11 @@ Test Case - Project Admin Add Labels To Repo
Push Image With Tag ${ip} user020 Test1@34 project${d} redis 3.2.10-alpine 3.2.10-alpine
Push Image With Tag ${ip} user020 Test1@34 project${d} redis 4.0.7-alpine 4.0.7-alpine
Go Into Project project${d}
Sleep 2
# Add labels
Switch To Project Label
Create New Labels label111
Create New Labels label22
Sleep 2
Switch To Project Repo
Go Into Repo project${d}/redis
Go Into Repo project${d} redis
Add Labels To Tag 3.2.10-alpine label111
Add Labels To Tag 4.0.7-alpine label22
Filter Labels In Tags label111 label22
@ -480,7 +436,6 @@ Test Case - Developer Operate Labels
Sign In Harbor ${HARBOR_URL} user022 Test1@34
Go Into Project project${d} has_image=${false}
Sleep 3
Retry Wait Until Page Not Contains Element xpath=//a[contains(.,'Labels')]
Close Browser
@ -493,17 +448,13 @@ Test Case - Copy A Image
Create An New Project And Go Into Project project${random_num1}${random_num2}
Create An New Project And Go Into Project project${random_num1}
Sleep 1
Push Image With Tag ${ip} user028 Test1@34 project${random_num1} redis ${image_tag}
Sleep 1
Go Into Repo project${random_num1}/redis
Go Into Repo project${random_num1} redis
Copy Image ${image_tag} project${random_num1}${random_num2} ${target_image_name}
Navigate To Projects
Go Into Project project${random_num1}${random_num2}
Sleep 1
Page Should Contain ${target_image_name}
Go Into Repo project${random_num1}${random_num2}/${target_image_name}
Sleep 1
Go Into Repo project${random_num1}${random_num2} ${target_image_name}
Retry Wait Until Page Contains Element xpath=${tag_value_xpath}
Close Browser
@ -527,12 +478,14 @@ Test Case - Copy A Image And Accessory
Docker Login ${ip} ${user} ${pwd}
Cosign Sign ${ip}/${source_project}/${image}:${tag}
Docker Logout ${ip}
Retry Double Keywords When Error Go Into Repo ${source_project}/${image} Should Be Signed By Cosign ${tag}
Go Into Repo ${source_project} ${image}
Should Be Signed By Cosign ${tag}
Copy Image ${tag} ${target_project} ${image}
Retry Double Keywords When Error Go Into Project ${target_project} Retry Wait Until Page Contains ${image}
Retry Double Keywords When Error Go Into Repo ${target_project}/${image} Retry Wait Until Page Contains Element //clr-dg-row[contains(.,${tag})]
Go Into Repo ${target_project} ${image}
Retry Wait Until Page Contains Element //clr-dg-row[contains(.,${tag})]
Should Be Signed By Cosign ${tag}
Close Browser
@ -590,15 +543,11 @@ Test Case - Project Quotas Control Under Copy
Create An New Project And Go Into Project project_b_${d} storage_quota=${storage_quota} storage_quota_unit=${storage_quota_unit}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project_a_${d} ${image_a} tag=${image_a_ver} tag1=${image_a_ver}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project_a_${d} ${image_b} tag=${image_b_ver} tag1=${image_b_ver}
Go Into Project project_a_${d}
Go Into Repo project_a_${d}/${image_a}
Go Into Repo project_a_${d} ${image_a}
Copy Image ${image_a_ver} project_b_${d} ${image_a}
Go Into Project project_a_${d}
Go Into Repo project_a_${d}/${image_b}
Go Into Repo project_a_${d} ${image_b}
Copy Image ${image_b_ver} project_b_${d} ${image_b} is_success=${false}
Sleep 2
Go Into Project project_b_${d}
Sleep 2
Retry Wait Until Page Contains Element xpath=//clr-dg-cell[contains(.,'${image_a}')]/a
Retry Wait Until Page Not Contains Element xpath=//clr-dg-cell[contains(.,'${image_b}')]/a
Close Browser
@ -609,8 +558,7 @@ Test Case - Tag CRUD
${d}= Get Current Date result_format=%m%s
Create An New Project And Go Into Project project${d}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} hello-world latest
Switch To Project Repo
Go Into Repo hello-world
Go Into Repo project${d} hello-world
Go Into Artifact latest
Should Contain Tag latest
# add more than one tag
@ -696,15 +644,12 @@ Test Case - Push Docker Manifest Index and Display
Docker Push Index ${ip} user010 Test1@34 ${ip}/test${d}/index${d}:index_tag${d} ${ip}/test${d}/${image_a}:${image_a_ver} ${ip}/test${d}/${image_b}:${image_b_ver}
Go Into Project test${d}
Wait Until Page Contains test${d}/index${d}
Go Into Repo test${d}/index${d}
Go Into Repo test${d} index${d}
Wait Until Page Contains index_tag${d}
Go Into Project test${d}
Wait Until Page Contains test${d}/index${d}
Go Into Repo test${d}/index${d}
Go Into Repo test${d} index${d}
Go Into Index And Contain Artifacts index_tag${d} total_artifact_count=2
Close Browser
@ -717,19 +662,15 @@ Test Case - Can Not Copy Image In ReadOnly Mode
Create An New Project And Go Into Project project${random_num1}${random_num2}
Create An New Project And Go Into Project project${random_num1}
Sleep 1
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${random_num1} redis ${image_tag}
Sleep 1
Enable Read Only
Go Into Repo project${random_num1}/redis
Go Into Repo project${random_num1} redis
Copy Image ${image_tag} project${random_num1}${random_num2} ${target_image_name} is_success=${false}
Retry Wait Element Not Visible ${repo_retag_confirm_dlg}
Navigate To Projects
Go Into Project project${random_num1}${random_num2} has_image=${false}
Sleep 10
Go Into Project project${random_num1}${random_num2} has_image=${false}
Disable Read Only
Sleep 10
Close Browser
Test Case - Read Only Mode
@ -742,7 +683,6 @@ Test Case - Read Only Mode
Cannot Push image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} busybox:latest
Disable Read Only
Sleep 5
Push image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} busybox:latest
Close Browser
@ -831,7 +771,8 @@ Test Case - Cosign And Cosign Deployment Security Policy
Push Image With Tag ${ip} ${user} ${pwd} project${d} ${image} ${tag}
Go Into Project project${d}
Retry Double Keywords When Error Go Into Repo project${d}/${image} Should Not Be Signed By Cosign ${tag}
Go Into Repo project${d} ${image}
Should Not Be Signed By Cosign ${tag}
Cannot Pull Image ${ip} ${user} ${pwd} project${d} ${image}:${tag} err_msg=The image is not signed in Cosign.
Cosign Generate Key Pair
Cosign Verify ${ip}/project${d}/${image}:${tag} ${false}
@ -863,8 +804,7 @@ Test Case - Audit Log And Purge
Push Image With Tag ${ip} ${user} ${pwd} project${d} ${image} ${tag1} ${tag1}
Clean All Local Images
Verify Log ${user} project${d}/${image}:${tag1} artifact create
Go Into Project project${d}
Go Into Repo ${image}
Go Into Repo project${d} ${image}
Go Into Artifact ${tag1}
# create tag
Add A New Tag ${tag2}
@ -878,8 +818,7 @@ Test Case - Audit Log And Purge
Docker Pull ${ip}/project${d}/${image}:${tag1}
Docker Logout ${ip}
Verify Log ${user} project${d}/${image}:${sha256} artifact pull
Go Into Project project${d}
Go Into Repo project${d}/${image}
Go Into Repo project${d} ${image}
# delete artifact
@{tag_list} Create List ${tag1}
Multi-delete Artifact @{tag_list}
@ -928,8 +867,7 @@ Test Case - Audit Log Forward
Retry Action Keyword Verify Log In Syslog Service ${HARBOR_ADMIN} project${d}/${image}:${tag1} artifact create
# Enable Skip Audit Log Database
Enable Skip Audit Log Database
Go Into Project project${d}
Go Into Repo ${image}
Go Into Repo project${d} ${image}
Go Into Artifact ${tag1}
# create tag
Add A New Tag ${tag2}
@ -939,8 +877,7 @@ Test Case - Audit Log Forward
Set Audit Log Forward ${null} Configuration has been successfully saved.
Retry Wait Element Should Be Disabled ${skip_audit_log_database_checkbox}
Checkbox Should Not Be Selected ${skip_audit_log_database_checkbox}
Go Into Project project${d}
Go Into Repo ${image}
Go Into Repo project${d} ${image}
Go Into Artifact ${tag1}
# delete tag
Delete A Tag ${tag2}
@ -979,17 +916,15 @@ Test Case - Export CVE
# scan images
Refresh Repositories
FOR ${image} IN @{images.keys()}
Go Into Repo ${image}
Go Into Repo project${d} ${image}
Scan Repo ${images['${image}']} Succeed
Back Project Home project${d}
END
Back Project Home project${d}
Switch To Project Label
Create New Labels ${labels}[1]
Switch To Project Repo
Go Into Repo nginx
Go Into Repo project${d} nginx
Add Labels To Tag ${images['nginx']} ${labels}[0]
Back Project Home project${d}
Go Into Repo redis
Go Into Repo project${d} redis
Add Labels To Tag ${images['redis']} ${labels}[1]
Navigate To Projects
Should Not Be Export CVEs
@ -1032,8 +967,7 @@ Test Case - Job Service Dashboard Job Queues
${retention_execution1}= Execute Dry Run photon 0/0
${retention_execution2}= Execute Run photon 0/0
# Triggers three IMAGE_SCAN jobs
Switch To Project Repo
Go Into Repo photon
Go Into Repo project${d} photon
Retry Element Click //clr-datagrid//label[contains(.,'Select All')]
Retry Button Click ${scan_artifact_btn}
# Triggers a GARBAGE_COLLECTION job
@ -1069,8 +1003,7 @@ Test Case - Job Service Dashboard Job Queues
Switch to Log Rotation
Purge Now 1 Days Running
# Triggers three IMAGE_SCAN jobs
Go Into Project project${d}
Go Into Repo photon
Go Into Repo project${d} photon
Retry Element Click //clr-datagrid//label[contains(.,'Select All')]
Retry Button Click ${scan_artifact_btn}
# Check job queues
@ -1117,15 +1050,15 @@ Test Case - Job Service Dashboard Workers
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Project And Go Into Project ${project_name}
Switch to Registries
Create A New Endpoint harbor ${endpoint_name} https://${LOCAL_REGISTRY} ${null} ${null}
Create A New Endpoint harbor ${endpoint_name} https://cicd.harbor.vmwarecna.net ${null} ${null}
Switch To Replication Manage
Create A Rule With Existing Endpoint ${rule_name} pull ${LOCAL_REGISTRY_NAMESPACE}/test_replication image ${endpoint_name} ${project_name} bandwidth=50 bandwidth_unit=Mbps
Create A Rule With Existing Endpoint ${rule_name} pull nightly/test_replication image ${endpoint_name} ${project_name} bandwidth=50 bandwidth_unit=Mbps
Select Rule And Replicate ${rule_name}
Retry Wait Until Page Contains Running
Check Latest Replication Job Status InProgress
Switch To Job Workers
Retry Wait Until Page Contains Element //clr-datagrid[.//button[text()='Worker ID']]//clr-dg-row//clr-dg-cell[text()='REPLICATION']
Retry Wait Until Page Contains Element //app-donut-chart//div[text()=' 1/10 ']
Check Worker Log REPLICATION copying ${LOCAL_REGISTRY_NAMESPACE}/test_replication
Check Worker Log REPLICATION copying nightly/test_replication
Switch To Replication Manage
Select Rule ${rule_name}
Retry Action Keyword Check Latest Replication Job Status Succeeded
@ -1146,22 +1079,20 @@ Test Case - Retain Image Last Pull Time
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image} ${tag} ${tag}
Switch To Configuration System Setting
Set Up Retain Image Last Pull Time enable
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image}
Go Into Repo ${project_name} ${image}
Scan Repo ${tag} Succeed
Sleep 15
Reload Page
Retry Wait Element Visible //clr-dg-row//clr-dg-cell[10]
${last_pull_time}= Get Text //clr-dg-row//clr-dg-cell[10]
Retry Wait Element Visible //clr-dg-row//clr-dg-cell[9]
${last_pull_time}= Get Text //clr-dg-row//clr-dg-cell[9]
Should Be Empty ${last_pull_time}
Switch To Configuration System Setting
Set Up Retain Image Last Pull Time disable
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image}
Go Into Repo ${project_name} ${image}
Scan Repo ${tag} Succeed
Sleep 15
Reload Page
Retry Wait Element Visible //clr-dg-row//clr-dg-cell[10]
${last_pull_time}= Get Text //clr-dg-row//clr-dg-cell[10]
Retry Wait Element Visible //clr-dg-row//clr-dg-cell[9]
${last_pull_time}= Get Text //clr-dg-row//clr-dg-cell[9]
Should Not Be Empty ${last_pull_time}
Close Browser

13
tests/robot-cases/Group1-Nightly/Common_GC.robot
View File

@ -45,8 +45,7 @@ Test Case - GC Untagged Images
Create An New Project And Go Into Project project${d}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} hello-world latest
# make hello-world untagged
Go Into Project project${d}
Go Into Repo hello-world
Go Into Repo project${d} hello-world
Go Into Artifact latest
Should Contain Tag latest
Delete A Tag latest
@ -55,18 +54,14 @@ Test Case - GC Untagged Images
GC Now
${latest_job_id}= Get Text ${latest_job_id_xpath}
Retry GC Should Be Successful ${latest_job_id} ${null}
Go Into Project project${d}
Switch To Project Repo
Go Into Repo hello-world
Go Into Repo project${d} hello-world
Should Contain Artifact
# run gc with param delete untagged artifacts checked, should delete hello-world
Switch To Garbage Collection
GC Now untag=${true}
${latest_job_id}= Get Text ${latest_job_id_xpath}
Retry GC Should Be Successful ${latest_job_id} ${null}
Go Into Project project${d}
Switch To Project Repo
Go Into Repo hello-world
Go Into Repo project${d} hello-world
Should Not Contain Any Artifact
Close Browser
@ -93,4 +88,4 @@ Test Case - Project Quotas Control Under GC
Sleep 1
END
Should Be Equal As Strings '${out2[0]}' 'PASS'
Close Browser
Close Browser

155
tests/robot-cases/Group1-Nightly/Replication.robot
View File

@ -33,7 +33,6 @@ ${REMOTE_SERVER_API_ENDPOINT} ${REMOTE_SERVER_URL}/api
*** Test Cases ***
Test Case - Get Harbor Version
#Just get harbor version and log it
Get Harbor Version
Test Case - Pro Replication Rules Add
@ -45,7 +44,6 @@ Test Case - Pro Replication Rules Add
Close Browser
Test Case - Harbor Endpoint Verification
#This case need vailid info and selfsign cert
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
@ -57,12 +55,11 @@ Test Case - Harbor Endpoint Verification
Close Browser
Test Case - Harbor Endpoint Add
#This case need vailid info and selfsign cert
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
${d}= Get Current Date result_format=%m%s
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Registries
Create A New Endpoint harbor testabc https://${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} Y
Create A New Endpoint harbor testabc https://${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} Y
Close Browser
Test Case - Harbor Endpoint Edit
@ -171,7 +168,7 @@ Test Case - Replication Of Push Images from Self To Harbor
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Select Rule And Replicate rule${d}
Sleep 20
Check Latest Replication Job Status Succeeded
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Image Should Be Replicated To Project project_dest${d} hello-world
@ -191,8 +188,7 @@ Test Case - Replication Exclusion Mode And Set Bandwidth
# push mode
Switch To System Labels
Create New Labels bad_${d}
Go Into Project project${d}
Go Into Repo project${d}/busybox
Go Into Repo project${d} busybox
Add Labels To Tag latest bad_${d}
Switch To Registries
Create A New Endpoint harbor e${d} https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
@ -207,10 +203,10 @@ Test Case - Replication Exclusion Mode And Set Bandwidth
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Replication Manage
Select Rule And Replicate rule${d}
Retry Wait Until Page Contains Succeeded
Check Latest Replication Job Status Succeeded
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Image Should Be Replicated To Project project_dest${d} hello-world period=0
Image Should Be Replicated To Project project_dest${d} hello-world
# make sure the excluded image is not replication
Retry Wait Until Page Contains 1 - 1 of 1 items
@ -223,8 +219,8 @@ Test Case - Replication Exclusion Mode And Set Bandwidth
Switch To Replication Manage
Create A Rule With Existing Endpoint rule${d} pull project${d}/* image e${d} project${d} filter_tag=3.10 filter_tag_model=excluding filter_label=bad_${d} filter_label_model=excluding bandwidth=2 bandwidth_unit=Mbps
Select Rule And Replicate rule${d}
Retry Wait Until Page Contains Succeeded
Image Should Be Replicated To Project project${d} hello-world period=0
Check Latest Replication Job Status Succeeded
Image Should Be Replicated To Project project${d} hello-world
# make sure the excluded image is not replication
Retry Wait Until Page Contains 1 - 1 of 1 items
Close Browser
@ -264,6 +260,7 @@ Test Case - Replication Of Pull Images from AWS-ECR To Self
Switch To Replication Manage
Create A Rule With Existing Endpoint rule${d} pull a/* image e${d} project${d}
Select Rule And Replicate rule${d}
Check Latest Replication Job Status Succeeded
Image Should Be Replicated To Project project${d} httpd
Image Should Be Replicated To Project project${d} alpine
Image Should Be Replicated To Project project${d} hello-world
@ -281,6 +278,7 @@ Test Case - Replication Of Pull Images from Google-GCR To Self
Create A Rule With Existing Endpoint rule${d} pull eminent-nation-87317/* image e${d} project${d}
Filter Replication Rule rule${d}
Select Rule And Replicate rule${d}
Check Latest Replication Job Status Succeeded
Image Should Be Replicated To Project project${d} httpd
Image Should Be Replicated To Project project${d} tomcat
Close Browser
@ -308,8 +306,8 @@ Test Case - Replication Of Push Images to Gitlab Triggered By Event
Test Case - Replication Of Pull Manifest List and CNAB from Harbor To Self
&{image1_with_tag}= Create Dictionary image=busybox tag=1.32.0 total_artifact_count=9 archive_count=0
&{image2_with_tag}= Create Dictionary image=index101603308079 tag=index_tag101603308079 total_artifact_count=2 archive_count=0
&{image3_with_tag}= Create Dictionary image=cnab011609785126 tag=cnab_tag011609785126 total_artifact_count=3 archive_count=2
&{image2_with_tag}= Create Dictionary image=index tag=index_tag total_artifact_count=2 archive_count=0
&{image3_with_tag}= Create Dictionary image=cnab tag=cnab_tag total_artifact_count=3 archive_count=2
${image1}= Get From Dictionary ${image1_with_tag} image
${image2}= Get From Dictionary ${image2_with_tag} image
${image3}= Get From Dictionary ${image3_with_tag} image
@ -386,20 +384,19 @@ Test Case - Robot Account Do Replication
Switch To Replication Manage
Create A Rule With Existing Endpoint rule_push_${d} push project${d}/* image e${d} project_dest${d}
Select Rule And Replicate rule_push_${d}
Retry Wait Until Page Contains Succeeded
Check Latest Replication Job Status Succeeded
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Image Should Be Replicated To Project project_dest${d} ${image1} period=0
Image Should Be Replicated To Project project_dest${d} ${image1}
Should Be Signed By Cosign ${tag1}
Image Should Be Replicated To Project project_dest${d} ${image2} period=0
Image Should Be Replicated To Project project_dest${d} ${image2}
Should Be Signed By Cosign ${tag2}
Back Project Home project_dest${d}
Retry Double Keywords When Error Go Into Repo project_dest${d}/${index} Should Be Signed By Cosign ${index_tag}
Back Project Home project_dest${d}
Go Into Repo project_dest${d}/${index}
Go Into Repo project_dest${d} ${index}
Should Be Signed By Cosign ${index_tag}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Back Project Home project_dest${d}
Go Into Repo project_dest${d}/${index}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed By Cosign ${image2_short_sha256}
# pull mode
Logout Harbor
@ -408,18 +405,17 @@ Test Case - Robot Account Do Replication
Switch To Replication Manage
Create A Rule With Existing Endpoint rule_pull_${d} pull project_dest${d}/* image e${d} project_dest${d}
Select Rule And Replicate rule_pull_${d}
Retry Wait Until Page Contains Succeeded
Image Should Be Replicated To Project project_dest${d} ${image1} period=0
Check Latest Replication Job Status Succeeded
Image Should Be Replicated To Project project_dest${d} ${image1}
Should Be Signed By Cosign ${tag1}
Image Should Be Replicated To Project project_dest${d} ${image2} period=0
Image Should Be Replicated To Project project_dest${d} ${image2}
Should Be Signed By Cosign ${tag2}
Back Project Home project_dest${d}
Retry Double Keywords When Error Go Into Repo project_dest${d}/${index} Should Be Signed By Cosign ${index_tag}
Back Project Home project_dest${d}
Go Into Repo project_dest${d}/${index}
Go Into Repo project_dest${d} ${index}
Should Be Signed By Cosign ${index_tag}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Back Project Home project_dest${d}
Go Into Repo project_dest${d}/${index}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed By Cosign ${image2_short_sha256}
Close Browser
@ -458,9 +454,9 @@ Test Case - Replication Triggered By Events
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Image Should Be Replicated To Project project_dest${d} ${image1} period=0
Image Should Be Replicated To Project project_dest${d} ${image2} period=0
Image Should Be Replicated To Project project_dest${d} ${index} period=0
Image Should Be Replicated To Project project_dest${d} ${image1}
Image Should Be Replicated To Project project_dest${d} ${image2}
Image Should Be Replicated To Project project_dest${d} ${index}
# sign
Cosign Generate Key Pair
Docker Login ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
@ -469,32 +465,28 @@ Test Case - Replication Triggered By Events
Cosign Sign ${ip}/project${d}/${index}@${image1sha256}
Logout Harbor
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Go Into Project project${d}
Retry Double Keywords When Error Go Into Repo project${d}/${image1} Should Be Signed By Cosign ${tag1}
Back Project Home project${d}
Retry Double Keywords When Error Go Into Repo project${d}/${index} Should Be Signed By Cosign ${index_tag}
Back Project Home project${d}
Go Into Repo project${d}/${index}
Go Into Repo project${d} ${image1}
Should Be Signed By Cosign ${tag1}
Go Into Repo project${d} ${index}
Should Be Signed By Cosign ${index_tag}
Go Into Repo project${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Back Project Home project${d}
Retry Double Keywords When Error Go Into Repo project${d}/${image2} Should Not Be Signed By Cosign ${tag2}
Back Project Home project${d}
Go Into Repo project${d}/${index}
Go Into Repo project${d} ${image2}
Should Not Be Signed By Cosign ${tag2}
Go Into Repo project${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed By Cosign ${image2_short_sha256}
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Go Into Project project_dest${d}
Retry Double Keywords When Error Go Into Repo project_dest${d}/${image1} Should Be Signed By Cosign ${tag1}
Back Project Home project_dest${d}
Retry Double Keywords When Error Go Into Repo project_dest${d}/${index} Should Be Signed By Cosign ${index_tag}
Back Project Home project_dest${d}
Go Into Repo project_dest${d}/${index}
Go Into Repo project_dest${d} ${image1}
Should Be Signed By Cosign ${tag1}
Go Into Repo project_dest${d} ${index}
Should Be Signed By Cosign ${index_tag}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Back Project Home project_dest${d}
Retry Double Keywords When Error Go Into Repo project_dest${d}/${image2} Should Not Be Signed By Cosign ${tag2}
Back Project Home project_dest${d}
Go Into Repo project_dest${d}/${index}
Go Into Repo project_dest${d} ${image2}
Should Not Be Signed By Cosign ${tag2}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed By Cosign ${image2_short_sha256}
Logout Harbor
# delete
@ -502,12 +494,10 @@ Test Case - Replication Triggered By Events
Go Into Project project${d}
Delete Repo project${d} ${image2}
Repo Not Exist project${d} ${image2}
Go Into Project project${d}
Go Into Repo project${d}/${image1}
Go Into Repo project${d} ${image1}
Retry Double Keywords When Error Delete Accessory ${tag1} Should be Accessory deleted ${tag1}
Should Not Be Signed By Cosign ${tag1}
Back Project Home project${d}
Go Into Repo project${d}/${index}
Go Into Repo project${d} ${index}
Retry Double Keywords When Error Delete Accessory ${index_tag} Should be Accessory deleted ${index_tag}
Should Not Be Signed By Cosign ${index_tag}
Click Index Achieve ${index_tag}
@ -516,14 +506,13 @@ Test Case - Replication Triggered By Events
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Go Into Project project_dest${d}
Go Into Repo project_dest${d}/${image2}
Go Into Repo project_dest${d} ${image2}
Wait Until Page Contains We couldn't find any artifacts!
Back Project Home project_dest${d}
Retry Double Keywords When Error Go Into Repo project_dest${d}/${image1} Should be Accessory deleted ${tag1}
Go Into Repo project_dest${d} ${image1}
Should be Accessory deleted ${tag1}
Should Not Be Signed By Cosign ${tag1}
Back Project Home project_dest${d}
Retry Double Keywords When Error Go Into Repo project_dest${d}/${index} Should be Accessory deleted ${index_tag}
Go Into Repo project_dest${d} ${index}
Should be Accessory deleted ${index_tag}
Should Not Be Signed By Cosign ${index_tag}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should be Accessory deleted ${image1_short_sha256}
Should Not Be Signed By Cosign ${image1_short_sha256}
@ -603,16 +592,15 @@ Test Case - Enable Replication Of Cosign Deployment Security Policy
Repo Exist project_pull_dest${d} ${image1}
Repo Exist project_pull_dest${d} ${image2}
Repo Exist project_pull_dest${d} ${index}
Retry Double Keywords When Error Go Into Repo project_pull_dest${d}/${image1} Should Be Signed By Cosign ${tag1}
Back Project Home project_pull_dest${d}
Retry Double Keywords When Error Go Into Repo project_pull_dest${d}/${image2} Should Be Signed By Cosign ${tag2}
Back Project Home project_pull_dest${d}
Retry Double Keywords When Error Go Into Repo project_pull_dest${d}/${index} Should Be Signed By Cosign ${index_tag}
Back Project Home project_pull_dest${d}
Go Into Repo project_pull_dest${d}/${index}
Go Into Repo project_pull_dest${d} ${image1}
Should Be Signed By Cosign ${tag1}
Go Into Repo project_pull_dest${d} ${image2}
Should Be Signed By Cosign ${tag2}
Go Into Repo project_pull_dest${d} ${index}
Should Be Signed By Cosign ${index_tag}
Go Into Repo project_pull_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Back Project Home project_pull_dest${d}
Go Into Repo project_pull_dest${d}/${index}
Go Into Repo project_pull_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image2_short_sha256}
# check project_push_dest
Go Into Project project_push_dest${d}
@ -620,16 +608,15 @@ Test Case - Enable Replication Of Cosign Deployment Security Policy
Repo Exist project_push_dest${d} ${image1}
Repo Exist project_push_dest${d} ${image2}
Repo Exist project_push_dest${d} ${index}
Retry Double Keywords When Error Go Into Repo project_push_dest${d}/${image1} Should Be Signed By Cosign ${tag1}
Back Project Home project_push_dest${d}
Retry Double Keywords When Error Go Into Repo project_push_dest${d}/${image2} Should Be Signed By Cosign ${tag2}
Back Project Home project_push_dest${d}
Retry Double Keywords When Error Go Into Repo project_push_dest${d}/${index} Should Be Signed By Cosign ${index_tag}
Back Project Home project_push_dest${d}
Go Into Repo project_push_dest${d}/${index}
Go Into Repo project_push_dest${d} ${image1}
Should Be Signed By Cosign ${tag1}
Go Into Repo project_push_dest${d} ${image2}
Should Be Signed By Cosign ${tag2}
Go Into Repo project_push_dest${d} ${index}
Should Be Signed By Cosign ${index_tag}
Go Into Repo project_push_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Back Project Home project_push_dest${d}
Go Into Repo project_push_dest${d}/${index}
Go Into Repo project_push_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image2_short_sha256}
Close Browser
@ -653,7 +640,7 @@ Test Case - Carvel Imgpkg Copy To Harbor
Imgpkg Copy From Registry To Registry ${ip}/project${d}/${repository}:${tag} ${ip1}/project_dest${d}/${repository}
Refresh Repositories
Repo Exist project_dest${d} ${repository}
Go Into Repo project_dest${d}/${repository}
Go Into Repo project_dest${d} ${repository}
Artifact Exist ${tag}
Back Project Home project_dest${d}
Delete Repo project_dest${d} ${repository}
@ -664,7 +651,7 @@ Test Case - Carvel Imgpkg Copy To Harbor
Refresh Repositories
Repo Exist project_dest${d} ${repository}
Retry Element Click ${repo_search_icon}
Go Into Repo project_dest${d}/${repository}
Go Into Repo project_dest${d} ${repository}
Artifact Exist ${tag}
Docker Logout ${ip}
Docker Logout ${ip1}

2
tests/robot-cases/Group1-Nightly/Routing.robot
View File

@ -93,7 +93,7 @@ Test Case - Open CVE Details Page
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} ${image} sha256=${sha256}
Go Into Repo project${d}/${image}
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Go Into Artifact ${sha256}
Retry Double Keywords When Error Click Link New Tab And Switch //hbr-artifact-vulnerabilities//clr-dg-row//a[contains(.,'${cve}')] Retry Wait Element //h1[contains(.,'${cve}')]

25
tests/robot-cases/Group1-Nightly/Schedule.robot
View File

@ -48,14 +48,13 @@ Test Case - Proxy Cache
Pull Image ${ip} ${test_user} ${test_pwd} project${d} ${user_namespace}/${manifest_index} tag=${manifest_tag}
Log To Console Start to Sleep 3 minitues......
Sleep 180
Go Into Project project${d}
Go Into Repo project${d}/${user_namespace}/${image}
Go Into Repo project${d} ${user_namespace}/${image}
FOR ${idx} IN RANGE 0 15
Log All Checking manifest ${idx} round......
Sleep 60
Go Into Project project${d}
${repo_out}= Run Keyword And Ignore Error Go Into Repo project${d}/${user_namespace}/${manifest_index}
${repo_out}= Run Keyword And Ignore Error Go Into Repo project${d} ${user_namespace}/${manifest_index}
Continue For Loop If '${repo_out[0]}'=='FAIL'
${artifact_out}= Run Keyword And Ignore Error Go Into Index And Contain Artifacts ${manifest_tag} total_artifact_count=1
Exit For Loop If '${artifact_out[0]}'=='PASS'
@ -78,7 +77,7 @@ Test Case - GC Schedule Job
Create An New Project And Go Into Project ${project_name}
Push image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image} sha256=${sha256}
Sleep 50
Go Into Repo ${project_name}/${image}
Go Into Repo ${project_name} ${image}
Switch To Garbage Collection
Set GC Schedule custom value=0 */2 * * * *
Sleep 480
@ -118,7 +117,7 @@ Test Case - Scan Schedule Job
Create An New Project And Go Into Project ${project_name}
Push image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image} sha256=${sha256}
Sleep 50
Go Into Repo ${project_name}/${image}
Go Into Repo ${project_name} ${image}
Retry Wait Until Page Contains Element ${not_scanned_icon}
Switch To Vulnerability Page
${flag}= Set Variable ${false}
@ -134,14 +133,12 @@ Test Case - Scan Schedule Job
# After scan custom schedule is set, image should stay in unscanned status.
Log To Console Sleep for 300 seconds......
Sleep 180
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image}
Go Into Repo ${project_name} ${image}
Retry Wait Until Page Contains Element ${not_scanned_icon}
Log To Console Sleep for 500 seconds......
Sleep 500
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image}
Go Into Repo ${project_name} ${image}
Scan Result Should Display In List Row ${sha256}
View Repo Scan Details Critical High
Close Browser
@ -175,11 +172,10 @@ Test Case - Replication Schedule Job
# After replication schedule is set, project should contain 2 images.
Log To Console Sleep for 720 seconds......
Sleep 720
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image_a}
Go Into Repo ${project_name} ${image_a}
Artifact Exist ${tag_a}
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image_b}
Go Into Repo ${project_name} ${image_b}
Artifact Exist ${tag_b}
# Delete repository
@ -190,11 +186,10 @@ Test Case - Replication Schedule Job
# After replication schedule is set, project should contain 2 images.
Log To Console Sleep for 600 seconds......
Sleep 600
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image_a}
Go Into Repo ${project_name} ${image_a}
Artifact Exist ${tag_a}
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image_b}
Go Into Repo ${project_name} ${image_b}
Artifact Exist ${tag_b}
Close Browser

7
tests/robot-cases/Group1-Nightly/Trivy.robot
View File

@ -54,10 +54,8 @@ Test Case - Scan A Tag In The Repo
Test Case - Scan As An Unprivileged User
Init Chrome Driver
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} library hello-world
Sign In Harbor ${HARBOR_URL} user024 Test1@34
Go Into Project library
Go Into Repo hello-world
Go Into Repo library hello-world
Select Object latest
Scan Is Disabled
Close Browser
@ -83,8 +81,7 @@ Test Case - Project Level Image Serverity Policy
${image}= Set Variable redis
Create An New Project And Go Into Project project${d}
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} ${image} sha256=${sha256}
Go Into Project project${d}
Go Into Repo ${image}
Go Into Repo project${d} ${image}
Scan Repo ${sha256} Succeed
Navigate To Projects
Go Into Project project${d}

8
tests/robot-cases/Group1-Nightly/multi_scanners.robot
View File

@ -38,16 +38,14 @@ Test Case - Switch Scanner
Create An New Project And Go Into Project project${d}
Push Image ${ip} admin Harbor12345 project${d} hello-world:latest
Go Into Project project${d}
Go Into Repo project${d}/hello-world
Go Into Repo project${d} hello-world
Scan Repo latest Succeed
Move To Summary Chart
Wait Until Page Contains No vulnerability
Switch To Scanners Page
Go Into Project project${d}
Go Into Repo project${d}/hello-world
Go Into Repo project${d} hello-world
Retry Wait Until Page Contains Element ${not_scanned_icon}
Scan Repo latest Succeed
Move To Summary Chart
@ -55,4 +53,4 @@ Test Case - Switch Scanner
Switch To Scanners Page
Set Default Scanner Trivy
Close Browser
Close Browser