Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-06-26 23:05:04 +02:00
Code Issues Projects Releases Wiki Activity

Add P2P Preheat Test case (#17089)

Browse Source 
1.Add P2P Preheat test case triggered by manual
2.Add P2P Preheat test case triggered by events(Artifact is pushed Artifact is scanned Artifact is labeled)
3.Add P2P Preheat test case triggered by schedule
4.Migrated Distribution CRUD and P2P Preheat Policy CRUD to P2P_Preheat.robot for classification

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
This commit is contained in:
Yang Jiao 2022-06-28 18:14:46 +08:00 committed by GitHub
parent aa63569de2
commit ff4eb7f27c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 254 additions and 49 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
tests/resources/Harbor-Pages/Configuration.robot
View File

@ -152,9 +152,7 @@ Switch To Project Quotas
Sleep 1
Switch To Distribution
Sleep 1
Retry Element Click xpath=//clr-main-container//clr-vertical-nav-group//span[contains(.,'Distributions')]
Sleep 1
Switch To Robot Account
Sleep 1
@ -380,7 +378,6 @@ Delete A Distribution
Retry Double Keywords When Error Retry Element Click ${distribution_action_btn_id} Wait Until Element Is Visible And Enabled ${distribution_del_btn_id}
Retry Double Keywords When Error Retry Element Click ${distribution_del_btn_id} Wait Until Element Is Visible And Enabled ${delete_confirm_btn}
Retry Double Keywords When Error Retry Element Click ${delete_confirm_btn} Retry Wait Until Page Not Contains Element ${delete_confirm_btn}
Sleep 10
Filter Distribution List ${name} ${endpoint} exsit=${is_exsit}
Edit A Distribution

12
tests/resources/Harbor-Pages/Project-P2P-Preheat-Elements.robot
View File

@ -28,7 +28,13 @@ ${p2p_preheat_edit_save_btn_id} //*[@id='edit-policy-save']
${p2p_preheat_action_btn_id} //*[@id='action-policy']
${p2p_preheat_del_btn_id} //*[@id='delete-policy']
${p2p_preheat_edit_btn_id} //*[@id='edit-policy']
${p2p_preheat_execute_btn_id} //*[@id='execute-policy']
${p2p_execution_header} //clr-main-container//project-detail//ng-component//h4[contains(.,'Executions')]
${p2p_preheat_confirm_execute_btn_id} //button[contains(.,'CONFIRM')]
${p2p_preheat_latest_execute_id_xpath} //clr-datagrid[contains(.,'ID')]//div//clr-dg-row[1]//clr-dg-cell[1]//a
${p2p_preheat_trigger_select} //select[@id='trigger-type']
${p2p_preheat_executions_refresh_xpath} //div[contains(@class,'col-lg-12')]//span[@class='refresh-btn']
${p2p_preheat_scheduled_type_select_id} //*[@id='inline-select']
${p2p_preheat_scheduled_edit_id} //*[@id='inline-edit']
${p2p_preheat_scheduled_cron_input_id} //*[@id='inline-target']
${p2p_preheat_scheduled_save_btn_xpath} //button[contains(.,'SAVE')]

110
tests/resources/Harbor-Pages/Project-P2P-Preheat.robot
View File

@ -23,13 +23,13 @@ Switch To P2P Preheat
Retry Element Click xpath=${project_p2p_preheat__tag_xpath}
Select Distribution For P2P Preheat
[Arguments] ${provider}
Retry Element Click ${p2p_preheat_provider_select_id}
Retry Element Click ${p2p_preheat_provider_select_id}//option[contains(.,'${provider}')]
[Arguments] ${provider}
Retry Element Click ${p2p_preheat_provider_select_id}
Retry Element Click ${p2p_preheat_provider_select_id}//option[contains(.,'${provider}')]
Select P2P Preheat Policy
[Arguments] ${name}
Retry Element Click //clr-dg-row[contains(.,'${name}')]//clr-radio-wrapper/label
[Arguments] ${name}
Retry Element Click //clr-dg-row[contains(.,'${name}')]//clr-radio-wrapper/label
P2P Preheat Policy Exist
[Arguments] ${name} ${repo}=${null}
@ -41,31 +41,121 @@ P2P Preheat Policy Not Exist
Retry Wait Until Page Not Contains Element //clr-dg-row[contains(.,'${name}')]
Create An New P2P Preheat Policy
[Arguments] ${policy_name} ${dist_name} ${repo} ${tag} ${trigger_type}=${null}
[Arguments] ${policy_name} ${dist_name} ${repo} ${tag} ${trigger_type}=Manual
Switch To P2P Preheat
Retry Element Click ${p2p_preheat_new_policy_btn_id}
Select Distribution For P2P Preheat ${dist_name}
Retry Text Input ${p2p_preheat_name_input_id} ${policy_name}
Retry Text Input ${p2p_preheat_repoinput_id} ${repo}
Retry Text Input ${p2p_preheat_tag_input_id} ${tag}
Select P2P Preheat Policy Trigger ${trigger_type}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_add_save_btn_id} Retry Wait Until Page Not Contains Element xpath=${p2p_preheat_add_save_btn_id}
P2P Preheat Policy Exist ${policy_name}
Select P2P Preheat Policy Trigger
[Arguments] ${mode}
Retry Element Click ${p2p_preheat_trigger_select}
Retry Element Click ${p2p_preheat_trigger_select}//option[contains(.,'${mode}')]
Edit A P2P Preheat Policy
[Arguments] ${name} ${repo} ${trigger_type}=${null}
[Arguments] ${name} ${repo} ${trigger_type}=${null}
Switch To P2P Preheat
Retry Double Keywords When Error Select P2P Preheat Policy ${name} Wait Until Element Is Visible ${p2p_execution_header}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_action_btn_id} Wait Until Element Is Visible And Enabled ${p2p_preheat_edit_btn_id}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_edit_btn_id} Wait Until Element Is Visible And Enabled ${p2p_preheat_name_input_id}
Retry Text Input ${p2p_preheat_repoinput_id} ${repo}
Run Keyword If '${trigger_type}' != '${null}' Select P2P Preheat Policy Trigger ${trigger_type}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_edit_save_btn_id} Retry Wait Until Page Not Contains Element xpath=${p2p_preheat_edit_save_btn_id}
P2P Preheat Policy Exist ${name} repo=${repo}
Delete A P2P Preheat Policy
[Arguments] ${name}
Switch To P2P Preheat
Set P2P Preheat Policy Schedule
[Arguments] ${name} ${type} ${cron}=${null}
Retry Double Keywords When Error Select P2P Preheat Policy ${name} Wait Until Element Is Visible ${p2p_execution_header}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_action_btn_id} Wait Until Element Is Visible And Enabled ${p2p_preheat_edit_btn_id}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_edit_btn_id} Wait Until Element Is Visible And Enabled ${p2p_preheat_name_input_id}
Retry Double Keywords When Error Select P2P Preheat Policy Trigger Scheduled Retry Element Click ${p2p_preheat_scheduled_edit_id}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_scheduled_type_select_id} Retry Element Click ${p2p_preheat_scheduled_type_select_id}//option[contains(.,'${type}')]
Run Keyword If '${type}' == 'Custom' Retry Text Input ${p2p_preheat_scheduled_cron_input_id} ${cron}
Retry Element Click ${p2p_preheat_scheduled_save_btn_xpath}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_edit_save_btn_id} Retry Wait Until Page Not Contains Element xpath=${p2p_preheat_edit_save_btn_id}
P2P Preheat Policy Exist ${name}
Delete A P2P Preheat Policy
[Arguments] ${name}
Switch To P2P Preheat
Retry Double Keywords When Error Select P2P Preheat Policy ${name} Wait Until Element Is Visible ${p2p_execution_header}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_action_btn_id} Wait Until Element Is Visible And Enabled ${p2p_preheat_del_btn_id}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_del_btn_id} Wait Until Element Is Visible And Enabled ${delete_confirm_btn}
Retry Double Keywords When Error Retry Element Click ${delete_confirm_btn} Retry Wait Until Page Not Contains Element ${delete_confirm_btn}
P2P Preheat Policy Not Exist ${name}
Execute P2P Preheat And Verify
[Arguments] ${project_name} ${policy_name} ${contain} ${not_contain}
Go Into Project ${project_name}
Switch To P2P Preheat
Execute P2P Preheat ${policy_name}
Verify Latest Execution Result ${project_name} ${policy_name} ${contain} ${not_contain}
Execute P2P Preheat
[Arguments] ${name} ${expected_status}=Success
Retry Double Keywords When Error Select P2P Preheat Policy ${name} Wait Until Element Is Visible ${p2p_execution_header}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_action_btn_id} Wait Until Element Is Visible And Enabled ${p2p_preheat_execute_btn_id}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_execute_btn_id} Wait Until Element Is Visible And Enabled ${p2p_preheat_confirm_execute_btn_id}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_confirm_execute_btn_id} Wait Until Element Is Visible And Enabled //clr-datagrid//div//clr-dg-row[1]//clr-dg-cell[2][contains(.,'${expected_status}')]
Verify Latest Execution Result
[Arguments] ${project_name} ${policy_name} ${contain} ${not_contain}=${null} ${expected_status}=Success
Retry Double Keywords When Error Select P2P Preheat Policy ${policy_name} Wait Until Element Is Visible ${p2p_preheat_executions_refresh_xpath}
${latest_execution_id}= Get Text ${p2p_preheat_latest_execute_id_xpath}
Retry P2P Preheat Be Successful ${project_name} ${policy_name} ${latest_execution_id} ${contain} ${not_contain}
Retry P2P Preheat Be Successful
[Arguments] ${project_name} ${policy_name} ${execution_id} ${contain} ${not_contain}=${null} ${expected_status}=Success
Retry Keyword N Times When Error 15 P2P Preheat Be Successful ${project_name} ${policy_name} ${execution_id} ${contain} ${not_contain} ${expected_status}
P2P Preheat Be Successful
[Arguments] ${project_name} ${policy_name} ${execution_id} ${contain} ${not_contain}=${null} ${expected_status}=Success
${rc} ${output}= Run And Return Rc And Output curl -u ${HARBOR_ADMIN}:${HARBOR_PASSWORD} -i --insecure -H "Content-Type: application/json" -X GET "https://${ip}/api/v2.0/projects/${project_name}/preheat/policies/${policy_name}/executions/${execution_id}/tasks"
Log All ${output}
Should Be Equal As Integers ${rc} 0
Should Contain Any ${output} ${expected_status} @{contain}
${out} Run Keyword And Ignore Error Get Length ${not_contain}
Run Keyword If '${out[0]}'=='PASS' Should Not Contain Any ${output} @{not_contain}
Verify Artifact Is Pushed Event
[Arguments] ${project_name} ${policy_name} ${image} ${tag}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image} ${tag} ${tag}
Go Into Project ${project_name}
Switch To P2P Preheat
${contain} Create List ${project_name}/${image}:${tag}
Verify Latest Execution Result ${project_name} ${policy_name} ${contain}
Verify Artifact Is Scanned Event
[Arguments] ${project_name} ${policy_name} ${image} ${tag}
Go Into Project ${project_name}
Scan Artifact ${project_name} ${image} //clr-dg-row[contains(.,'${tag}')]//label[1]
Back Project Home ${project_name}
Switch To P2P Preheat
${contain} Create List ${project_name}/${image}:${tag}
Verify Latest Execution Result ${project_name} ${policy_name} ${contain}
Verify Artifact Is Labeled Event
[Arguments] ${project_name} ${policy_name} ${image} ${tag} ${label}
Go Into Project ${project_name}
Switch To Project Label
Create New Labels ${label}
Switch To Project Repo
Go Into Repo ${image}
Add Labels To Tag ${tag} ${label}
Back Project Home ${project_name}
Switch To P2P Preheat
${contain} Create List ${project_name}/${image}:${tag}
Verify Latest Execution Result ${project_name} ${policy_name} ${contain}
Get P2P Preheat Logs
[Arguments] ${project_name} ${policy_name}
${cmd}= Set Variable curl -u ${HARBOR_ADMIN}:${HARBOR_PASSWORD} -s --insecure -H "Content-Type: application/json" -X GET "https://${ip}/api/v2.0/projects/${project_name}/preheat/policies/${policy_name}/executions"
Log All cmd:${cmd}
${rc} ${output}= Run And Return Rc And Output ${cmd}
Log All ${output}
[Return] ${output}

33
tests/robot-cases/Group1-Nightly/Common.robot
View File

@ -783,39 +783,6 @@ Test Case - Read Only Mode
Push image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} busybox:latest
Close Browser
Test Case - Distribution CRUD
${d}= Get Current Date result_format=%m%s
${name}= Set Variable distribution${d}
${endpoint}= Set Variable https://32.1.1.2
${endpoint_new}= Set Variable https://10.65.65.42
Init Chrome Driver
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Distribution Dragonfly ${name} ${endpoint}
Edit A Distribution ${name} ${endpoint} new_endpoint=${endpoint_new}
Delete A Distribution ${name} ${endpoint_new}
Close Browser
Test Case - P2P Preheat Policy CRUD
${d}= Get Current Date result_format=%m%s
${pro_name}= Set Variable project_p2p${d}
${dist_name}= Set Variable distribution${d}
${endpoint}= Set Variable https://20.76.1.2
${policy_name}= Set Variable policy${d}
${repo}= Set Variable alpine
${repo_new}= Set Variable redis*
${tag}= Set Variable v1.0
Init Chrome Driver
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Distribution Dragonfly ${dist_name} ${endpoint}
Create An New Project And Go Into Project ${pro_name}
Create An New P2P Preheat Policy ${policy_name} ${dist_name} ${repo} ${tag}
Edit A P2P Preheat Policy ${policy_name} ${repo_new}
Delete A Distribution ${dist_name} ${endpoint} deletable=${false}
Go Into Project ${pro_name} has_image=${false}
Delete A P2P Preheat Policy ${policy_name}
Delete A Distribution ${dist_name} ${endpoint}
Close Browser
Test Case - System Robot Account Cover All Projects
[Tags] sys_robot_account_cover
${d}= Get Current Date result_format=%m%s

110
tests/robot-cases/Group1-Nightly/P2P_Preheat.robot Normal file
View File

@ -0,0 +1,110 @@
# Copyright Project Harbor Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License
*** Settings ***
Documentation Harbor BATs
Resource ../../resources/Util.robot
Default Tags Nightly
*** Variables ***
${HARBOR_URL} https://${ip}
${HARBOR_ADMIN} admin
*** Test Cases ***
Test Case - Distribution CRUD
${d}= Get Current Date result_format=%m%s
${name}= Set Variable distribution${d}
${endpoint}= Set Variable https://32.1.1.2
${endpoint_new}= Set Variable https://10.65.65.42
Init Chrome Driver
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Distribution Dragonfly ${name} ${endpoint}
Edit A Distribution ${name} ${endpoint} new_endpoint=${endpoint_new}
Delete A Distribution ${name} ${endpoint_new}
Close Browser
Test Case - P2P Preheat Policy CRUD
${d}= Get Current Date result_format=%m%s
${project_name}= Set Variable project_p2p${d}
${dist_name}= Set Variable distribution${d}
${endpoint}= Set Variable https://20.76.1.2
${policy_name}= Set Variable policy${d}
${repo}= Set Variable alpine
${repo_new}= Set Variable redis*
${tag}= Set Variable v1.0
Init Chrome Driver
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Distribution Dragonfly ${dist_name} ${endpoint}
Create An New Project And Go Into Project ${project_name}
Create An New P2P Preheat Policy ${policy_name} ${dist_name} ${repo} ${tag}
Edit A P2P Preheat Policy ${policy_name} ${repo_new}
Delete A Distribution ${dist_name} ${endpoint} deletable=${false}
Go Into Project ${project_name} has_image=${false}
Delete A P2P Preheat Policy ${policy_name}
Delete A Distribution ${dist_name} ${endpoint}
Close Browser
Test Case - P2P Preheat By Manual
[Tags] p2p_preheat_by_manual need_distribution_endpoint
${d}= Get Current Date result_format=%m%s
${project_name}= Set Variable project_p2p${d}
${dist_name}= Set Variable distribution${d}
${policy_name}= Set Variable policy${d}
${image1}= Set Variable busybox
${image2}= Set Variable hello-world
${tag1}= Set Variable latest
${tag2}= Set Variable stable
Init Chrome Driver
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Distribution Dragonfly ${dist_name} ${DISTRIBUTION_ENDPOINT}
Create An New Project And Go Into Project ${project_name}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image1} ${tag1} ${tag1}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image1} ${tag2} ${tag2}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image2} ${tag1} ${tag1}
Create An New P2P Preheat Policy ${policy_name} ${dist_name} ${image1} ${tag1}
${contain} Create List ${project_name}/${image1}:${tag1}
${not_contain} Create List ${project_name}/${image1}:${tag2} ${project_name}/${image2}:${tag1}
Retry Action Keyword Execute P2P Preheat And Verify ${project_name} ${policy_name} ${contain} ${not_contain}
Delete A P2P Preheat Policy ${policy_name}
Delete A Distribution ${dist_name} ${DISTRIBUTION_ENDPOINT}
Close Browser
Test Case - P2P Preheat By Event
[Tags] p2p_preheat_by_event need_distribution_endpoint
${d}= Get Current Date result_format=%m%s
${project_name}= Set Variable project_p2p${d}
${dist_name}= Set Variable distribution${d}
${policy_name}= Set Variable policy${d}
${image1}= Set Variable busybox
${image2}= Set Variable hello-world
${tag1}= Set Variable latest
${tag2}= Set Variable stable
${label}= Set Variable p2p_preheat
Init Chrome Driver
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Distribution Dragonfly ${dist_name} ${DISTRIBUTION_ENDPOINT}
Create An New Project And Go Into Project ${project_name}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image1} ${tag1} ${tag1}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image1} ${tag2} ${tag2}
Create An New P2P Preheat Policy ${policy_name} ${dist_name} ** ** Event based
Retry Double Keywords When Error Select P2P Preheat Policy ${policy_name} Wait Until Element Is Visible ${p2p_execution_header}
# Artifact is pushed event
Retry Action Keyword Verify Artifact Is Pushed Event ${project_name} ${policy_name} ${image2} ${tag1}
# Artifact is scanned event
Retry Action Keyword Verify Artifact Is Scanned Event ${project_name} ${policy_name} ${image1} ${tag1}
# Artifact is labeled event
Retry Action Keyword Verify Artifact Is Labeled Event ${project_name} ${policy_name} ${image1} ${tag2} ${label}
Delete A P2P Preheat Policy ${policy_name}
Delete A Distribution ${dist_name} ${DISTRIBUTION_ENDPOINT}
Close Browser

35
tests/robot-cases/Group1-Nightly/Schedule.robot
View File

@ -104,6 +104,7 @@ Test Case - GC Schedule Job
END
#Only return latest 10 records for GC job
Should Be True ${len} > 3 and ${len} < 6
Close Browser
Test Case - Scan Schedule Job
[tags] Scan_schedule
@ -143,6 +144,7 @@ Test Case - Scan Schedule Job
Go Into Repo ${project_name}/${image}
Scan Result Should Display In List Row ${sha256}
View Repo Scan Details Critical High
Close Browser
Test Case - Replication Schedule Job
[tags] Replication_schedule
@ -194,3 +196,36 @@ Test Case - Replication Schedule Job
Go Into Project ${project_name}
Go Into Repo ${project_name}/${image_b}
Artifact Exist ${tag_b}
Close Browser
Test Case - P2P Preheat Schedule Job
[Tags] p2p_preheat_schedule need_distribution_endpoint
Init Chrome Driver
${d}= Get Current Date result_format=%M
${project_name}= Set Variable p2p_preheat_schedule_proj${d}
${dist_name}= Set Variable distribution${d}
${policy_name}= Set Variable policy${d}
${image}= Set Variable busybox
${tag}= Set Variable latest
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Distribution Dragonfly ${dist_name} ${DISTRIBUTION_ENDPOINT}
Create An New Project And Go Into Project ${project_name}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image} ${tag}
Create An New P2P Preheat Policy ${policy_name} ${dist_name} ** **
Set P2P Preheat Policy Schedule ${policy_name} Custom 0 */2 * * * *
Sleep 480
Edit A P2P Preheat Policy ${policy_name} ** Manual
Sleep 180
${logs}= Get P2P Preheat Logs ${project_name} ${policy_name}
${logs}= Evaluate json.loads("""${logs}""") json
${len}= Get Length ${logs}
FOR ${log} IN @{logs}
Log ${log}
Should Be Equal As Strings ${log["trigger"]} scheduled
Should Be Equal As Strings ${log["status"]} Success
Should Be Equal As Strings ${log["vendor_type"]} P2P_PREHEAT
END
Should Be True ${len} > 3 and ${len} < 6
Delete A P2P Preheat Policy ${policy_name}
Delete A Distribution ${dist_name} ${DISTRIBUTION_ENDPOINT}
Close Browser