Add Job Service Dashboard Schedules testcase (#18555)

Fix #17949

Signed-off-by: Yang Jiao <jiaoya@vmware.com>
This commit is contained in:
Yang Jiao 2023-04-20 11:11:59 +08:00 committed by GitHub
parent 8df99a5116
commit c7cbf0fcdb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 266 additions and 5 deletions

View File

@ -19,6 +19,10 @@ Documentation This resource provides any keywords related to the Harbor private
Switch To Job Queues
Retry Double Keywords When Error Retry Element Click //clr-main-container//clr-vertical-nav-group//span[contains(.,'Job Service Dashboard')] Retry Wait Until Page Contains Element ${job_service_stop_btn}
Switch To Job Schedules
Retry Element Click //clr-main-container//clr-vertical-nav-group//span[contains(.,'Job Service Dashboard')]
Retry Double Keywords When Error Retry Button Click ${job_service_schedules_btn} Retry Wait Until Page Contains Vendor Type
Select Jobs
[Arguments] @{job_types}
FOR ${job_type} IN @{job_types}
@ -117,3 +121,77 @@ Check Button Status
Select Jobs IMAGE_SCAN
Retry Wait Element Should Be Disabled ${job_service_resume_btn}
Retry Wait Element Should Be Disabled ${job_service_pause_btn}
Check Schedule List
[Arguments] ${schedule_cron}
# Check retention policy schedule
Retry Wait Until Page Contains Element //clr-dg-row[.//clr-dg-cell[text()='RETENTION'] and .//clr-dg-cell[text()='${schedule_cron}']]
# Check preheat policy schedule
Retry Wait Until Page Contains Element //clr-dg-row[.//clr-dg-cell[text()='P2P_PREHEAT'] and .//clr-dg-cell[text()='${schedule_cron}']]
# Check replication policy schedule
Retry Wait Until Page Contains Element //clr-dg-row[.//clr-dg-cell[text()='REPLICATION'] and .//clr-dg-cell[text()='${schedule_cron}']]
# Check scan all schedule
Retry Wait Until Page Contains Element //clr-dg-row[.//clr-dg-cell[text()='SCAN_ALL'] and .//clr-dg-cell[text()='${schedule_cron}']]
# Check GC schedule
Retry Wait Until Page Contains Element //clr-dg-row[.//clr-dg-cell[text()='GARBAGE_COLLECTION'] and .//clr-dg-cell[text()='${schedule_cron}']]
# Check log rotation schedule
Retry Wait Until Page Contains Element //clr-dg-row[.//clr-dg-cell[text()='PURGE_AUDIT_LOG'] and .//clr-dg-cell[text()='${schedule_cron}']]
# Check execution sweep schedule
Retry Wait Until Page Contains Element //clr-dg-row[.//clr-dg-cell[text()='EXECUTION_SWEEP'] and .//clr-dg-cell[text()='0 0 * * * *']]
# Check system artifact cleanup schedule
Retry Wait Until Page Contains Element //clr-dg-row[.//clr-dg-cell[text()='SYSTEM_ARTIFACT_CLEANUP'] and .//clr-dg-cell[text()='0 0 0 * * *']]
Pause All Schedules
Retry Double Keywords When Error Retry Button Click ${job_service_schedules_pause_all_btn} Retry Button Click ${confirm_btn}
Retry Wait Until Page Contains Paused all the schedules successfully
Retry Wait Until Page Contains Element //app-schedule-card//span[text()='Paused']
Resume All Schedules
Retry Double Keywords When Error Retry Button Click ${job_service_schedules_resume_all_btn} Retry Button Click ${confirm_btn}
Retry Wait Until Page Contains Resumed all the schedules successfully
Retry Wait Until Page Contains Element //app-schedule-card//span[text()='Running']
Check Schedules Status Is Pause
[Arguments] ${project_name} ${replication_rule_name} ${p2p_policy_name}
# Check that the retention policy schedule is Pause
Go Into Project ${project_name}
Switch To Tag Retention
Retry Wait Until Page Contains Element //span[text()='Schedule has been paused']
# Check that the preheat policy schedule is Pause
Switch To P2P Preheat
Retry Wait Until Page Contains Element //clr-dg-row[.//clr-dg-cell[text()='${p2p_policy_name}'] and .//clr-dg-cell[text()=' Scheduled(Paused) ']]
# Check that the replication policy schedule is Pause
Switch To Replication Manage
Retry Wait Until Page Contains Element //clr-dg-row[.//clr-dg-cell[text()='${replication_rule_name}'] and .//clr-dg-cell[text()=' Scheduled(Paused) ']]
# Check that the scan all schedule is Pause
Switch To Vulnerability Page
Retry Wait Until Page Contains Element //span[text()='Schedule to scan all has been paused']
# Check that the GC schedule is Pause
Switch To Garbage Collection
Retry Wait Until Page Contains Element //span[text()='Schedule to GC has been paused']
# Check that the log rotation schedule is Pause
Switch To Log Rotation
Retry Wait Until Page Contains Element //span[text()='Schedule to purge has been paused']
Check Schedules Status Is Not Pause
[Arguments] ${project_name} ${replication_rule_name} ${p2p_policy_name}
# Check that the retention policy schedule is not Pause
Go Into Project ${project_name}
Switch To Tag Retention
Retry Wait Until Page Not Contains Element //span[text()='Schedule has been paused']
# Check that the preheat policy schedule is not Pause
Switch To P2P Preheat
Retry Wait Until Page Not Contains Element //clr-dg-row[.//clr-dg-cell[text()='${p2p_policy_name}'] and .//clr-dg-cell[text()=' Scheduled(Paused) ']]
# Check that the replication policy schedule is not Pause
Switch To Replication Manage
Retry Wait Until Page Not Contains Element //clr-dg-row[.//clr-dg-cell[text()='${replication_rule_name}'] and .//clr-dg-cell[text()=' Scheduled(Paused) ']]
# Check that the scan all schedule is not Pause
Switch To Vulnerability Page
Retry Wait Until Page Not Contains Element //span[text()='Schedule to scan all has been paused']
# Check that the GC schedule is not Pause
Switch To Garbage Collection
Retry Wait Until Page Not Contains Element //span[text()='Schedule to GC has been paused']
# Check that the log rotation schedule is not Pause
Switch To Log Rotation
Retry Wait Until Page Not Contains Element //span[text()='Schedule to purge has been paused']

View File

@ -21,3 +21,6 @@ ${job_service_stop_all_btn} //button[contains(.,'STOP ALL')]
${job_service_pause_btn} //button[contains(text(),'PAUSE')]
${job_service_resume_btn} //button[contains(text(),'RESUME')]
${job_service_refresh_btn} //clr-datagrid//span[contains(@class,'refresh-btn')]
${job_service_schedules_btn} //*[@id='schedules']
${job_service_schedules_pause_all_btn} //button[.//span[text()=' PAUSE ALL ']]
${job_service_schedules_resume_all_btn} //button[.//span[text()=' RESUME ALL ']]

View File

@ -49,3 +49,9 @@ Should Contain Artifact
Should Not Contain Any Artifact
Retry Wait Until Page Not Contains Element xpath=//artifact-list-tab//clr-dg-row
Get The Specific Artifact
[Arguments] ${project_name} ${repo_name} ${reference}
${cmd}= Set Variable curl -u ${HARBOR_ADMIN}:${HARBOR_PASSWORD} -s --insecure -H "Content-Type: application/json" -X GET "${HARBOR_URL}/api/v2.0/projects/${project_name}/repositories/${repo_name}/artifacts/${reference}?page=1&page_size=10&with_tag=true&with_label=true&with_scan_overview=true&with_accessory=true&with_signature=true&with_immutable_status=true"
${rc} ${output}= Run And Return Rc And Output ${cmd}
[Return] ${output}

View File

@ -41,7 +41,7 @@ P2P Preheat Policy Not Exist
Retry Wait Until Page Not Contains Element //clr-dg-row[contains(.,'${name}')]
Create An New P2P Preheat Policy
[Arguments] ${policy_name} ${dist_name} ${repo} ${tag} ${trigger_type}=Manual
[Arguments] ${policy_name} ${dist_name} ${repo} ${tag} ${trigger_type}=Manual ${schedule_type}=${null} ${schedule_cron}=${null}
Switch To P2P Preheat
Retry Element Click ${p2p_preheat_new_policy_btn_id}
Select Distribution For P2P Preheat ${dist_name}
@ -49,6 +49,10 @@ Create An New P2P Preheat Policy
Retry Text Input ${p2p_preheat_repoinput_id} ${repo}
Retry Text Input ${p2p_preheat_tag_input_id} ${tag}
Select P2P Preheat Policy Trigger ${trigger_type}
Run Keyword If '${trigger_type}' == 'Scheduled' Retry Element Click ${p2p_preheat_scheduled_edit_id}
Run Keyword If '${trigger_type}' == 'Scheduled' Retry Double Keywords When Error Retry Element Click ${p2p_preheat_scheduled_type_select_id} Retry Element Click ${p2p_preheat_scheduled_type_select_id}//option[contains(.,'${schedule_type}')]
Run Keyword If '${schedule_type}' == 'Custom' Retry Text Input ${p2p_preheat_scheduled_cron_input_id} ${schedule_cron}
Run Keyword If '${trigger_type}' == 'Scheduled' Retry Element Click ${p2p_preheat_scheduled_save_btn_xpath}
Retry Double Keywords When Error Retry Element Click ${p2p_preheat_add_save_btn_id} Retry Wait Until Page Not Contains Element xpath=${p2p_preheat_add_save_btn_id}
P2P Preheat Policy Exist ${policy_name}

View File

@ -19,7 +19,6 @@ Resource ../../resources/Util.robot
*** Variables ***
*** Keywords ***
Add A Tag Retention Rule
Retry Element Click xpath=${project_tag_retention_add_rule_xpath}
Retry Element Click xpath=${project_tag_retention_template_xpath}
@ -85,6 +84,14 @@ Set Daily Schedule
Retry Element Click xpath=${project_tag_retention_schedule_ok_xpath}
Retry Wait Until Page Contains Element xpath=${project_tag_retention_span_daily_xpath}
Set Tag Retention Policy Schedule
[Arguments] ${type} ${cron}=${null}
Retry Double Keywords When Error Retry Element Click ${project_tag_retention_edit_schedule_xpath} Retry Wait Element Visible ${project_tag_retention_schedule_cancel_btn}
Retry Double Keywords When Error Retry Element Click ${project_tag_retention_select_policy_xpath} Retry Element Click //option[@value='${type}']
Run Keyword If '${type}' == 'Custom' Retry Text Input ${project_tag_retention_schedule_cron_input} ${cron}
Run Keyword If '${type}' == 'None' Retry Element Click ${project_tag_retention_config_save_xpath}
... ELSE Retry Double Keywords When Error Retry Element Click ${project_tag_retention_config_save_xpath} Retry Button Click ${project_tag_retention_schedule_ok_xpath}
Execute Result Should Be
[Arguments] ${image} ${result}
FOR ${idx} IN RANGE 0 20

View File

@ -42,6 +42,8 @@ ${project_tag_retention_delete_button_xpath} //div[contains(@class,'dropdown-me
${project_tag_retention_edit_button_xpath} //div[contains(@class,'dropdown-menu')]//button[contains(.,'Edit')]
${project_tag_retention_modal_title_edit_xpath} //h3[contains(.,'Edit Tag Retention Rule')]
${project_tag_retention_latest_execution_id_xpath} //clr-dg-row[1]//clr-dg-cell[1]
${project_tag_retention_schedule_cancel_btn} //button[contains(.,'CANCEL')]
${project_tag_retention_schedule_cron_input} //*[@id='targetCron']
${project_tag_immutability_scope_input_xpath} //*[@id='scope-input']
${project_tag_immutability_tag_input_xpath} //*[@id='tag-input']

View File

@ -237,6 +237,7 @@ Delete Replication Rule
Click Delete Button
Wait Until Page Contains Element ${dialog_delete}
Retry Double Keywords When Error Retry Element Click ${dialog_delete} Retry Wait Until Page Not Contains Element ${dialog_delete}
Reload Page
Filter Replication Rule ${name} exist=${false}
Rename Endpoint

View File

@ -14,8 +14,8 @@ Disable Scan Schedule
Set Scan Schedule
[Arguments] ${type} ${value}=${null}
Retry Double Keywords When Error Retry Element Click ${vulnerability_edit_btn} Retry Wait Until Page Not Contains Element ${vulnerability_edit_btn}
Retry Element Click ${vulnerability_dropdown_list}
Run Keyword If '${type}'=='custom' Run Keywords Retry Element Click ${vulnerability_dropdown_list_item_custom} AND Retry Text Input ${targetCron_id} ${value}
Retry Double Keywords When Error Retry Element Click ${vulnerability_dropdown_list} Retry Element Click //select[@id='selectPolicy']//option[contains(.,'${type}')]
Run Keyword If '${type}'=='Custom' Retry Text Input ${targetCron_id} ${value}
Retry Double Keywords When Error Retry Element Click ${vulnerability_save_btn} Retry Wait Until Page Not Contains Element ${vulnerability_save_btn}
Trigger Scan Now And Wait Until The Result Appears

View File

@ -550,3 +550,57 @@ Verify Webhook By Quota Exceed Event
Cannot Push image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image}:${tag} err_msg=adding 21.1 MiB of storage resource, which when updated to current usage of 48.5 MiB will exceed the configured upper limit of ${storage_quota}.0 MiB.
&{quota_exceed_property}= Create Dictionary type=QUOTA_EXCEED name=${image} namespace=${project_name}
Verify Request &{quota_exceed_property}
Create Schedules For Job Service Dashboard Schedules
[Arguments] ${project_name} ${schedule_type} ${schedule_cron} ${distribution_endpoint}=${null}
${d}= Get Current Date result_format=%m%s
${distribution_name}= Set Variable distribution${d}
${distribution_endpoint}= Set Variable If "${distribution_endpoint}" == "${null}" https://${d} ${distribution_endpoint}
${p2p_policy_name}= Set Variable policy${d}
${replication_policy_name}= Set Variable rule${d}
# Create a retention policy triggered by schedule
Switch To Tag Retention
Add A Tag Retention Rule
Set Tag Retention Policy Schedule ${schedule_type} ${schedule_cron}
# Create a preheat policy triggered by schedule
Create An New Distribution Dragonfly ${distribution_name} ${distribution_endpoint}
Go Into Project ${project_name}
Create An New P2P Preheat Policy ${p2p_policy_name} ${distribution_name} ** ** Scheduled ${schedule_type} ${schedule_cron}
# Create a replication policy triggered by schedule
Switch to Registries
Create A New Endpoint docker-hub docker-hub${d} ${null} ${null} ${null} Y
Switch To Replication Manage
Create A Rule With Existing Endpoint ${replication_policy_name} pull goharbor/harbor-core image docker-hub${d} ${project_name} filter_tag=dev mode=Scheduled cron=${schedule_cron}
# Set up a schedule to scan all
Switch To Vulnerability Page
Set Scan Schedule Custom value=${schedule_cron}
# Set up a schedule to GC
Switch To Garbage Collection
Set GC Schedule custom value=${schedule_cron}
# Set up a schedule to log rotation
Switch To Log Rotation
${exclude_operations} Create List Pull
Set Log Rotation Schedule 1 Days ${schedule_type} ${schedule_cron} ${exclude_operations}
[Return] ${replication_policy_name} ${p2p_policy_name} ${distribution_name}
Reset Schedules For Job Service Dashboard Schedules
[Arguments] ${project_name} ${replication_policy_name} ${p2p_policy_name}
Go Into Project ${project_name}
# Reset the schedule of retention policy
Switch To Tag Retention
Set Tag Retention Policy Schedule None
# Reset the schedule of preheat policy
Switch To P2P Preheat
Delete A P2P Preheat Policy ${p2p_policy_name}
# Reset the schedule of replication policy
Switch To Replication Manage
Delete Replication Rule ${replication_policy_name}
# Reset the schedule of scan all
Switch To Vulnerability Page
Set Scan Schedule None
# Reset the schedule of GC
Switch To Garbage Collection
Set GC Schedule None
# Reset the schedule of log rotation
Switch To Log Rotation
Set Log Rotation Schedule 2 Days None

View File

@ -1086,3 +1086,23 @@ Test Case - Job Service Dashboard Job Queues
Check Jobs Latency IMAGE_SCAN=${true} PURGE_AUDIT_LOG=${true}
Resume Jobs IMAGE_SCAN PURGE_AUDIT_LOG
Close Browser
Test Case - Job Service Dashboard Schedules
[Tags] job_service_schedules
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
${schedule_type}= Set Variable Custom
${schedule_cron}= Set Variable 0 0 12 * * ?
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Project And Go Into Project project${d}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} photon 2.0 2.0
${replication_policy_name} ${p2p_policy_name} ${distribution_name}= Create Schedules For Job Service Dashboard Schedules project${d} ${schedule_type} ${schedule_cron}
Switch To Job Schedules
Check Schedule List ${schedule_cron}
Pause All Schedules
Check Schedules Status Is Pause project${d} ${replication_policy_name} ${p2p_policy_name}
Switch To Job Schedules
Resume All Schedules
Check Schedules Status Is Not Pause project${d} ${replication_policy_name} ${p2p_policy_name}
Reset Schedules For Job Service Dashboard Schedules project${d} ${replication_policy_name} ${p2p_policy_name}
Close Browser

View File

@ -128,7 +128,7 @@ Test Case - Scan Schedule Job
${left} = Evaluate ${minite_int}%10
Log To Console ${i}/${left}
Sleep 55
Run Keyword If ${left} <= 3 and ${left} != 0 Run Keywords Set Scan Schedule custom value=0 */10 * * * * AND Set Suite Variable ${flag} ${true}
Run Keyword If ${left} <= 3 and ${left} != 0 Run Keywords Set Scan Schedule Custom value=0 */10 * * * * AND Set Suite Variable ${flag} ${true}
Exit For Loop If '${flag}' == '${true}'
END
# After scan custom schedule is set, image should stay in unscanned status.
@ -256,3 +256,89 @@ Test Case - Log Rotation Schedule Job
END
Should Be True ${len} > 3 and ${len} < 6
Close Browser
Test Case - Job Service Dashboard Schedules
[Tags] job_service_schedules
Init Chrome Driver
${d}= Get Current Date result_format=%m%s
${schedule_type}= Set Variable Custom
${schedule_cron}= Set Variable 0 */2 * * * *
${image}= Set Variable photon
${tag}= Set Variable 2.0
${project_name}= Set Variable project${d}
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Create An New Project And Go Into Project ${project_name}
Push Image With Tag ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} ${project_name} ${image} ${tag} ${tag}
${replication_policy_name} ${p2p_policy_name} ${distribution_name}= Create Schedules For Job Service Dashboard Schedules ${project_name} ${schedule_type} ${schedule_cron} ${DISTRIBUTION_ENDPOINT}
Switch To Job Schedules
Check Schedule List ${schedule_cron}
Sleep 150
Pause All Schedules
# Check schedule is running
Go Into Project ${project_name}
# Check that the tag tetention schedule is running
Switch To Tag Retention
Wait Until Element Is Visible And Enabled //tag-retention//clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
${tag_retention_start_time1}= Get Text //tag-retention//clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
# Check that the preheat policy schedule is running
Switch To P2P Preheat
Select P2P Preheat Policy ${p2p_policy_name}
Wait Until Element Is Visible And Enabled //div[./h4[text()='Executions']]//clr-datagrid//clr-dg-row[1]//clr-dg-cell[4]
${preheat_start_time1}= Get Text //div[./h4[text()='Executions']]//clr-datagrid//clr-dg-row[1]//clr-dg-cell[4]
# Check that the replication schedule is running
Switch To Replication Manage
Select Rule ${replication_policy_name}
Wait Until Element Is Visible And Enabled //clr-datagrid[.//span[text()='Total']]//clr-dg-row[1]//clr-dg-cell[4]
${replication_start_time1}= Get Text //clr-datagrid[.//span[text()='Total']]//clr-dg-row[1]//clr-dg-cell[4]
# Check that the scan all schedule is running
${artifact_info}= Get The Specific Artifact ${project_name} ${image} ${tag}
${artifact_info}= Evaluate json.loads("""${artifact_info}""") json
${scan_all_start_time1}= Set Variable ${artifact_info["scan_overview"]["application/vnd.security.vulnerability.report; version=1.1"]["start_time"]}
# Check that the GC schedule is running
Switch To Garbage Collection
Wait Until Element Is Visible And Enabled //clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
${gc_start_time1}= Get Text //clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
# Check that the log rotation schedule is running
Switch To Log Rotation
Wait Until Element Is Visible And Enabled //clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
${log_rotation_start_time1}= Get Text //clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
Sleep 150
# Check schedules is paused
Go Into Project ${project_name}
# Check that the tag tetention schedule is paused
Switch To Tag Retention
Wait Until Element Is Visible And Enabled //tag-retention//clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
${tag_retention_start_time2}= Get Text //tag-retention//clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
Should Be Equal ${tag_retention_start_time1} ${tag_retention_start_time2}
# Check that the preheat policy schedule is paused
Switch To P2P Preheat
Select P2P Preheat Policy ${p2p_policy_name}
Wait Until Element Is Visible And Enabled //div[./h4[text()='Executions']]//clr-datagrid//clr-dg-row[1]//clr-dg-cell[4]
${preheat_start_time2}= Get Text //div[./h4[text()='Executions']]//clr-datagrid//clr-dg-row[1]//clr-dg-cell[4]
Should Be Equal ${preheat_start_time1} ${preheat_start_time2}
# Check that the replication schedule is paused
Switch To Replication Manage
Select Rule ${replication_policy_name}
Wait Until Element Is Visible And Enabled //clr-datagrid[.//span[text()='Total']]//clr-dg-row[1]//clr-dg-cell[4]
${replication_start_time2}= Get Text //clr-datagrid[.//span[text()='Total']]//clr-dg-row[1]//clr-dg-cell[4]
Should Be Equal ${replication_start_time1} ${replication_start_time2}
# Check that the scan all schedule is paused
${artifact_info}= Get The Specific Artifact ${project_name} ${image} ${tag}
${artifact_info}= Evaluate json.loads("""${artifact_info}""") json
${scan_all_start_time2}= Set Variable ${artifact_info["scan_overview"]["application/vnd.security.vulnerability.report; version=1.1"]["start_time"]}
Should Be Equal ${scan_all_start_time1} ${scan_all_start_time2}
# Check that the GC schedule is paused
Switch To Garbage Collection
Wait Until Element Is Visible And Enabled //clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
${gc_start_time2}= Get Text //clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
Should Be Equal ${gc_start_time1} ${gc_start_time2}
# Check that the log rotation schedule is paused
Switch To Log Rotation
Wait Until Element Is Visible And Enabled //clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
${log_rotation_start_time2}= Get Text //clr-datagrid//clr-dg-row[1]//clr-dg-cell[5]
Should Be Equal ${log_rotation_start_time1} ${log_rotation_start_time2}
Reset Schedules For Job Service Dashboard Schedules ${project_name} ${replication_policy_name} ${p2p_policy_name}
Delete A Distribution ${distribution_name} ${DISTRIBUTION_ENDPOINT}
Switch To Job Schedules
Resume All Schedules
Close Browser