Merge pull request #4108 from ywk253100/180123_manual_case

Update manual test cases for replication

tests/testcases/Group7-Replication/7-01-Proj-replication-rules-add.md

@@ -16,26 +16,19 @@ User guide
 # Test Steps:
 
 1. Login UI as admin user.
-2. In Project replication page, add a replication rule using an existing endpoint with enable checked.  
+2. In `Administration->Replications` page, create a new rule and fill in name and description.  
-3. In Project replication page, add a replication rule using an existing endpoint without enable checked.  
+3. Choose a project by clicking the icon on the right of `Projects`.  
-4. In Project replication page, add a replication rule using a new endpoint.
+4. Add repository and tag filter.
-5. In Project replication page, add a replication rule using a new endpoint. Provide invalid values of input to see if validation works:
+5. Choose an endpoint, create a new one if no endpoint exists.  
-
+6. Select a triggering condition: Immediate/Manaul/Scheduled, check/uncheck the `Delete remote images when locally deleted` if choosing `Immediate`.  
-* endpoint name or ip address duplicate with an existing endpoint.  
+7. Check/uncheck the option `Replicate existing images immediately`.  
-* endpoint ip address incorrect.  
+8. Save the rule.  
-* endpoint username or password incorrect.  
-
-6. Add another rule with different name using the same endpoint.  
 
+Repeat steps 1-8 under `Projects->Project_Name->Replication` page.
 
 # Expected Outcome:
 
-* In step2, a rule with given name will be added. And all images will be replicated to remote.   
+* In step8, a rule will be added.  
-* In step3, a rule will be added and enabled,no image replication job is started.  
-* In step4, a rule using new endpoint will be added.  
-* In step5, rule can not be added if use duplicate name or ip, 
-* In step5, if input wrong username/password/ip,rule can be added,but will cause test connection fail.  
-* In step6, rule can not be added.  
 
 # Possible Problems:
 None

tests/testcases/Group7-Replication/7-02-Proj-replication-rules-edit.md

@@ -1,9 +1,9 @@
-Test 7-02 - Project replication rules edit  
+Test 7-02 - Edit replication rules  
 =======
   
 # Purpose:  
   
-To verify project manager can edit repliciation rules.  
+To verify admin user can edit repliciation rules.  
   
 # References:  
 User guide  
@@ -11,18 +11,18 @@ User guide
 # Environment:  
 
 * This test requires that at least two Harbor instances are running and available.  
-* Need at least one project that has at least one replication rule.  
+* Need at least one replication rule has been created.  
   
 # Test Steps:  
   
-1. Login UI as admin user  
+1. Login UI as admin user.  
-2. In project replication page, choose a disabled rule and edit rule name, description, rule enable status and endpoint name, username, password.  
+2. In `Administration->Replications` page, choose a rule and edit its configurations.  
-3. In project replication page, choose an enabled rule and edit rule name, description, rule enable status and endpoint name, username, password.  
+
+Repeat steps 1-2 under `Projects->Project_Name->Replication` page.
 
 # Expect Outcome:  
   
-* In step 2, Rule can be edited and error hint works correctly,if rule is enabled, should see jobs start.  
+* In step 2, Rule can be edited.  
-* In step 3, Rule can not be edited.  
   
 # Possible Problems:  
 None  

tests/testcases/Group7-Replication/7-03-Proj-replication-rules-delete.md

@@ -1,4 +1,4 @@
-Test 7-05 Project replication rules delete
+Test 7-03 Delete replication rules  
 =======
 
 # Purpose:
@@ -14,15 +14,15 @@ User guide
 # Test Steps:
 
 1. Login as admin user.  
-2. In project replication page,delete an enabled rule.
+2. In `Administration->Replications` page,delete a rule which has no pending/running/retrying jobs.
-3. Disable an enabled rule with running job, delete the rule while there is unfinished job.  
+3. Delete a rule which has pending/running/retrying jobs.  
-4. In project replication page, delete a disabled rule.
+
+Repeat steps 1-3 under `Projects->Project_Name->Replication` page.
 
 # Expected Outcome:
 
-* In step2 rule can not be deleted.
+* In step2 rule can be deleted.
 * In step3 rule can not be deleted.  
-* In step4 rule can be deleted.
 
 # Possible Problems:
 None

tests/testcases/Group7-Replication/7-03-Proj-replication-rules-disable.md

@@ -1,25 +0,0 @@
-Test 7-03 Project replication rules disable
-=======
-
-# Purpose:
-
-To verify that an admin user can disable replication rules.  
-
-# References:
-User Guide
-
-# Environment:
-
-* This test requires that at least two Harbor instances are running and available.  
-* Need at least one project that has at least one enabled rule.
-
-# Test Steps:
-1. Login UI as admin user.  
-2. In project replication page, disable a rule.
-
-# Expected outcome:
-
-* Rule can be disabled. Replication jobs in queues will be canceled
-
-# Possible Problems:
-None

tests/testcases/Group7-Replication/7-04-Proj-replication-rules-enable.md

@@ -1,27 +0,0 @@
-Test 7-04 Project replication rules enable
-=======
-
-# Purpose:
-
-To verify an admin user can enable disabled rules.
-
-# References:
-User guide
-
-# Environment:
-
-* This test requires that at least two Harbor instance are running and avaliable.  
-* Need at least one project that has at least one disabled rule.
-
-# Test Steps:
-
-1. Login UI as admin user.  
-2. Push an image to project.  
-3. In project replication page, enable a disabled rule.
-
-# Expected Outcome:
-
-* In step3 rule become enabled, replication should start.   
-
-# Possible Problems:
-None

tests/testcases/Group7-Replication/7-04-Proj-replication-rules-filter.md

@@ -1,4 +1,4 @@
-Test 7-06 Project replication rules filter
+Test 7-04 Filter replication rules  
 =======
 
 # Purpose:
@@ -9,12 +9,14 @@ To verify rule filter works correctly.
 User guide
 
 # Environment:
-* This test requires that two Harbor instance are running and avaiable and there are at least 5 rules of a project.  
+* This test requires that two Harbor instance are running and avaiable.  
 
 # Test Steps:
 
-1. Login source registry ui as user.
+1. Login as admin user.
-2. In project replication page,input some character in rule filter and then clear the filter.
+2. In `Administration->Replications`,input some character in rule filter and then clear the filter.
+
+Repeat steps 1-2 under `Projects->Project_Name->Replication` page.
 
 # Expected Outcome:
 

tests/testcases/Group7-Replication/7-05-Trigger-manual.md

@@ -0,0 +1,29 @@
+Test 7-05 - Manual trigger  
+=======
+  
+# Purpose:  
+  
+To verify admin user can trigger a replication manually.  
+  
+# References:  
+User guide  
+  
+# Environment:  
+
+* This test requires that at least two Harbor instances are running and available.  
+* Create a new replication rule whose triggering condition is set to manual and no filter is configured.  
+* Need at least one image is pushed to the project that the rule is applied to
+  
+# Test Steps:  
+  
+1. Login UI as admin user.  
+2. In `Administration->Replications` page, choose the rule and click the `REPLICATE` button.  
+
+Repeat steps 1-2 under `Projects->Project_Name->Replication` page.
+
+# Expect Outcome:  
+  
+* In step 2, the operation should success and a few jobs should be started. All the images under the project should be replicated to the remote registry.  
+  
+# Possible Problems:  
+None  

tests/testcases/Group7-Replication/7-06-Trigger-immediate.md

@@ -0,0 +1,34 @@
+Test 7-06 - Immediate trigger  
+=======
+  
+# Purpose:  
+  
+To verify the immediate replication rule can work as expected. 
+  
+# References:  
+User guide  
+  
+# Environment:  
+
+* This test requires that at least two Harbor instances are running and available.  
+* Create a new replication rule whose triggering condition is set to immediate and no filter is configured.  
+  
+# Test Steps:  
+  
+1. Push an image to the project that the replication is applied to.  
+2. Login UI as admin user.  
+3. In `Administration->Replications` page, edit the rule and make sure the `Delete remote images when locally deleted` is checked.  
+4. Delete the image pushed in step 1 on UI.
+5. In `Administration->Replications` page, edit the rule and make sure the `Delete remote images when locally deleted` is unchecked.  
+6. Push the image to the project that the replication is applied to again.  
+7. Delete the image pushed in step 6 on UI.  
+
+# Expect Outcome:  
+  
+* In step 1, a job should be started and the image should be replicated to the remote registry.
+* In step 4, a job should be started and the image should be deleted from the remote registry.
+* In step 6, a job should be started and the image should be replicated to the remote registry.
+* In step 7, a job should be started and the image should not be deleted from the remote registry.  
+  
+# Possible Problems:  
+None  

tests/testcases/Group7-Replication/7-07-Proj-replication-jobs-filter.md

@@ -1,27 +0,0 @@
-Test 7-07 Project replication jobs filter
-=======
-
-# Purpose:
-
-To verify replication jobs filter works correctly.  
-
-# References:
-User guide
-
-# Environment:
-
-* This test requires at least two Harbor instance are running and available, and a project with a few jobs.
-
-# Test Steps:
-
-1. Login source registry as admin user.
-2. Create some project and create replication rules.
-3. Push some images into the project to start replication jobs and keep them running for step 4.  
-4. In project replication page, input some character in jobs log filter, and then clear the filter.  
-
-# Expected Outcome:
-
-* In step4, jobs can be filtered, and after clear filter, all jobs are shown again.   
-
-# Possible Problems:
-None

tests/testcases/Group7-Replication/7-07-Trigger-scheduled.md

@@ -0,0 +1,27 @@
+Test 7-07 - Scheduled trigger  
+=======
+  
+# Purpose:  
+  
+To verify the scheduled replication rule can work as expected.  
+  
+# References:  
+User guide  
+  
+# Environment:  
+
+* This test requires that at least two Harbor instances are running and available.  
+* Create a new replication rule whose triggering condition is set to scheduled and no filter is configured.  
+* Need at least one image is pushed to the project that the rule is applied to
+  
+# Test Steps:  
+  
+1. Login UI as admin user.  
+2. Go to `Administration->Replications` page.  
+
+# Expect Outcome:  
+  
+* In step 2, a few jobs should be started when the time configured in schedule trigger comes. All the images under the project should be replicated to the remote registry.  
+  
+# Possible Problems:  
+None  

tests/testcases/Group7-Replication/7-08-Proj-replication-jobs-filter.md

@@ -0,0 +1,28 @@
+Test 7-08 Project replication jobs filter
+=======
+
+# Purpose:
+
+To verify replication jobs filter works correctly.  
+
+# References:
+User guide
+
+# Environment:
+
+* This test requires at least two Harbor instance are running and available, and a project with a few jobs.
+
+# Test Steps:
+
+1. Login as admin user.
+2. In `Administration->Replications` page, click a rule which has replication jobs.
+3. Input some characters in jobs log filter, and then clear the filter.  
+
+Repeat steps 1-3 under `Projects->Project_Name->Replication` page.
+
+# Expected Outcome:
+
+* In step3, jobs can be filtered, and after clear filter, all jobs are shown again.   
+
+# Possible Problems:
+None

tests/testcases/Group7-Replication/7-09-Proj-replication-jobs-log-view.md

@@ -1,4 +1,4 @@
-Test 7-08 Project replication jobs log view
+Test 7-09 Project replication jobs log view
 =======
 
 # Purpose:
@@ -16,7 +16,9 @@ User guide
 # Test Steps:
 
 1. Login source registry UI as admin user.
-2. In project replication page, select a replication job and view job log.
+2. In `Administration->Replications` page, select a replication job and view job log.
+
+Repeat steps 1-2 under `Projects->Project_Name->Replication` page.
 
 # Expected Outcome:
 

tests/testcases/Group7-Replication/7-10-Filter.md

@@ -0,0 +1,28 @@
+Test 7-10 - Filter  
+=======
+  
+# Purpose:  
+  
+To verify the filer in replication rule can work as expected.  
+  
+# References:  
+User guide  
+  
+# Environment:  
+
+* This test requires that at least two Harbor instances are running and available.  
+* Create a new replication rule whose triggering condition is set to manual and tag filer is set to release-\*.  
+  
+# Test Steps:  
+  
+1. Push an image whose tag is release-1.0.  
+2. Push an image whose tag is dev-1.0.  
+3. Login UI with admin user.  
+4. Select the rule and click `REPLICATE` button.
+
+# Expect Outcome:  
+  
+* In step 4, one or two jobs should be started. The image in step 1 should be replicated to the remote registry and the image in step 2 should not be replicated to remote registry.  
+  
+# Possible Problems:  
+None  

tests/testcases/Group7-Replication/7-11-Endpoints-endpoints-add.md

@@ -1,27 +0,0 @@
-Test 7-11 Endpoints endpoints add
-=======
-
-# Purpose
-
-To verify admin user can add an endpoint
-
-# References:
-
-User guide
-
-# Environments:
-
-* This test requires at least two Harbor instance is running and available.
-
-# Test Steps:
-
-1. Login UI as admin user.
-2. In replication page, add an endpoint with valid ip/hostname/username/password,click test connection to test connection.  
-3. In replication page, add an endpoint with invalid ip/hostname/username/password, click test connection to test connection.  
-
-# Expected Outcome:
-
-* In step2 if endpoint is alive,test connection will successful, endpoint will add success.
-* In step3 test connection will fail, endpoint will add success.  
-# Possible Problem:
-None

tests/testcases/Group7-Replication/7-11-Project-admin-readonly.md

@@ -0,0 +1,29 @@
+Test 7-11 Project admin read-only privilege
+=======
+
+# Purpose
+
+To verify project admin user has read-only privilege for replication rules and jobs.
+
+# References:
+
+User guide
+
+# Environment:
+
+* This test requires at least two Harbor instance are running and available.  
+* At least a replication rule is created.  
+* The replication rule has at least one job.  
+* A member has been added to the project that the rule is applied to as admin.  
+
+# Test Steps:
+
+1. Login UI using the project admin user.  
+2. Go to `Projects->Project_Name->Replication` page.  
+
+# Expected Outcome:
+ 
+* In step2, the user should have read-only privilege to all rules and jobs.  
+
+# Possible Problems:
+None

tests/testcases/Group7-Replication/7-12-Endpoints-endpoints-add.md

@@ -0,0 +1,31 @@
+Test 7-12 Endpoints endpoints add
+=======
+
+# Purpose
+
+To verify admin user can add an endpoint
+
+# References:
+
+User guide
+
+# Environments:
+
+* This test requires at least two Harbor instance is running and available.
+
+# Test Steps:
+
+1. Login UI as admin user.
+2. In `Administration->Registries` page, add an endpoint with valid URL(HTTP), username and password, click test connection to test connection and save the endpoint.  
+3. In `Administration->Registries` page, add an endpoint with invalid URL(HTTP), username or password, click test connection to test connection and save the endpoint.  
+4. In `Administration->Registries` page, add an endpoint with valid URL(HTTPS with self-signed certificate), username and password, select the `Verify Remote Cert` checkbox and click test connection to test connection and save the endpoint.  
+5. In `Administration->Registries` page, add an endpoint with valid URL(HTTPS with self-signed certificate), username and password, uncheck the `Verify Remote Cert` checkbox and click test connection to test connection and save the endpoint.  
+
+# Expected Outcome:
+
+* In step2 if endpoint is alive,test connection will successful, endpoint will add success.
+* In step3 test connection will fail, endpoint will add success.  
+* In step4 test connection will fail, endpoint will add success.
+* In step5 if endpoint is alive,test connection will successful, endpoint will add success.
+# Possible Problem:
+None

tests/testcases/Group7-Replication/7-13-Endpoints-endpoints-edit.md

@@ -1,4 +1,4 @@
-Test 7-12 Endpoints endpoints edit
+Test 7-13 Endpoints endpoints edit
 =======
 
 # Purpose
@@ -16,13 +16,11 @@ User guide
 # Test Steps:
 
 1. Login UI as admin user.  
-2. In replication page, choose an endpoint in use by a rule and edit setting.  
+2. In `Administration->Registries` page, choose an endpoint and edit setting.  
-3. In replication page, choose an endpoint not in use by a rule and edit setting.
 
 # Expected Outcome:
  
-* In step2, endpoint info can not be edited.  
+* In step2, endpoint info can be edited.  
-* In step3, endpoint info can be edited.  
 
 # Possible Problems:
 None

tests/testcases/Group7-Replication/7-14-Endpoints-endpoints-delete.md

@@ -1,4 +1,4 @@
-Test 7-13 Endpoints endpoint delete
+Test 7-14 Endpoints endpoint delete
 =======
 
 # Purpose
@@ -16,8 +16,8 @@ User guide
 # Test Steps:
 
 1. Login UI as admin user.  
-2. In replication page, delete an endpoint in use by a rule.  
+2. In `Administration->Registries` page, delete an endpoint in use by a rule.  
-3. In replication page, delete an endpoint not in use by a rule.  
+3. In `Administration->Registries` page, delete an endpoint not in use by a rule.  
 
 # Expected Outcome:
 

tests/testcases/Group7-Replication/7-15-Endpoints-endpoints-filter.md

@@ -1,4 +1,4 @@
-Test 7-14 Endpoints endpoints filter
+Test 7-15 Endpoints endpoints filter
 =======
 
 # Purpose:
@@ -15,7 +15,7 @@ User guide
 # Test Steps:
 
 1. Login UI as admin user.    
-2. In replication page,add some endpoints.    
+2. In `Administration->Registries`,add some endpoints.    
 3. Input some characters in endpoints filter and then clear the filter.  
 
 # Expected Outcome:

tests/testcases/Group7-Replication/7-15-Rules-rule-edit.md

@@ -1,29 +0,0 @@
-Test 7-15 Rules: rule edit
-=======
-
-# Purpose:
-
-To verify admin user can edit replication rules.
-
-# References:
-User guide
-
-# Environment:
-
-* This test requires at least two Harbor instances are running and available.
-* Project has at least one enabled and one disabled replication rule.  
-
-# Test Steps:
-
-1. Login UI as admin user.  
-2. In replication page,edit name, description, enable status, endpoint info of enabled replication rules.
-3. Push an image to the project.  
-4. In replication page,edit name, description, enable status, endpoint info of disabled replication rules.
-
-# Expected Outcome:
-
-* In step2, rules can not be edited.  
-* In step4, rules can be edited. If enable rule, replication job will start.   
-
-# Possible Problems:
-None

tests/testcases/Group7-Replication/7-16-Rules-rule-disable.md

@@ -1,26 +0,0 @@
-Test 7-16 Rules rule disable
-=======
-
-# Purpose:
-
-To verify admin user can disable replication rles.
-
-# References:
-User guide
-
-# Environment:
-
-* This test requires at least two Harbor instances are running and available.
-* Need at least one project that has at least one enabled rule.  
-
-# Test Steps:
-
-1. Login UI as admin user.  
-2. In Replication rule page, disable an enabled replication rule.
-
-# Expected Outcome:
-
-* Rule can be disabled and replication jobs in queues will be canceled. Jobs are running will be stopped.  
-
-# Possible Problems:
-None

tests/testcases/Group7-Replication/7-17-Rules-rule-enable.md

@@ -1,27 +0,0 @@
-Test 7-17 Rules rule enable
-=======
-
-# Purpose:
-
-To verify admin user can enable disabled rules.
-
-# References:
-User guide
-
-# Environment:
-
-* This test requires two Harbor instances are running and available.
-* Projects has at least one disabled replication rule or more.  
-
-# Test Steps:
-
-1. Login UI as admin user.  
-2. Push an image to a project has at least one disabled rule.  
-3. In replication page, choose a disabled rule,enable it.
-
-# Expected Outcome:
-
-* Disabled rule can be enabled,after enable the rule, replication job will start.  
-
-# Possible Problems:
-None

tests/testcases/Group7-Replication/7-18-Rules-rule-delete.md

@@ -1,31 +0,0 @@
-Test 7-18 Rules rule delete
-=======
-
-# Purpose:
-
-To verify admin user can delete replication rules.
-
-# References:
-User guide
-
-# Environment:
-
-* This test requires at least two Harbor instances are running and available.
-* Source registry has at least one endpoint.  
-* Projects has at least one replication rule or more.
-
-# Test Steps:
-
-1. Login UI as admin user.  
-2. In replication page, delete an enabled rule.  
-3. Disable an enabled rule with running job, delete the rule while there is unfinished job.  
-4. In replication page, delete a disabled rule.  
-
-# Expected Outcome:
-
-* In step2, rule can not be deleted.
-* In step3, rule can not be deleted.
-* In step4, rule can be deleted.  
-
-# Possible Problems:
-None

tests/testcases/Group7-Replication/7-19-Rules-rule-filter.md

@@ -1,26 +0,0 @@
-Test 7-19 Rules rule filter
-=======
-
-# Purpose:
-
-To verify admin user can filter replication rules.
-
-# References:
-User guide
-
-# Environment:
-
-* This test requires at least one Harbor instance is running and available.
-* Need at least one project with at least one replication rule.
-
-# Test Steps:
-
-1. Login UI as admin user.  
-2. In replication page,input some charactor in replication rules filter and then clear the filter.
-
-# Expected Outcome:
-
-* Rules can be filtered, after clearing the filter, all rules are shown again.  
-
-# Possible Problems:
-None

tests/testcases/Group7-Replication/7-21-Replication-push-images.md

@@ -1,32 +0,0 @@
-Test 7-21 - Replication-push-images  
-=======
-
-# Purpose:
-
-To verify pushed images can be replicated to remote.  
-
-# Reference:
-
-User guide
-
-# Environment:
-
-* This test requires that at least two Harbor instance are running and available.  
-* Need at least one project has at least one replication rule. 
-
-# Test Steps:  
-**NOTE:** In below test, Harbor instance should have at least one available endpoint.  
-
-1. Login UI as admin user;  
-2. Create a project and add an enabled replication rule.  
-3. Push an image to the created project.  
-4. Check replication job.  
-5. Check remote registry to see if the image has been replicated if project existing on remote registry.  
-6. Check remote registry to see if the image has been replicated if project not existing on remote registry.  
-
-# Expect Outcome:  
-
-* On remote registry can see duplicated projects and images. 
-
-# Possible Problems:
-None

tests/testcases/Group7-Replication/7-22-Replication-del-images.md

@@ -1,35 +0,0 @@
-Test 7-22 - Replication-delete-images  
-=======
-
-# Purpose:
-
-To verify remote images can be delete when it is deleted.  
-
-# Reference:
-
-User guide  
-
-# Environment:
-
-* This test requires that at least two Harbor instance are running and available.
-* Need at least one project has at least one replication rule.
-
-# Test Steps:  
-**NOTE:** In below test, Harbor instance should have at least one available endpoint.  
-
-1. Login source registry UI as admin user.  
-2. Create a project and add an enabled replication rule.
-3. Push at least one image to the created project, and wait until replication job is done.  
-4. Check the job log.  
-5. Delete a pushed image in UI from the source registry.  
-6. Check the replication job.  
-7. Check remote registry to see if the image has been deleted.  
-
-# Expeced Outcome:
-
-* In step3, the remote will see replicated project and images.  
-* In step6, there will be a replication job and the deleted image will be deleted from remote too.  
-* In step7, images on remote registry should be deleted.  
-
-# Possible Problems:
-None

tests/testcases/Group7-Replication/7-23-Replication-verify-remote-cert.md

@@ -1,31 +0,0 @@
-Test 7-23- Replication verify remote cert  
-=======
-
-# Purpose:  
-To verify that if the verify remote cert setting works.  
-
-# Reference:
-
-User guide  
-
-# Environment:
-
-* This test requires at least two Harbor instances run and available.  
-* The remote endpoint instance should be configured to use https and the remote harbor instance is set up with self-signed certificate.  
-
-# Test Steps:  
-
-1. Login source registry as admin user.  
-2. In configuration page, make sure replication verify remote cert is checked on(by default).  
-3. In replication page, add an https endpoint use self-signed cert. Click test connection button before save.  
-4. In configuraiton page, uncheck replication verify remote cert checkbox and save.  
-5. In replication page, choose the added https endpoint, and click test connection.  
-
-# Expected Outcome:
-
-* In step3, test will fail with a certificate error.  
-* In step5, the test will successful.  
-
-# Possible Problems:
-
-None