Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-11-23 02:35:17 +01:00
Code Issues Projects Releases Wiki Activity

Merge pull request #4108 from ywk253100/180123_manual_case

Browse Source 
Update manual test cases for replication
This commit is contained in:
Wenkai Yin 2018-01-24 18:22:42 +08:00 committed by GitHub
commit 39bccc6bf2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
27 changed files with 249 additions and 391 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
tests/testcases/Group7-Replication/7-01-Proj-replication-rules-add.md
View File

@ -16,26 +16,19 @@ User guide
# Test Steps:
1. Login UI as admin user.
2. In Project replication page, add a replication rule using an existing endpoint with enable checked.
3. In Project replication page, add a replication rule using an existing endpoint without enable checked.
4. In Project replication page, add a replication rule using a new endpoint.
5. In Project replication page, add a replication rule using a new endpoint. Provide invalid values of input to see if validation works:
* endpoint name or ip address duplicate with an existing endpoint.
* endpoint ip address incorrect.
* endpoint username or password incorrect.
6. Add another rule with different name using the same endpoint.
2. In `Administration->Replications` page, create a new rule and fill in name and description.
3. Choose a project by clicking the icon on the right of `Projects`.
4. Add repository and tag filter.
5. Choose an endpoint, create a new one if no endpoint exists.
6. Select a triggering condition: Immediate/Manaul/Scheduled, check/uncheck the `Delete remote images when locally deleted` if choosing `Immediate`.
7. Check/uncheck the option `Replicate existing images immediately`.
8. Save the rule.
Repeat steps 1-8 under `Projects->Project_Name->Replication` page.
# Expected Outcome:
* In step2, a rule with given name will be added. And all images will be replicated to remote.
* In step3, a rule will be added and enabled,no image replication job is started.
* In step4, a rule using new endpoint will be added.
* In step5, rule can not be added if use duplicate name or ip,
* In step5, if input wrong username/password/ip,rule can be added,but will cause test connection fail.
* In step6, rule can not be added.
* In step8, a rule will be added.
# Possible Problems:
None

18
tests/testcases/Group7-Replication/7-02-Proj-replication-rules-edit.md
View File

@ -1,9 +1,9 @@
Test 7-02 - Project replication rules edit
Test 7-02 - Edit replication rules
=======
# Purpose:
To verify project manager can edit repliciation rules.
To verify admin user can edit repliciation rules.
# References:
User guide
@ -11,18 +11,18 @@ User guide
# Environment:
* This test requires that at least two Harbor instances are running and available.
* Need at least one project that has at least one replication rule.
* Need at least one replication rule has been created.
# Test Steps:
1. Login UI as admin user
2. In project replication page, choose a disabled rule and edit rule name, description, rule enable status and endpoint name, username, password.
3. In project replication page, choose an enabled rule and edit rule name, description, rule enable status and endpoint name, username, password.
1. Login UI as admin user.
2. In `Administration->Replications` page, choose a rule and edit its configurations.
Repeat steps 1-2 under `Projects->Project_Name->Replication` page.
# Expect Outcome:
* In step 2, Rule can be edited and error hint works correctly,if rule is enabled, should see jobs start.
* In step 3, Rule can not be edited.
* In step 2, Rule can be edited.
# Possible Problems:
None

12
tests/testcases/Group7-Replication/7-05-Proj-replication-rules-delete.md → tests/testcases/Group7-Replication/7-03-Proj-replication-rules-delete.md
View File

@ -1,4 +1,4 @@
Test 7-05 Project replication rules delete
Test 7-03 Delete replication rules
=======
# Purpose:
@ -14,15 +14,15 @@ User guide
# Test Steps:
1. Login as admin user.
2. In project replication page,delete an enabled rule.
3. Disable an enabled rule with running job, delete the rule while there is unfinished job.
4. In project replication page, delete a disabled rule.
2. In `Administration->Replications` page,delete a rule which has no pending/running/retrying jobs.
3. Delete a rule which has pending/running/retrying jobs.
Repeat steps 1-3 under `Projects->Project_Name->Replication` page.
# Expected Outcome:
* In step2 rule can not be deleted.
* In step2 rule can be deleted.
* In step3 rule can not be deleted.
* In step4 rule can be deleted.
# Possible Problems:
None

25
tests/testcases/Group7-Replication/7-03-Proj-replication-rules-disable.md
View File

@ -1,25 +0,0 @@
Test 7-03 Project replication rules disable
=======
# Purpose:
To verify that an admin user can disable replication rules.
# References:
User Guide
# Environment:
* This test requires that at least two Harbor instances are running and available.
* Need at least one project that has at least one enabled rule.
# Test Steps:
1. Login UI as admin user.
2. In project replication page, disable a rule.
# Expected outcome:
* Rule can be disabled. Replication jobs in queues will be canceled
# Possible Problems:
None

27
tests/testcases/Group7-Replication/7-04-Proj-replication-rules-enable.md
View File

@ -1,27 +0,0 @@
Test 7-04 Project replication rules enable
=======
# Purpose:
To verify an admin user can enable disabled rules.
# References:
User guide
# Environment:
* This test requires that at least two Harbor instance are running and avaliable.
* Need at least one project that has at least one disabled rule.
# Test Steps:
1. Login UI as admin user.
2. Push an image to project.
3. In project replication page, enable a disabled rule.
# Expected Outcome:
* In step3 rule become enabled, replication should start.
# Possible Problems:
None

10
tests/testcases/Group7-Replication/7-06-Proj-replication-rules-filter.md → tests/testcases/Group7-Replication/7-04-Proj-replication-rules-filter.md
View File

@ -1,4 +1,4 @@
Test 7-06 Project replication rules filter
Test 7-04 Filter replication rules
=======
# Purpose:
@ -9,12 +9,14 @@ To verify rule filter works correctly.
User guide
# Environment:
* This test requires that two Harbor instance are running and avaiable and there are at least 5 rules of a project.
* This test requires that two Harbor instance are running and avaiable.
# Test Steps:
1. Login source registry ui as user.
2. In project replication page,input some character in rule filter and then clear the filter.
1. Login as admin user.
2. In `Administration->Replications`,input some character in rule filter and then clear the filter.
Repeat steps 1-2 under `Projects->Project_Name->Replication` page.
# Expected Outcome:

29
tests/testcases/Group7-Replication/7-05-Trigger-manual.md Normal file
View File

@ -0,0 +1,29 @@
Test 7-05 - Manual trigger
=======
# Purpose:
To verify admin user can trigger a replication manually.
# References:
User guide
# Environment:
* This test requires that at least two Harbor instances are running and available.
* Create a new replication rule whose triggering condition is set to manual and no filter is configured.
* Need at least one image is pushed to the project that the rule is applied to
# Test Steps:
1. Login UI as admin user.
2. In `Administration->Replications` page, choose the rule and click the `REPLICATE` button.
Repeat steps 1-2 under `Projects->Project_Name->Replication` page.
# Expect Outcome:
* In step 2, the operation should success and a few jobs should be started. All the images under the project should be replicated to the remote registry.
# Possible Problems:
None

34
tests/testcases/Group7-Replication/7-06-Trigger-immediate.md Normal file
View File

@ -0,0 +1,34 @@
Test 7-06 - Immediate trigger
=======
# Purpose:
To verify the immediate replication rule can work as expected.
# References:
User guide
# Environment:
* This test requires that at least two Harbor instances are running and available.
* Create a new replication rule whose triggering condition is set to immediate and no filter is configured.
# Test Steps:
1. Push an image to the project that the replication is applied to.
2. Login UI as admin user.
3. In `Administration->Replications` page, edit the rule and make sure the `Delete remote images when locally deleted` is checked.
4. Delete the image pushed in step 1 on UI.
5. In `Administration->Replications` page, edit the rule and make sure the `Delete remote images when locally deleted` is unchecked.
6. Push the image to the project that the replication is applied to again.
7. Delete the image pushed in step 6 on UI.
# Expect Outcome:
* In step 1, a job should be started and the image should be replicated to the remote registry.
* In step 4, a job should be started and the image should be deleted from the remote registry.
* In step 6, a job should be started and the image should be replicated to the remote registry.
* In step 7, a job should be started and the image should not be deleted from the remote registry.
# Possible Problems:
None

27
tests/testcases/Group7-Replication/7-07-Proj-replication-jobs-filter.md
View File

@ -1,27 +0,0 @@
Test 7-07 Project replication jobs filter
=======
# Purpose:
To verify replication jobs filter works correctly.
# References:
User guide
# Environment:
* This test requires at least two Harbor instance are running and available, and a project with a few jobs.
# Test Steps:
1. Login source registry as admin user.
2. Create some project and create replication rules.
3. Push some images into the project to start replication jobs and keep them running for step 4.
4. In project replication page, input some character in jobs log filter, and then clear the filter.
# Expected Outcome:
* In step4, jobs can be filtered, and after clear filter, all jobs are shown again.
# Possible Problems:
None

27
tests/testcases/Group7-Replication/7-07-Trigger-scheduled.md Normal file
View File

@ -0,0 +1,27 @@
Test 7-07 - Scheduled trigger
=======
# Purpose:
To verify the scheduled replication rule can work as expected.
# References:
User guide
# Environment:
* This test requires that at least two Harbor instances are running and available.
* Create a new replication rule whose triggering condition is set to scheduled and no filter is configured.
* Need at least one image is pushed to the project that the rule is applied to
# Test Steps:
1. Login UI as admin user.
2. Go to `Administration->Replications` page.
# Expect Outcome:
* In step 2, a few jobs should be started when the time configured in schedule trigger comes. All the images under the project should be replicated to the remote registry.
# Possible Problems:
None

28
tests/testcases/Group7-Replication/7-08-Proj-replication-jobs-filter.md Normal file
View File

@ -0,0 +1,28 @@
Test 7-08 Project replication jobs filter
=======
# Purpose:
To verify replication jobs filter works correctly.
# References:
User guide
# Environment:
* This test requires at least two Harbor instance are running and available, and a project with a few jobs.
# Test Steps:
1. Login as admin user.
2. In `Administration->Replications` page, click a rule which has replication jobs.
3. Input some characters in jobs log filter, and then clear the filter.
Repeat steps 1-3 under `Projects->Project_Name->Replication` page.
# Expected Outcome:
* In step3, jobs can be filtered, and after clear filter, all jobs are shown again.
# Possible Problems:
None

6
tests/testcases/Group7-Replication/7-08-Proj-replication-jobs-log-view.md → tests/testcases/Group7-Replication/7-09-Proj-replication-jobs-log-view.md
View File

@ -1,4 +1,4 @@
Test 7-08 Project replication jobs log view
Test 7-09 Project replication jobs log view
=======
# Purpose:
@ -16,7 +16,9 @@ User guide
# Test Steps:
1. Login source registry UI as admin user.
2. In project replication page, select a replication job and view job log.
2. In `Administration->Replications` page, select a replication job and view job log.
Repeat steps 1-2 under `Projects->Project_Name->Replication` page.
# Expected Outcome:

28
tests/testcases/Group7-Replication/7-10-Filter.md Normal file
View File

@ -0,0 +1,28 @@
Test 7-10 - Filter
=======
# Purpose:
To verify the filer in replication rule can work as expected.
# References:
User guide
# Environment:
* This test requires that at least two Harbor instances are running and available.
* Create a new replication rule whose triggering condition is set to manual and tag filer is set to release-\*.
# Test Steps:
1. Push an image whose tag is release-1.0.
2. Push an image whose tag is dev-1.0.
3. Login UI with admin user.
4. Select the rule and click `REPLICATE` button.
# Expect Outcome:
* In step 4, one or two jobs should be started. The image in step 1 should be replicated to the remote registry and the image in step 2 should not be replicated to remote registry.
# Possible Problems:
None

27
tests/testcases/Group7-Replication/7-11-Endpoints-endpoints-add.md
View File

@ -1,27 +0,0 @@
Test 7-11 Endpoints endpoints add
=======
# Purpose
To verify admin user can add an endpoint
# References:
User guide
# Environments:
* This test requires at least two Harbor instance is running and available.
# Test Steps:
1. Login UI as admin user.
2. In replication page, add an endpoint with valid ip/hostname/username/password,click test connection to test connection.
3. In replication page, add an endpoint with invalid ip/hostname/username/password, click test connection to test connection.
# Expected Outcome:
* In step2 if endpoint is alive,test connection will successful, endpoint will add success.
* In step3 test connection will fail, endpoint will add success.
# Possible Problem:
None

29
tests/testcases/Group7-Replication/7-11-Project-admin-readonly.md Normal file
View File

@ -0,0 +1,29 @@
Test 7-11 Project admin read-only privilege
=======
# Purpose
To verify project admin user has read-only privilege for replication rules and jobs.
# References:
User guide
# Environment:
* This test requires at least two Harbor instance are running and available.
* At least a replication rule is created.
* The replication rule has at least one job.
* A member has been added to the project that the rule is applied to as admin.
# Test Steps:
1. Login UI using the project admin user.
2. Go to `Projects->Project_Name->Replication` page.
# Expected Outcome:
* In step2, the user should have read-only privilege to all rules and jobs.
# Possible Problems:
None

31
tests/testcases/Group7-Replication/7-12-Endpoints-endpoints-add.md Normal file
View File

@ -0,0 +1,31 @@
Test 7-12 Endpoints endpoints add
=======
# Purpose
To verify admin user can add an endpoint
# References:
User guide
# Environments:
* This test requires at least two Harbor instance is running and available.
# Test Steps:
1. Login UI as admin user.
2. In `Administration->Registries` page, add an endpoint with valid URL(HTTP), username and password, click test connection to test connection and save the endpoint.
3. In `Administration->Registries` page, add an endpoint with invalid URL(HTTP), username or password, click test connection to test connection and save the endpoint.
4. In `Administration->Registries` page, add an endpoint with valid URL(HTTPS with self-signed certificate), username and password, select the `Verify Remote Cert` checkbox and click test connection to test connection and save the endpoint.
5. In `Administration->Registries` page, add an endpoint with valid URL(HTTPS with self-signed certificate), username and password, uncheck the `Verify Remote Cert` checkbox and click test connection to test connection and save the endpoint.
# Expected Outcome:
* In step2 if endpoint is alive,test connection will successful, endpoint will add success.
* In step3 test connection will fail, endpoint will add success.
* In step4 test connection will fail, endpoint will add success.
* In step5 if endpoint is alive,test connection will successful, endpoint will add success.
# Possible Problem:
None

10
tests/testcases/Group7-Replication/7-12-Endpoints-endpoints-edit.md → tests/testcases/Group7-Replication/7-13-Endpoints-endpoints-edit.md
View File

@ -1,4 +1,4 @@
Test 7-12 Endpoints endpoints edit
Test 7-13 Endpoints endpoints edit
=======
# Purpose
@ -16,13 +16,11 @@ User guide
# Test Steps:
1. Login UI as admin user.
2. In replication page, choose an endpoint in use by a rule and edit setting.
3. In replication page, choose an endpoint not in use by a rule and edit setting.
2. In `Administration->Registries` page, choose an endpoint and edit setting.
# Expected Outcome:
* In step2, endpoint info can not be edited.
* In step3, endpoint info can be edited.
* In step2, endpoint info can be edited.
# Possible Problems:
None

6
tests/testcases/Group7-Replication/7-13-Endpoints-endpoints-delete.md → tests/testcases/Group7-Replication/7-14-Endpoints-endpoints-delete.md
View File

@ -1,4 +1,4 @@
Test 7-13 Endpoints endpoint delete
Test 7-14 Endpoints endpoint delete
=======
# Purpose
@ -16,8 +16,8 @@ User guide
# Test Steps:
1. Login UI as admin user.
2. In replication page, delete an endpoint in use by a rule.
3. In replication page, delete an endpoint not in use by a rule.
2. In `Administration->Registries` page, delete an endpoint in use by a rule.
3. In `Administration->Registries` page, delete an endpoint not in use by a rule.
# Expected Outcome:

4
tests/testcases/Group7-Replication/7-14-Endpoints-endpoints-filter.md → tests/testcases/Group7-Replication/7-15-Endpoints-endpoints-filter.md
View File

@ -1,4 +1,4 @@
Test 7-14 Endpoints endpoints filter
Test 7-15 Endpoints endpoints filter
=======
# Purpose:
@ -15,7 +15,7 @@ User guide
# Test Steps:
1. Login UI as admin user.
2. In replication page,add some endpoints.
2. In `Administration->Registries`,add some endpoints.
3. Input some characters in endpoints filter and then clear the filter.
# Expected Outcome:

29
tests/testcases/Group7-Replication/7-15-Rules-rule-edit.md
View File

@ -1,29 +0,0 @@
Test 7-15 Rules: rule edit
=======
# Purpose:
To verify admin user can edit replication rules.
# References:
User guide
# Environment:
* This test requires at least two Harbor instances are running and available.
* Project has at least one enabled and one disabled replication rule.
# Test Steps:
1. Login UI as admin user.
2. In replication page,edit name, description, enable status, endpoint info of enabled replication rules.
3. Push an image to the project.
4. In replication page,edit name, description, enable status, endpoint info of disabled replication rules.
# Expected Outcome:
* In step2, rules can not be edited.
* In step4, rules can be edited. If enable rule, replication job will start.
# Possible Problems:
None

26
tests/testcases/Group7-Replication/7-16-Rules-rule-disable.md
View File

@ -1,26 +0,0 @@
Test 7-16 Rules rule disable
=======
# Purpose:
To verify admin user can disable replication rles.
# References:
User guide
# Environment:
* This test requires at least two Harbor instances are running and available.
* Need at least one project that has at least one enabled rule.
# Test Steps:
1. Login UI as admin user.
2. In Replication rule page, disable an enabled replication rule.
# Expected Outcome:
* Rule can be disabled and replication jobs in queues will be canceled. Jobs are running will be stopped.
# Possible Problems:
None

27
tests/testcases/Group7-Replication/7-17-Rules-rule-enable.md
View File

@ -1,27 +0,0 @@
Test 7-17 Rules rule enable
=======
# Purpose:
To verify admin user can enable disabled rules.
# References:
User guide
# Environment:
* This test requires two Harbor instances are running and available.
* Projects has at least one disabled replication rule or more.
# Test Steps:
1. Login UI as admin user.
2. Push an image to a project has at least one disabled rule.
3. In replication page, choose a disabled rule,enable it.
# Expected Outcome:
* Disabled rule can be enabled,after enable the rule, replication job will start.
# Possible Problems:
None

31
tests/testcases/Group7-Replication/7-18-Rules-rule-delete.md
View File

@ -1,31 +0,0 @@
Test 7-18 Rules rule delete
=======
# Purpose:
To verify admin user can delete replication rules.
# References:
User guide
# Environment:
* This test requires at least two Harbor instances are running and available.
* Source registry has at least one endpoint.
* Projects has at least one replication rule or more.
# Test Steps:
1. Login UI as admin user.
2. In replication page, delete an enabled rule.
3. Disable an enabled rule with running job, delete the rule while there is unfinished job.
4. In replication page, delete a disabled rule.
# Expected Outcome:
* In step2, rule can not be deleted.
* In step3, rule can not be deleted.
* In step4, rule can be deleted.
# Possible Problems:
None

26
tests/testcases/Group7-Replication/7-19-Rules-rule-filter.md
View File

@ -1,26 +0,0 @@
Test 7-19 Rules rule filter
=======
# Purpose:
To verify admin user can filter replication rules.
# References:
User guide
# Environment:
* This test requires at least one Harbor instance is running and available.
* Need at least one project with at least one replication rule.
# Test Steps:
1. Login UI as admin user.
2. In replication page,input some charactor in replication rules filter and then clear the filter.
# Expected Outcome:
* Rules can be filtered, after clearing the filter, all rules are shown again.
# Possible Problems:
None

32
tests/testcases/Group7-Replication/7-21-Replication-push-images.md
View File

@ -1,32 +0,0 @@
Test 7-21 - Replication-push-images
=======
# Purpose:
To verify pushed images can be replicated to remote.
# Reference:
User guide
# Environment:
* This test requires that at least two Harbor instance are running and available.
* Need at least one project has at least one replication rule.
# Test Steps:
**NOTE:** In below test, Harbor instance should have at least one available endpoint.
1. Login UI as admin user;
2. Create a project and add an enabled replication rule.
3. Push an image to the created project.
4. Check replication job.
5. Check remote registry to see if the image has been replicated if project existing on remote registry.
6. Check remote registry to see if the image has been replicated if project not existing on remote registry.
# Expect Outcome:
* On remote registry can see duplicated projects and images.
# Possible Problems:
None

35
tests/testcases/Group7-Replication/7-22-Replication-del-images.md
View File

@ -1,35 +0,0 @@
Test 7-22 - Replication-delete-images
=======
# Purpose:
To verify remote images can be delete when it is deleted.
# Reference:
User guide
# Environment:
* This test requires that at least two Harbor instance are running and available.
* Need at least one project has at least one replication rule.
# Test Steps:
**NOTE:** In below test, Harbor instance should have at least one available endpoint.
1. Login source registry UI as admin user.
2. Create a project and add an enabled replication rule.
3. Push at least one image to the created project, and wait until replication job is done.
4. Check the job log.
5. Delete a pushed image in UI from the source registry.
6. Check the replication job.
7. Check remote registry to see if the image has been deleted.
# Expeced Outcome:
* In step3, the remote will see replicated project and images.
* In step6, there will be a replication job and the deleted image will be deleted from remote too.
* In step7, images on remote registry should be deleted.
# Possible Problems:
None

31
tests/testcases/Group7-Replication/7-23-Replication-verify-remote-cert.md
View File

@ -1,31 +0,0 @@
Test 7-23- Replication verify remote cert
=======
# Purpose:
To verify that if the verify remote cert setting works.
# Reference:
User guide
# Environment:
* This test requires at least two Harbor instances run and available.
* The remote endpoint instance should be configured to use https and the remote harbor instance is set up with self-signed certificate.
# Test Steps:
1. Login source registry as admin user.
2. In configuration page, make sure replication verify remote cert is checked on(by default).
3. In replication page, add an https endpoint use self-signed cert. Click test connection button before save.
4. In configuraiton page, uncheck replication verify remote cert checkbox and save.
5. In replication page, choose the added https endpoint, and click test connection.
# Expected Outcome:
* In step3, test will fail with a certificate error.
* In step5, the test will successful.
# Possible Problems:
None