usecases

tests/testcases/Group1-user-management/1-14-LDAP-admin-role-general.md

@@ -0,0 +1,29 @@
+Test 1-14 - LDAP Mode Admin Role General Functions
+=======
+
+# Purpose:
+
+To verify that Harbor's UI of a user with system admin role works properly in LDAP mode.
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against an AD or LDAP server. (auth_mode is set to **ldap_auth** .) 
+* An Active Directory (AD) or LDAP server has been set up and it has a few users available for testing.
+
+# Test Steps:
+
+**NOTE:** The below non-admin user M should NOT be the same as the non-admin user in Test 1-07.
+
+1. Assign an non-admin user M with system admin role and act as an admin user. 
+2. Repeat all steps in Test 1-07.
+
+# Expected Outcome:
+
+* A user with system admin role can perform all operations the same as the admin user. 
+* Same as Test 1-07.
+
+# Possible Problems:
+None

tests/testcases/Group1-user-management/1-15-admin-role-create-delete-user.md

@@ -0,0 +1,29 @@
+Test 1-15 - Admin Role User Create, Delete and Recreate a User(DB Mode)
+=======
+
+# Purpose:
+
+To verify that an admin user can create/delete/recreate a user when users are managed locally by Harbor (DB mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against a local database. 
+* A linux host with Docker CLI installed (Docker client).
+
+# Test Steps:
+
+**NOTE:** The below non-admin user M should NOT be the same as the non-admin user in Test 1-09.
+
+1. Assign an non-admin user M with system admin role and act as an admin user. 
+2. Repeat all steps in Test 1-09.
+
+# Expected Outcome:
+
+* A user with system admin role can perform all operations the same as the admin user. 
+* Same as Test 1-09.
+
+# Possible Problems:
+None

tests/testcases/Group2-image-management/2-21-admin-view-project.md

@@ -12,7 +12,7 @@ User guide
 * This test requires that a Harbor instance is running and available.
 * Harbor is set to authenticate against a local database. ( auth_mode is set to **db_auth** .) The user data is stored in a local database.
 * A linux host with Docker CLI installed (Docker client).
-* At least a non-admin users is in Harbor. 
+* At least a non-admin user is in Harbor. 
 
 # Test Steps:
 

tests/testcases/Group2-image-management/2-23-admin-delete-images.md

@@ -1,4 +1,4 @@
-Test 2-22 - Admin User Delete Images (DB Mode)
+Test 2-23 - Admin User Delete Images (DB Mode)
 =======
 
 # Purpose:

tests/testcases/Group2-image-management/2-31-admin-role-view-project.md

@@ -0,0 +1,30 @@
+Test 2-31 - Admin Role View Projects (DB Mode)
+=======
+
+# Purpose:
+
+To verify that a user with system admin role can view all projects when users are managed locally by Harbor (DB mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against a local database. ( auth_mode is set to **db_auth** .) The user data is stored in a local database.
+* A linux host with Docker CLI installed (Docker client).
+* At least two non-admin users are in Harbor. 
+
+# Test Steps:
+
+**NOTE:** The below non-admin user M should NOT be the same as the non-admin user in Test 2-21.
+
+1. Assign an non-admin user M with system admin role and act as an admin user. 
+2. Repeat all steps in Test 2-21.
+
+# Expected Outcome:
+
+* A user with system admin role can perform all operations the same as the admin user. 
+* Same as Test 2-21.
+
+# Possible Problems:
+None

tests/testcases/Group2-image-management/2-32-admin-role-search-projects.md

@@ -0,0 +1,31 @@
+Test 2-32 - Admin Role User Search Projects (DB Mode)
+=======
+
+# Purpose:
+
+To verify that a user with system admin role can search projects when users are managed locally by Harbor (DB mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against a local database. ( auth_mode is set to **db_auth** .) The user data is stored in a local database.
+* A linux host with Docker CLI installed (Docker client).
+* At least two non-admin users are in Harbor. 
+
+# Test Steps:
+
+
+**NOTE:** The below non-admin user M should NOT be the same as the non-admin user in Test 2-22.
+
+1. Assign an non-admin user M with system admin role and act as an admin user. 
+2. Repeat all steps in Test 2-22.
+
+# Expected Outcome:
+
+* A user with system admin role can perform all operations the same as the admin user. 
+* Same as Test 2-22.
+
+# Possible Problems:
+None

tests/testcases/Group2-image-management/2-33-admin-role-delete-images.md

@@ -0,0 +1,30 @@
+Test 2-33 - Admin Role User Delete Images (DB Mode)
+=======
+
+# Purpose:
+
+To verify that a user with system admin role can delete images owned by other users when users are managed locally by Harbor (DB mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against a local database. ( auth_mode is set to **db_auth** .) The user data is stored in a local database.
+* A linux host with Docker CLI installed (Docker client).
+* At least tow non-admin users. 
+
+# Test Steps:
+
+**NOTE:** The below non-admin user M should NOT be the same as the non-admin user in Test 2-23.
+
+1. Assign an non-admin user M with system admin role and act as an admin user. 
+2. Repeat all steps in Test 2-23.
+
+# Expected Outcome:
+
+* A user with system admin role can perform all operations the same as the admin user. 
+* Same as Test 2-23.
+
+# Possible Problems:
+None

tests/testcases/Group2-image-management/2-34-admin-role-delete-projects.md

@@ -0,0 +1,30 @@
+Test 2-34 - Admin User Delete Projects (DB Mode)
+=======
+
+# Purpose:
+
+To verify that a user with system admin role can delete other user's projects when users are managed locally by Harbor (DB mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that two(2) Harbor instances are running and available.
+* Harbor is set to authenticate against a local database. ( auth_mode is set to **db_auth** .) The user data is stored in a local database.
+* A linux host with Docker CLI installed (Docker client).
+* At least two non-admin users. 
+
+# Test Steps:
+
+**NOTE:** The below non-admin user M should NOT be the same as the non-admin user in Test 2-24.
+
+1. Assign an non-admin user M with system admin role and act as an admin user. 
+2. Repeat all steps in Test 2-24.
+
+# Expected Outcome:
+
+* A user with system admin role can perform all operations the same as the admin user. 
+* Same as Test 2-24.
+
+# Possible Problems:
+None

tests/testcases/Group3-RBAC/3-21-DB-admin-role-user-manage-project-members.md

@@ -0,0 +1,31 @@
+Test 3-21 - Admin Role User Manages Project Members (DB Mode)
+=======
+
+# Purpose:
+
+To verify that a user with system admin role can add members of various roles to a project. Users are managed locally by Harbor (DB mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against a local database. ( auth_mode is set to **db_auth** .) The user data is stored in a local database.
+* A linux host with Docker CLI installed (Docker client).
+* At least three non-admin users are in Harbor. 
+* At least one project that admin user is not a member of.
+
+# Test Steps:
+
+**NOTE:** The below non-admin user M should NOT be the same as the non-admin user in Test 3-11.
+
+1. Assign an non-admin user M with system admin role and act as an admin user. 
+2. Repeat all steps in Test 3-11.
+
+# Expected Outcome:
+
+* A user with system admin role can perform all operations the same as the admin user. 
+* Same as Test 3-11.
+
+# Possible Problems:
+None

tests/testcases/Group3-RBAC/3-22-DB-admin-role-user-search-project-members.md

@@ -0,0 +1,32 @@
+Test 3-22 - Admin Role User Search Project Members (DB Mode)
+=======
+
+# Purpose:
+
+To verify that a user with system admin role can search members of a project when users are managed locally by Harbor (DB mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against a local database. ( auth_mode is set to **db_auth** .) The user data is stored in a local database.
+* A linux host with Docker CLI installed (Docker client).
+* At least five(5) non-admin users are in Harbor. 
+* At least 5 members in a project that admin user is not a member of.
+
+
+# Test Steps:
+
+**NOTE:** The below non-admin user M should NOT be the same as the non-admin user in Test 3-12.
+
+1. Assign an non-admin user M with system admin role and act as an admin user. 
+2. Repeat all steps in Test 3-12.
+
+# Expected Outcome:
+
+* A user with system admin role can perform all operations the same as the admin user. 
+* Same as Test 3-12.
+
+# Possible Problems:
+None

tests/testcases/Group4-logging/4-02-LDAP-user-view-logs.md

@@ -0,0 +1,39 @@
+Test 4-02 - User Views Logs (LDAP Mode)
+=======
+
+# Purpose:
+
+To verify that a non-admin user can views logs when users are managed externally by LDAP or AD (LDAP mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against an LDAP or AD server. ( auth_mode is set to **ldap_auth** .) The user data is stored in an LDAP or AD server.
+* A linux host with Docker CLI installed (Docker client).
+
+# Test Steps:
+1. On a Docker client host, use `docker login <harbor_host>` command to log in as a non-admin user. 
+2. Run some `docker push` and `docker pull` commands to push images to the registry and pull from the registry.
+3. Log in to the UI as the non-admin user.
+4. Delete a few images from the project. 
+5. View the logs of the project. 
+6. Try below search criteria to see if the search result is correct:
+
+* push only
+* pull only
+* pull and push
+* delete only
+* all
+* push and delete
+* different date ranges 
+* date range and push
+
+# Expected Outcome:
+* All operations in Step 2 & 4 should be logged.
+* Logs can be viewed in Step 5, check if the time and operations are correct.
+* Logs can be filtered in Step 6.
+
+# Possible Problems:
+None

tests/testcases/Group4-logging/4-03-DB-admin-view-logs.md

@@ -0,0 +1,40 @@
+Test 4-03 - Admin Views Logs (DB Mode)
+=======
+
+# Purpose:
+
+To verify that an admin user can views logs when users are managed locally by Harbor (DB mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against a local database. ( auth_mode is set to **db_auth** .) The user data is stored in a local database.
+* A linux host with Docker CLI installed (Docker client).
+
+# Test Steps:
+1. On a Docker client host, use `docker login <harbor_host>` command to log in as a non-admin user. 
+2. Run some `docker push` and `docker pull` commands to push images to the registry and pull from the registry.
+3. Log in to the UI as the non-admin user.
+4. Delete a few images from the project. 
+5. Log out non-admin user and log in to the UI as the admin user.
+6. View the logs of the project of the non-admin user. 
+7. Try below search criteria to see if the search result is correct:
+
+* push only
+* pull only
+* pull and push
+* delete only
+* all
+* push and delete
+* different date ranges 
+* date range and push
+
+# Expected Outcome:
+* All operations of non-admin users in Step 2 & 4 should be logged.
+* Logs can be viewed in Step 6, check if the time and operations are correct.
+* Logs can be filtered in Step 7.
+
+# Possible Problems:
+None

tests/testcases/Group4-logging/4-04-DB-admin-role-view-logs.md

@@ -0,0 +1,30 @@
+Test 4-04 - User with Admin Role Views Logs (DB Mode)
+=======
+
+# Purpose:
+
+To verify that a user with system admin role can views logs when users are managed locally by Harbor (DB mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against a local database. ( auth_mode is set to **db_auth** .) The user data is stored in a local database.
+* A linux host with Docker CLI installed (Docker client).
+
+# Test Steps:
+
+**NOTE:** The below non-admin user A should NOT be the same as the non-admin user in Test 4-03.
+
+1. Assign an non-admin user A with system admin role. 
+2. Repeat all steps in Test 4-03.
+
+
+# Expected Outcome:
+
+* A user with admin role can perform all operations the same as the admin user. 
+* Outcome should be the same as Test 4-03.
+
+# Possible Problems:
+None

tests/testcases/Group4-logging/4-05-LDAP-admin-role-view-logs.md

@@ -0,0 +1,30 @@
+Test 4-05 - User with Admin Role Views Logs (LDAP Mode)
+=======
+
+# Purpose:
+
+To verify that a user with system admin role can views logs when users are managed externally by LDAP or AD (LDAP mode).
+
+# References:
+User guide
+
+# Environment:
+* This test requires that a Harbor instance is running and available.
+* Harbor is set to authenticate against an LDAP or AD server. ( auth_mode is set to **ldap_auth** .) The user data is stored in an LDAP or AD server.
+* A linux host with Docker CLI installed (Docker client).
+
+# Test Steps:
+
+**NOTE:** The below non-admin user A should NOT be the same as the non-admin user in Test 4-03.
+
+1. Assign an non-admin user M with system admin role and act as an admin user. 
+2. Repeat all steps in Test 4-03.
+
+
+# Expected Outcome:
+
+* A user with system admin role can perform all operations the same as the admin user. 
+* Outcome should be the same as Test 4-03.
+
+# Possible Problems:
+None

tests/testcases/Group5-OVA-install-config/5-01-OVA-networking.md

@@ -0,0 +1,39 @@
+Test 5-01 - OVA Networking Settings During Deployment
+=======
+
+# Purpose:
+
+To verify that an OVA version of Harbor can be configured to obtain IP address from DHCP or static IP.
+
+# References:
+User guide, installation guide of Harbor OVA version.
+
+# Environment:
+* This test requires an OVA binary of Harbor.
+* A vCenter, at least an ESX host, and a network that supports DHCP.
+* A linux host with Docker CLI installed (Docker client).
+
+# Test Steps:
+1. From vSphere Web Client, import Harbor's OVA onto an ESX host.
+2. In the deployment wizard, enter different passwords of Linux root user, Harbor admin user and MySQL root user.
+3. Leave the networking settings blank. 
+4. Power on the imported OVA.
+5. Wait a few minutes for the VM's booting and its IP address comes up in vCenter. (may need to refresh in Web Client)
+6. Open a browser and enter http://VM_IP_address.
+7. Log in as admin user of Harbor.
+8. Create a new project. 
+9. On a Docker client host, use `docker login <harbor_host>` command to log in as the admin user. 
+10. Run some `docker push` and `docker pull` commands to push images to the project and pull from the project.
+11. On vSphere, open the console of Harbor's VM, log in as root user using the password entered during deployment.
+12. Check the network IP address of the VM.
+13. From vSphere Web Client, import a new Harbor's OVA onto an ESX host.
+14. Repeat Step 2-12, except in Step 3, enter static network settings of the OVA, such as static IP, DNS, gateway, hostname.
+
+
+# Expected Outcome:
+* In Step 1-11, everything should work without errors. The passwords entered during deployment should work in Step 7, 9 and 11.
+* In Step 14, basically is to test if static networking works for the OVA. The outcome should be the same as Step 1-12, except the networking settings are from the deployment wizard.
+Verify the IP address, hostname, DNS are correctly set inside the VM.
+
+# Possible Problems:
+None

tests/testcases/Group5-OVA-install-config/5-02-OVA-reboot.md

@@ -0,0 +1,43 @@
+Test 5-02 - OVA Reboot
+=======
+
+# Purpose:
+
+To verify that an OVA version of Harbor can be rebooted. Its configuration remains unchanged and should work the same way as before a reboot.
+
+# References:
+User guide, installation guide of Harbor OVA version.
+
+# Environment:
+* This test requires an OVA binary of Harbor.
+* A vCenter, at least an ESX host, and a network that supports DHCP.
+* A linux host with Docker CLI installed (Docker client).
+
+# Test Steps:
+
+1. From vSphere Web Client, import Harbor's OVA onto an ESX host.
+2. In the deployment wizard, enter different passwords of Linux root user, Harbor admin user and MySQL root user.
+3. Leave the networking settings blank. 
+4. Configure email settings. (can be mail server that does not exist )
+5. Power on the imported OVA.
+6. Wait a few minutes for the VM's booting and its IP address comes up in vCenter. (may need to refresh in Web Client)
+7. Open a browser and enter http://VM_IP_address.
+8. Log in as admin user of Harbor.
+9. Create a new project. 
+10. On a Docker client host, use `docker login <harbor_host>` command to log in as the admin user. 
+11. Run some `docker push` and `docker pull` commands to push images to the project and pull from the project.
+12. On vSphere, open the console of Harbor's VM, log in as root user using the password entered during deployment.
+13. On vCenter Web Client, reboot the VM. (soft reboot)
+14. After the VM starts up, repeat Step 7-12, should work the same.
+15. Power off the VM, and the power it on. (hard reboot)
+16. After the VM starts up, repeat Step 7-12, should work the same.
+17. On vSphere, open the console of Harbor's VM, log in as root user, type `ovfenv` command to verify environment variables are the same as those entered during deployment. 
+
+
+# Expected Outcome:
+* In Step 1-12, everything should work without errors. The passwords entered during deployment should work in Step 7, 9 and 11.
+* In Step 14, 16, the VM should work the same before the reboot or power-off.
+* Step 17, environment variables should remain unchanged as those entered during deployment.
+
+# Possible Problems:
+None

tests/testcases/Group5-OVA-install-config/5-03-OVA-garbage-collection.md

@@ -0,0 +1,40 @@
+Test 5-03 - OVA Garbage Collection
+=======
+
+# Purpose:
+
+To verify that an OVA version of Harbor can perform garbage collection to release unused space of deleted images.
+
+# References:
+User guide, installation guide of Harbor OVA version.
+
+# Environment:
+* This test requires an OVA binary of Harbor.
+* A vCenter, at least an ESX host, and a network that supports DHCP.
+* A linux host with Docker CLI installed (Docker client).
+
+# Test Steps:
+1. Deploy an OVA version of Harbor, with "Garbage Collection" set to false.
+2. Create a project in Harbor.
+3. On a Docker client host, use `docker login <harbor_host>` command to log in as the admin user. 
+4. Run some `docker push` to push some images to the project. The size of the images should be at least 500MB.
+5. In Harbor's UI, delete the newly pushed images.
+6. On vSphere, open the console of Harbor's VM, log in as root user, type `df -h /data` command to get the space usage of the /data volume. Take a note of the **Used** space.
+7. Power off the VM.
+8. Right click on the VM and select "Edit Settings".
+9. Set "Garbage Collection" to true.
+10. Power on the VM. 
+11. Wait for a while until Harbor service is available (check by a browser)
+12. On vSphere, open the console of Harbor's VM, log in as root user, type `df -h /data` command to get the space usage of the /data volume and compare with previous number.
+13. Check the log file of garbage collection under /data to see if there was any error.
+14. Repeat Step 3-7, to create some deleted images. 
+15. Repeat Step 10-13, to see the garbage collection works on the second reboot.
+
+
+# Expected Outcome:
+* Step 12, the used space should be reduced. The space of deleted images should be recycled.
+* Step 13, log file should contain no errors.
+* Step 15, verify garbage collection works for the second time.
+
+# Possible Problems:
+None

tests/testcases/Group5-OVA-install-config/5-04-OVA-HTTPS.md

@@ -0,0 +1,25 @@
+Test 5-04 - OVA Uses HTTPS
+=======
+
+# Purpose:
+
+To verify that an OVA version of Harbor can set up for HTTPS.
+
+# References:
+User guide, installation guide of Harbor OVA version.
+
+# Environment:
+* This test requires an OVA binary of Harbor.
+* A vCenter, at least an ESX host, and a network that supports DHCP.
+* A linux host with Docker CLI installed (Docker client).
+
+# Test Steps:
+
+TBD
+
+# Expected Outcome:
+
+TBD
+
+# Possible Problems:
+None

tests/testcases/Group5-OVA-install-config/5-05-OVA-LDAP-integration.md

@@ -0,0 +1,25 @@
+Test 5-05 - OVA LDAP integration
+=======
+
+# Purpose:
+
+To verify that an OVA version of Harbor can work with AD and LDAP. The LDAP/AD configuration can be updated after power-off/on.
+
+# References:
+User guide, installation guide of Harbor OVA version.
+
+# Environment:
+* This test requires an OVA binary of Harbor.
+* A vCenter, at least an ESX host, and a network that supports DHCP.
+* A linux host with Docker CLI installed (Docker client).
+
+# Test Steps:
+
+TBD
+
+# Expected Outcome:
+
+TBD
+
+# Possible Problems:
+None