add vulnerability case (#3206)

* Add vulnerability manual case

tests/testcases/Group10-Vulnerability/10-01-Cliar-can-not-scan-some-images.md

@@ -0,0 +1,29 @@
+Test 10-01  Clair can not scan some images 
+=======
+  
+# Purpose:    
+To verify that clair can not scan some image.  
+  
+# References:    
+User guide  
+  
+# Environment:  
+* This test requires that a Harbor instance is running and available.  
+* Harbor is installed with clair enable.  
+* A linux host with Docker CLI installed.  
+* Clair has been updated to the latest.    
+  
+# Test Steps:  
+1. Login Harbor as admin.  
+2. Create a project and push an image clair does not recognize(for example: photon).  
+3. Scan the pushed image.  
+4. View the scan result.  
+5. Rescan it.  
+
+# Expected Outcome:  
+* Step3 should show view log link.  
+* Step4 the scan log should be can not scan.  
+* Step5 should be the same as step3.  
+  
+# Possible Problems:  
+None  

tests/testcases/Group10-Vulnerability/10-02-Clair-scan-a-image-with-vulnerability.md

@@ -0,0 +1,30 @@
+Test 10-02  Clair scan a image with vulnerability
+=======
+  
+# Purpose:  
+To verify that clair can scan a image with vulnerability.  
+
+# References:  
+User guide  
+  
+# Environment:  
+* This test requires that a Harbor instance is running and available.  
+* Harbor is installed with clair enable.  
+* A linux host with Docker CLI installed.    
+* Clair has been updated to the latest.  
+
+# Test Step  
+1. Login Harbor as admin.  
+2. Push an image with vulnerability.  
+3. Scan the image.  
+4. Click tag link enter detail page.  
+5. Rescan the image.  
+
+# Expected Outcome:  
+* Step3 should exist result summary chart.  
+* Step4 should see cve link and detail is match with summary in step3.  
+* Step4 cve link should clickable.  
+* Step5 rescan should successful.
+
+# Possible Problems:  
+None  

tests/testcases/Group10-Vulnerability/10-03-Clair-scan-0-package-image.md

@@ -0,0 +1,26 @@
+Test 10-03 Clair scan 0 package image
+=======
+  
+# Purpose:  
+To verify clair scan result of a 0 package image.  
+  
+# References:  
+User guide  
+  
+# Environment:  
+* This test requires that a Harbor instance is running and available.  
+* Harbor is installed with clair enable.  
+* A linux host with Docker CLI installed.  
+* Clair has been updated to the latest.    
+
+# Test Step  
+1. Login Harbor as admin.  
+2. Create a project.  
+3. Push a helloworld or busybox image to the created project.  
+4. Scan the image.  
+  
+# Expected Outcome:  
+* Step 4 result should be Severity:Unknown.  
+  
+# Possible Problems:  
+None  

tests/testcases/Group10-Vulnerability/10-04-Clair-data-not-ready-hint.md

@@ -0,0 +1,29 @@
+Test 10-04 Clair data not ready hint
+=======
+  
+# Purpose:  
+To verify there will be a hint if vulnerability data is not ready.  
+  
+# References:  
+User guide  
+  
+# Environment:  
+* This test requires that a Harbor instance is running and available.  
+* Harbor is installed with clair enable.  
+* A linux host with Docker CLI installed.  
+* Limit the Harbor's bandwith to less than 1Mbps after Harbor is installed.  
+
+# Test Step:  
+**NOTE:This test need to be done as soon as possible after Harbor is installed**  
+
+1. Login Harbor as admin.  
+2. Go into project page.  
+3. Go to configuration.  
+4. Click vulnerability.  
+
+# Expected Outcome:  
+* Step2 should see Vulnerability database might not be fully ready hint.  
+* Step4 should see a warning symbol with Vulnerability database might not be fully ready hint.  
+
+# Possible Problems:  
+None  

tests/testcases/Group10-Vulnerability/10-05-Schedule-daily-scan.md

@@ -0,0 +1,31 @@
+Test 10-05  Schedule daily scan
+=======
+  
+# Purpose:  
+To verify daily scan work correctly  
+# References:  
+User guide  
+  
+# Environment:  
+* This test requires that a Harbor instance is running and available.  
+* Harbor is installed with clair enable.  
+* A linux host with Docker CLI installed.  
+* Clair has been updated to the latest.  
+
+# Test Step  
+1. Login Harbor as admin.  
+2. Create at least 3 projects.  
+3. Push at least 2 images to each project.  
+4. Schedule a daily scan that time is before current time.  
+5. Wait one day.  
+6. Check the timestamp of scan.  
+7. Schedule a daily scan that time is after current time.  
+8. Wait until time is after the scheduled time.  
+9. Check the timestamp of scan.  
+  
+# Expected Outcome:    
+* Step6 the timestamp should be the same as scheduled.  
+* Step9 the timestamp should be the same as scheduled.  
+
+# Posssible Problems:  
+None  

tests/testcases/Group10-Vulnerability/10-06-Change-scan-configuration.md

@@ -0,0 +1,33 @@
+Test 10-06 Change scan configuration
+=======
+   
+# Purpose:   
+To verify scan configuration can be changed correctly.   
+   
+# References:   
+User guide   
+   
+# Environment:   
+* This test requires that a Harbor instance is running and available.   
+* Harbor is installed with clair enable.   
+* A linux host with Docker CLI installed.   
+   
+# Test Step   
+   
+1. Login harbor as admin.   
+2. Push a image to a project.   
+3. Click configuration.   
+4. Click vunlerablity.   
+5. Set scan all to none.   
+6. Click scan now.   
+7. Check scan now button.  
+8. Check the image scan time.   
+   
+# Expected Outcome:   
+* Step 4 if a new installed Harbor,vulnerability not ready hit should exist.  
+* Step 5 schedule time input should disappear.  
+* Step 7 after click scan now, scan now button should be disabled.
+* Step 8 scan timestamp should be the time when click scan now.  
+
+# Possible Problems:   
+None   

tests/testcases/Group10-Vulnerability/10-07-Non-admin-can-not-scan.md

@@ -0,0 +1,26 @@
+Test 10-07 non-admin can not scan
+=======
+
+# Purpose:  
+To verify a non-admin user can not execute scan action.  
+# References:  
+User guide  
+  
+# Environment:  
+* This test requires that a Harbor instance is running and available.  
+* Harbor is installed with clair enable.  
+* A linux host with Docker CLI installed.  
+  
+# Test Step  
+  
+1. Login harbor as non admin user.  
+2. Create a project.  
+3. Push an image.  
+4. Go to project and expand repo.  
+5. Click tag menu.  
+  
+# Expected Outcome:  
+* Step 5 user can not see scan button.  
+  
+# Possible Problems:  
+None  

tests/testcases/Group10-Vulnerability/10-08-User-can-not-pull-image-exceed-vulnerability-severity-setting.md

@@ -0,0 +1,26 @@
+10-08  user can not pull image exceed vulnerability severity
+=======
+# Purpose:  
+To verify user can not pull image exceed vulnerability severity setting. 
+
+# Reference:
+User guide.  
+
+# Environment:
+* This test requires that one Harbor instance is running and availiable.  
+* Harbor is installed with clair enable.  
+* A Linux host with Docker client installed.  
+* Clair has been updated to the latest.  
+
+# Test Step:
+1. Login harbor as admin.  
+2. Go to configuration.  
+3. Set vulnerability severity limit to medium and save configuration.  
+4. Push some images with vulnerability and scan them.  
+5. On a docker client, user pull an image with high vulneability severity. 
+
+# Expect outcome:
+* Step5 pull request should be refused.  
+
+# Possible problem:
+None

tests/testcases/Group10-Vulnerability/10-09-User-can-pull-scaned-images.md

@@ -0,0 +1,29 @@
+10-09  
+=======
+# Purpose:
+To test normal user can pull scaned images.  
+
+# Reference:
+User guide.  
+
+# Environment:
+* This test requires that one Harbor instance is running and availiable.  
+* Harbor is installed with clair enable.  
+* A Linux host with Docker client installed.  
+* Clair has been updated to the latest.  
+* Vulnerability severity limit is disabled.  
+
+# Test Step:
+**Note**:  
+In below test, user A is a non-admin user. User A should be replaced by meaningful names.  
+
+1. Login as user A. 
+2. Create a project and push an image. 
+3. Login as admin and scan the image.  
+4. On a Docker client, user A pull the scaned image.  
+
+# Expect Outcome:
+* Step 4 user can pull the image.  
+
+# Possible Problems:
+None

tests/testcases/Group10-Vulnerability/10-10-Manual-scan-all.md

@@ -0,0 +1,27 @@
+10-10 Manual scan all
+=======
+# Purpose:
+To verify scan all button works correctly.  
+
+# Reference:
+User guide.  
+
+# Environment:
+* This test requires that one Harbor instance is running and availiable.  
+* Harbor is installed with clair enable.  
+* A Linux host with Docker client installed.  
+* Clair has been updated to the latest.  
+
+# Test Step:
+1. Login Harbor as admin.  
+2. Create several projects.  
+3. Push some images to each project.  
+4. Goto configuration and click vulnerability.  
+5. Click scan all.  
+6. View scan result.  
+
+# Expect Outcome:
+* Step6 timestamp should be time when click scan all.  
+
+# Possible Problems:
+None

tests/testcases/Group10-Vulnerability/10-11-User-fix-vulnerability.md

@@ -0,0 +1,26 @@
+10-07  user fix vulnerability
+=======
+# Purpose:
+To test clair scan image vulnerablity correct after user fix it.  
+
+# Reference:
+User guide.  
+
+# Environment:
+* This test requires that one Harbor instance is running and availiable.  
+* Harbor is installed with clair enable.  
+* A Linux host with Docker client installed.  
+* Clair has been updated to the latest.  
+
+# Test Steps:
+1. Login Harbor as admin.  
+2. Push some images with vulnerability.  
+3. Scan an image with vulnerability and record the result.  
+4. Upgrade the image to fix vulnearbility.  
+5. Rescan the same image and check the result.  
+
+# Expect Outcome:
+* Step5 number of vulnerability should less than step3.  
+
+# Possible Problem:
+None