2019-08-01 09:10:54 +02:00
// Copyright (c) 2017 VMware, Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
*** Settings ***
Documentation Harbor BATs
Resource ../../resources/Util.robot
Default Tags Nightly
*** Variables ***
${HARBOR_URL} https://${ip}
${SSH_USER} root
${HARBOR_ADMIN} admin
*** Test Cases ***
2019-11-07 05:53:24 +01:00
Test Case - Clair Is Default Scanner And It Is Immutable
2019-08-01 09:10:54 +02:00
Init Chrome Driver
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
2019-11-07 05:53:24 +01:00
Switch To Scanners Page
Should Display The Default Clair Scanner
Clair Is Immutable Scanner
2019-08-01 09:10:54 +02:00
Test Case - Disable Scan Schedule
Init Chrome Driver
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Vulnerability Page
Disable Scan Schedule
Logout Harbor
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Vulnerability Page
Retry Wait Until Page Contains None
Close Browser
Test Case - Scan A Tag In The Repo
2020-04-13 09:29:18 +02:00
Body Of Scan A Tag In The Repo hello-world latest
2019-08-01 09:10:54 +02:00
Test Case - Scan As An Unprivileged User
Init Chrome Driver
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} library hello-world
Sign In Harbor ${HARBOR_URL} user024 Test1@34
Go Into Project library
Go Into Repo hello-world
Select Object latest
Scan Is Disabled
Close Browser
Test Case - Scan Image With Empty Vul
Init Chrome Driver
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} library busybox
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Go Into Project library
Go Into Repo busybox
Scan Repo latest Succeed
Move To Summary Chart
2019-11-07 05:53:24 +01:00
Wait Until Page Contains No vulnerability
2019-08-01 09:10:54 +02:00
Close Browser
Test Case - Manual Scan All
Init Chrome Driver
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} library redis
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Switch To Vulnerability Page
2019-11-12 11:47:15 +01:00
Trigger Scan Now And Wait Until The Result Appears
2019-08-01 09:10:54 +02:00
Navigate To Projects
Go Into Project library
Go Into Repo redis
Summary Chart Should Display latest
Close Browser
Test Case - View Scan Error
Init Chrome Driver
${d}= get current date result_format=%m%s
Sign In Harbor ${HARBOR_URL} user026 Test1@34
Create An New Project project${d}
Push Image ${ip} user026 Test1@34 project${d} vmware/photon:1.0
Go Into Project project${d}
Go Into Repo project${d}/vmware/photon
Scan Repo 1.0 Fail
View Scan Error Log
Close Browser
Test Case - Scan Image On Push
[Tags] run-once
Init Chrome Driver
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} library hello-world
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Go Into Project library
Goto Project Config
Enable Scan On Push
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} library memcached
Navigate To Projects
Go Into Project library
Go Into Repo memcached
Summary Chart Should Display latest
Close Browser
Test Case - View Scan Results
[Tags] run-once
Init Chrome Driver
${d}= get current date result_format=%m%s
Sign In Harbor ${HARBOR_URL} user025 Test1@34
Create An New Project project${d}
Push Image ${ip} user025 Test1@34 project${d} tomcat
Go Into Project project${d}
Go Into Repo project${d}/tomcat
Scan Repo latest Succeed
Summary Chart Should Display latest
View Repo Scan Details
Close Browser
Test Case - Project Level Image Serverity Policy
[Tags] run-once
Init Chrome Driver
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
${d}= get current date result_format=%m%s
2020-01-14 04:11:12 +01:00
#For docker-hub registry
#${sha256}= Set Variable 9755880356c4ced4ff7745bafe620f0b63dd17747caedba72504ef7bac882089
#For internal CPE harbor registry
${sha256}= Set Variable 0e67625224c1da47cb3270e7a861a83e332f708d3d89dde0cbed432c94824d9a
2019-11-15 09:30:08 +01:00
${image}= Set Variable redis
2019-08-01 09:10:54 +02:00
Create An New Project project${d}
2019-09-12 07:56:56 +02:00
Push Image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} ${image} sha256=${sha256}
2019-08-01 09:10:54 +02:00
Go Into Project project${d}
Go Into Repo ${image}
Scan Repo ${sha256} Succeed
Navigate To Projects
Go Into Project project${d}
2019-11-15 09:30:08 +01:00
Set Vulnerabilty Serverity 3
2019-08-01 09:10:54 +02:00
Cannot pull image ${ip} ${HARBOR_ADMIN} ${HARBOR_PASSWORD} project${d} ${image} tag=${sha256}
Close Browser
2019-08-21 08:12:12 +02:00
#Important Note: All CVE IDs in CVE Whitelist cases must unique!
Test Case - Verfiy System Level CVE Whitelist
2020-04-02 07:57:40 +02:00
Body Of Verfiy System Level CVE Whitelist mariadb b5e273ed46d2b5a1c96bf8f3ae37aa5e90c6c481e7f7ae66744610d7df79cbd1 CVE-2019-13050\nCVE-2018-19591\nCVE-2018-11236\nCVE-2018-11237\nCVE-2019-13627\nCVE-2018-20839\nCVE-2019-2923\nCVE-2019-2922\nCVE-2019-2911\nCVE-2019-2914\nCVE-2019-2924\nCVE-2019-2910\nCVE-2019-2938\nCVE-2019-2993\nCVE-2019-2974\nCVE-2019-2960\nCVE-2019-2948\nCVE-2019-2946 CVE-2019-2969
2019-08-21 08:12:12 +02:00
Test Case - Verfiy Project Level CVE Whitelist
2020-04-02 07:57:40 +02:00
Body Of Verfiy Project Level CVE Whitelist mariadb b5e273ed46d2b5a1c96bf8f3ae37aa5e90c6c481e7f7ae66744610d7df79cbd1 CVE-2019-13050\nCVE-2018-19591\nCVE-2018-11236\nCVE-2018-11237\nCVE-2019-13627\nCVE-2018-20839\nCVE-2019-2923\nCVE-2019-2922\nCVE-2019-2911\nCVE-2019-2914\nCVE-2019-2924\nCVE-2019-2910\nCVE-2019-2938\nCVE-2019-2993\nCVE-2019-2974\nCVE-2019-2960\nCVE-2019-2948\nCVE-2019-2946 CVE-2019-2969
2019-08-21 08:12:12 +02:00
Test Case - Verfiy Project Level CVE Whitelist By Quick Way of Add System
2020-04-02 07:57:40 +02:00
Body Of Verfiy Project Level CVE Whitelist By Quick Way of Add System mariadb b5e273ed46d2b5a1c96bf8f3ae37aa5e90c6c481e7f7ae66744610d7df79cbd1 CVE-2019-13050\nCVE-2018-19591\nCVE-2018-11236\nCVE-2018-11237\nCVE-2019-13627\nCVE-2018-20839\nCVE-2019-2923\nCVE-2019-2922\nCVE-2019-2911\nCVE-2019-2914\nCVE-2019-2924\nCVE-2019-2910\nCVE-2019-2938\nCVE-2019-2993\nCVE-2019-2974\nCVE-2019-2960\nCVE-2019-2948\nCVE-2019-2946\nCVE-2019-2969