Commit Graph

75 Commits

Author SHA1 Message Date
danfengliu
a8f159728b Add API E2E pytest of GC with untag flag enabled
1. Fix issue of keyword Go Into Repo, the verification logic could be more strict;
2. Add API E2E pytest of GC with untag flag enabled;
3. Add sleep in test_user_view_logs.py for delete log ocurred;
4. Test Case - Tag CRUD is not stable. Although add button was clicked, but the tag was'nt added successfully.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-23 14:48:04 +08:00
danfengliu
5d8a308158 Add retry for keyword go into index
1. Add retry for keyword go into index;
2. Add retry for keyword go into project.
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-05-27 08:55:58 +00:00
danfengliu
a07cc7bb2a
Merge pull request #11967 from danfengliu/fix-nightly-log-verification
Modify nightly quotas error message verification
2020-05-20 14:38:37 +08:00
danfengliu
b4a54adfeb Add Scan schedule test in nightly
1. Add Scan schedule test in nightly;
2. Remove redundancy steps when create a new project;

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-05-20 12:16:33 +08:00
danfengliu
2fb90cf258 Modify nightly quotas error message verification
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-05-19 14:59:21 +08:00
danfengliu
d10dfc8167 Fix issue of GC untag images case
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-30 16:10:03 +08:00
danfengliu
cd0463047b Add tag for ui test case of add nuew user
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-28 19:47:47 +08:00
AllForNothing
8ff4003438 Add new nightly case admin add new users
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-04-28 11:10:06 +08:00
AllForNothing
46320641b4 Add new nightly case for GC untagged images
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-04-14 09:25:06 +08:00
Will Sun
009662a317
Merge pull request #11426 from AllForNothing/nightly-tag
Add new nightly cases for tag CRUD
2020-04-13 12:15:18 +08:00
danfengliu
a869bba59f Modify quota error info due to product changed
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-10 21:06:05 +08:00
danfengliu
66dfd59c80 Add Multi-Scanner test file
1. Add Multi-Scanner test file for harbor is deployed with multi scanners;
2. Modify notary doc description;
3. Add paragraph for e2e-api-python-based-scripting-guide;
4. Fix delete project issue;
5. Remove count qoutas in nightly and modify some of it;
6. Add Trivy in git hub offline action;

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-08 16:26:31 +08:00
AllForNothing
e342b4ef0b Add new nightly cases for tag CRUD
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-04-08 09:56:27 +08:00
danfengliu
aa26e624b9
Merge pull request #11379 from danfengliu/add-nightly-test-of-push-helm-chart
Add nightly test of push helm chart
2020-04-02 10:29:21 +08:00
danfengliu
e3b31bb8dd Add nightly test of push helm chart
1. Add push helm chart oci test in nightly;
2. Modify archive icon locator xpath;

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-01 18:53:19 -07:00
AllForNothing
e6e3f0a6af Fix nightly cases for webhook
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-04-01 16:27:40 +08:00
danfengliu
8f9822d168 Add nightly test case of push docker manifest list
1. Add notary tool in E2E Dockerfile;
2. Add push docker manifest list test in nightly;
3. Modify document for e2e user guide;
4. Add CNAB tool  in E2E Dockerfile;

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-01 06:51:13 +00:00
danfengliu
08a070a124 Modify nightly test case for user view logs and other
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-23 14:19:13 +08:00
danfengliu
deb225b0f2 Modify elements locator in nightly tests
1. Modify get signature API in sign image test;
2. Modify locator for tag in artifact list;

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-22 20:48:37 +08:00
Danfeng Liu (c)
d9c0a4ae67 Switch registry from docker-hub ot internal harbor registry
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
:q
2020-01-17 10:43:20 +08:00
Danfeng Liu (c)
d02f15fb0f Add retry for prject list display
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2020-01-13 15:44:31 +08:00
Danfeng Liu (c)
9d651128d6 Add nightly test case for tag immutability
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2020-01-06 15:54:08 +08:00
danfengliu
79f93af471
Merge pull request #10333 from danfengliu/add-time-waiting-for-tag-switch
Replace keyword without waiting to one which had waiting
2019-12-30 14:45:24 +08:00
danfengliu
de2ec7d063 Replace keyword without waiting to one which had waiting
1. Replace all keywords without using waiting;
2. Add a debug line in Go Into Repo for UI hung trouble shooting.

Signed-off-by: danfengliu <danfengl@vmware.com>
2019-12-27 17:13:25 +08:00
danfengliu
933dc91ba5 Notary test failed in Helm Pipeline
In Helm pipeline, harbor access address is by domain name instead of IP, so cert directory should be created by domain name.

Signed-off-by: danfengliu <danfengl@vmware.com>
2019-12-27 17:02:28 +08:00
Danfeng Liu (c)
8bfc5d12c3 Change parameter image to optional
1. Change parameter in Keyword Body Of Admin Push Signed Image to an
optional one.
2. Loose the restriction for Quotas error message verification.
3. Get cert for notary from API instead of local file.

Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2019-12-19 16:54:59 +08:00
Danfeng Liu (c)
7dbe84d9c6 1. Add nightly test case of Robot-Account; 2. In Go Into Repo keyword, filter element should not be clicked since list in loading state, so add some sleep and waiting for the cirlce icon is gone; 3. Move read-only test case in the end of common.robot.
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2019-12-03 14:46:54 +08:00
Danfeng Liu (c)
156b1b0687 As we like to use a local registry for nightly quotas test, so that the sample image will not be update without noticed, so modify quotas test case to use local image for pulling.
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2019-11-29 14:56:41 +08:00
Danfeng Liu (c)
b7bd34d181 It's a temporary solution for quotas case, after internal registry in service, image size will not change any longer.
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2019-11-26 15:27:50 +08:00
Danfeng Liu (c)
f0750f73de 1. Add element verification for keyword Go Into Project; Make big loop effective by stop assert in keyword Retry Double Keywords When Error; 2. Quota string verification is too restrictive, make it loose a little.
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2019-11-25 17:44:15 +08:00
Danfeng Liu (c)
6e1f560544 Modify 2 points, 1st is add wait time right after scan all; 2nd is add name length for project name to avoid naming repeatly.
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2019-11-21 15:12:21 +08:00
sshijun
14364727ae Nightly case for tag-retention
Signed-off-by: sshijun <sshijun@vmware.com>
2019-11-11 16:58:33 +08:00
Yogi_Wang
58936aae42 Modify nightly case of scanner
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-11-11 13:19:37 +08:00
Danfeng Liu (c)
76c5cb85ec Before user keyword Enable Self Reg, explore should be opened and navigate to harbor log in page.
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2019-10-28 17:24:07 +08:00
Yogi_Wang
988279cdcb Add webhook nightly case
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-10-25 15:25:32 +08:00
Danfeng Liu (c)
732eda4196 Project quotas counting result is growing a little bit for unknow reason, images redis size was caculating from 25.8 to 25.9, so error message should be updated.
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2019-10-18 10:30:05 +08:00
danfengliu
ac0d3c991a Add nightly test case of 'Retag should be forbidden when system configuration read-only mode was set'.
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-10-10 17:13:19 +08:00
danfengliu
feeb6fb2d8 1. Nightly test case failed due to more vulnerbility CVE IDs in image redis, so add 2 new CVE IDs in CVE Whitelist test cases. 2. A Keyword parameter error was fixed in this PR.
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-10-02 16:43:50 +08:00
Danfeng Liu (c)
20dc245631 In nightly helm pipeline, test case 'Project Quotas Control Under GC' failed, it should be timing issue that the valuse in UI is not ready, so I add a keyword 'Retry Keyword When Return Value Mismatch' to retry value getting action.
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2019-09-25 17:02:05 +08:00
Danfeng Liu (c)
5d1913842c Script test case for project quotas, there will be 2 or 3 test cases in this PR, like project quota edit, prject quota functionality.
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2019-09-18 17:55:25 +08:00
danfengliu
b7803571d4
Currently, nightly test pipeline was only in one level category of auth_mode and upgrade, now we like to add a level base on the original level which is harbor deployment parameters. (#8398)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-08-01 15:10:54 +08:00
danfengliu
8665274406 In nighly test case <Project Level Image Serverity Policy>, scan result of haproxy image is nolonger high but medium, it should be a bug of clair, so we replace haproxy by postgres, and also we pull postgres with sha256 to make sure the image under test is alway the specific one.
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-07-25 10:27:10 +08:00
danfengliu
9b61bf4fce
In nightly test case 'list helm chart', delete chart (#8280)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-07-17 15:32:10 +08:00
danfengliu
a8300d91e6
Add a nightly test case helm CLI, in this case, we verify help repo add and helm push. (#8240)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-07-16 14:00:30 +08:00
danfengliu
a2b651969c
Fix bug for manage-project-member, due to keywords changed for oidc-auth-mode, test case manage-project-member should be debug at the same time. (#7888)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-05-24 15:22:28 +08:00
danfengliu
fed310a974
Add test case for oidc auth mode (#7860)
* Nightly test case user-view-logs failed several times, after debugging, the reason is checking too fast although the operation has not finsished, so I add a checking for end of the operation.

Signed-off-by: danfengliu <danfengl@vmware.com>

* Add test case Manage-Project-Member for oidc auth mode Jenkins pipeline, this test case reuse the same test case in common file.

Signed-off-by: danfengliu <danfengl@vmware.com>
2019-05-23 15:43:59 +08:00
danfengliu
a503944110
OIDC authentication is a new feature in version 1.8.0, so add nightly GUI test cases for OIDC authentication scenario. (#7745)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-05-22 11:19:08 +08:00
FangyuanCheng
82716d492d move vulnerability and gc ui
Signed-off-by: FangyuanCheng <fangyuanc@vmware.com>
2019-04-03 14:39:46 +08:00
danfengliu
3db3f3a62e
add test cases for jenkins upgrade project, wrap test cases body into new keywords, so that case body appears only in one single place (#7243)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-03-28 14:37:03 +08:00
danfengliu
6b19b99c97
test case Disable Scan Schedule failed at page should contain part, it missing wait until and retry, so add one (#7228)
Signed-off-by: danfengliu <danfengl@vmware.com>
2019-03-27 15:33:31 +08:00