danfengliu
e1eac2b4d0
Merge pull request #12688 from danfengliu/fix-upgrade-test-wrong-keyword-issue
...
Fix wrong keyword issue in upgrade test
2020-08-14 10:11:37 +08:00
danfengliu
9a2188641d
Upgrade python and robot for drone build
...
1.Separate Dockerfile to drone and nightly e2e, one Dockerfile should cover
both scenarios, it will be resolved later;
2.Fix docker api err type issue, API error has no message attibute;
3.Upgrade docker in nightly e2e images.
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-08-12 10:26:12 +00:00
danfengliu
ab7767f539
Fix wrong keyword issue in upgrade test
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-08-10 14:47:38 +00:00
danfengliu
4968b6619b
Recovery drone build script
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-08-10 14:23:44 +00:00
Flávio Ramalho
3d3fabbd20
Make notary-remove-image-signature more flexible ( #12536 )
...
Instead of explicitly defining the expect path to /usr/local/bin/expect,
use the env command, so the expect executable is searched for and launched
from wherever it is first found.
Signed-off-by: Flávio Ramalho <framalho@suse.com>
2020-07-31 15:49:45 +08:00
danfengliu
8328e95b63
Add GCS&S3 push-based replicatiton tests
...
1. Add GCS push-based replicatiton tests;
There is an issue against it, so it's commented out;
2. Add S3 push-based replicatiton tests;
3. Add filter registry keyword;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-07-29 12:44:49 +00:00
danfengliu
8a691c5df1
Add OIDC user onboard test in nightly
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-07-27 13:56:18 +08:00
danfengliu
e20cb51d6b
Fix issues in replication tests
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-07-20 09:42:55 +08:00
danfengliu
aa43afb601
Add push-based replication test in nightly
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-07-16 19:50:42 +08:00
danfengliu
0cb378d8fa
Verify error message when deleting signed image
...
1. Verify error message when deleting signed image;
2. Add test case - Replication From Dockerhub;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-07-06 10:56:31 +08:00
AllForNothing
fff6f7529a
Replace all whitelist with allowlist
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-06-24 16:17:17 +08:00
danfengliu
a8f159728b
Add API E2E pytest of GC with untag flag enabled
...
1. Fix issue of keyword Go Into Repo, the verification logic could be more strict;
2. Add API E2E pytest of GC with untag flag enabled;
3. Add sleep in test_user_view_logs.py for delete log ocurred;
4. Test Case - Tag CRUD is not stable. Although add button was clicked, but the tag was'nt added successfully.
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-23 14:48:04 +08:00
danfengliu
67ccc98c58
Add helm and helm3 CLI py-test
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-16 16:27:31 +08:00
danfengliu
4b5080d44e
Add singularity py-test
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-10 11:09:41 +08:00
danfengliu
e020b90bf0
add oras cli py-test
...
1. Add oras cli py-test;
2. Add env for notary url, allow to input different notary port instead of solid 4443;
3. Add retry for keyword Cannot Pull Image and make it longer during retry.
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-08 11:10:07 +08:00
danfengliu
3f2850ffdf
Scan result behavior changed after trivy upgrade
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-04 16:25:21 +08:00
danfengliu
ceaa0a57b3
Add replication schedule test
...
1. Add replication schedule test;
2. Add some sleep after project creation before push image to avoid push image v2 error.
It fixed issue# 12094: Scan image vmware/photon:1.0 return unsupported occasionally.
3. Add some sleep in test_user_view_logs.py, can't get operation logs sometimes.
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-06-02 15:51:10 +08:00
danfengliu
5d8a308158
Add retry for keyword go into index
...
1. Add retry for keyword go into index;
2. Add retry for keyword go into project.
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-05-27 08:55:58 +00:00
danfengliu
a07cc7bb2a
Merge pull request #11967 from danfengliu/fix-nightly-log-verification
...
Modify nightly quotas error message verification
2020-05-20 14:38:37 +08:00
danfengliu
b4a54adfeb
Add Scan schedule test in nightly
...
1. Add Scan schedule test in nightly;
2. Remove redundancy steps when create a new project;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-05-20 12:16:33 +08:00
danfengliu
2fb90cf258
Modify nightly quotas error message verification
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-05-19 14:59:21 +08:00
danfengliu
ac19f02bf5
Enhance verificaton for tag retention pytest
...
1. Modify scan related keywords, some of keywords are not effective.
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-05-12 16:41:47 +08:00
danfengliu
d10dfc8167
Fix issue of GC untag images case
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-30 16:10:03 +08:00
danfengliu
768b292b15
Add double confirm for clicking untag checkbox
...
1. Add double confirm for clicking untag checkbox;
2. Add retry for waiting GC completion;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-29 20:21:07 +08:00
danfengliu
cd0463047b
Add tag for ui test case of add nuew user
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-28 19:47:47 +08:00
danfengliu
f8ce615e12
Add ctr command for pulling oci
...
1. Manifest list can be pulled by ctr;
2. ui-test missing key checkpoint, fix it by add checking scan detail result;
3. add tag retension untag image test;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-28 15:01:49 +08:00
AllForNothing
8ff4003438
Add new nightly case admin add new users
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-04-28 11:10:06 +08:00
danfengliu
fdded0744a
Clean up gitaction env and add tag for pytest
...
1. Add docker prune cmd to release some space;
2. Add tag for pytest in case of debugging requirement;
3. Replace image to smaller size in robot account pytest;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-20 18:19:28 +08:00
Danfeng Liu (c)
1cc5d0261b
Add robot-account verification for upgrade test
...
1. Add Clair should be default scanner in upgrade test;
2. Add tag retention verification in upgrade test;
3. Add tag retention verification in upgrade test;
4. Add tag Immutability verification in upgrade test;
5. Add webhook verification in upgrade test;
6. Add CVE whitelist in upgrade test;
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2020-04-16 22:16:33 +08:00
Yogi_Wang
589d05a894
nightly move switch scanner case to mutiple-scanner file
...
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-04-16 10:05:08 +08:00
Yogi_Wang
8835813439
[test-case] add switch scanner case
...
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-04-15 14:53:17 +08:00
AllForNothing
46320641b4
Add new nightly case for GC untagged images
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-04-14 09:25:06 +08:00
Will Sun
009662a317
Merge pull request #11426 from AllForNothing/nightly-tag
...
Add new nightly cases for tag CRUD
2020-04-13 12:15:18 +08:00
danfengliu
a869bba59f
Modify quota error info due to product changed
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-10 21:06:05 +08:00
danfengliu
66dfd59c80
Add Multi-Scanner test file
...
1. Add Multi-Scanner test file for harbor is deployed with multi scanners;
2. Modify notary doc description;
3. Add paragraph for e2e-api-python-based-scripting-guide;
4. Fix delete project issue;
5. Remove count qoutas in nightly and modify some of it;
6. Add Trivy in git hub offline action;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-08 16:26:31 +08:00
AllForNothing
e342b4ef0b
Add new nightly cases for tag CRUD
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-04-08 09:56:27 +08:00
Yogi_Wang
2610fe530f
[Test Case] Add nightly case for CVE
...
1. add nightly case for cve
2. change translate words
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-04-03 16:59:45 +08:00
jwangyangls
4ea7b13215
Merge pull request #11385 from jwangyangls/nightly-case-trivy-2
...
[Nightly] Project Level Image Serverity Policy
2020-04-02 11:14:21 +08:00
danfengliu
aa26e624b9
Merge pull request #11379 from danfengliu/add-nightly-test-of-push-helm-chart
...
Add nightly test of push helm chart
2020-04-02 10:29:21 +08:00
Yogi_Wang
01f8291bb7
[Nightly] Project Level Image Serverity Policy
...
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-04-02 10:16:57 +08:00
danfengliu
e3b31bb8dd
Add nightly test of push helm chart
...
1. Add push helm chart oci test in nightly;
2. Modify archive icon locator xpath;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-01 18:53:19 -07:00
danfengliu
f11e713ada
Merge pull request #11380 from jwangyangls/nightly-case-trivy-1
...
[Nightly] add case about trivy
2020-04-01 22:45:40 +08:00
Yogi_Wang
24b57715ab
[Night] add case about trivy
...
`
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-04-01 17:06:34 +08:00
AllForNothing
e6e3f0a6af
Fix nightly cases for webhook
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-04-01 16:27:40 +08:00
danfengliu
8f9822d168
Add nightly test case of push docker manifest list
...
1. Add notary tool in E2E Dockerfile;
2. Add push docker manifest list test in nightly;
3. Modify document for e2e user guide;
4. Add CNAB tool in E2E Dockerfile;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-04-01 06:51:13 +00:00
danfengliu
77e3b0d828
Fix nightly ldap test cases
...
1. Modify robot account test, checkout "Never Expired" button;
2. Modify OIDC get user API;
3. Modify verification for docker pull command;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-27 20:24:43 +08:00
Yogi_Wang
1f3e00c782
[Nightly] Add case for trivy
...
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-03-27 15:20:39 +08:00
danfengliu
08a070a124
Modify nightly test case for user view logs and other
...
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-23 14:19:13 +08:00
danfengliu
deb225b0f2
Modify elements locator in nightly tests
...
1. Modify get signature API in sign image test;
2. Modify locator for tag in artifact list;
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-22 20:48:37 +08:00
danfengliu
35786e8efd
Analyze nightly failed cases
...
1. Modify delete repo button locator;
2. Modify verifcation for GC log result message;
3. Modify locator for tag retention.
4. Add empty Trivy pipeline.
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-03-21 11:30:15 +08:00