Commit Graph

328 Commits

Author SHA1 Message Date
Shengwen YU
3f75f0db32
feat: add python testing script for stop scan job (#15779)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>

Co-authored-by: Shengwen Yu <yshengwen@vmware.com>
2021-10-18 13:13:53 +08:00
YangJiao
5533015fd1 Add Go To Harbor Api Page TestCase
Signed-off-by: YangJiao <jiaoya@vmware.com>
2021-09-17 16:06:11 +08:00
YangJiao
a4a82d763a Add TestCase to download helmchart from UI
Signed-off-by: YangJiao <jiaoya@vmware.com>
2021-09-13 14:06:44 +08:00
danfengliu
a4331f6b60 add-notary-resign-test-in-upgade-pipeline
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-08-10 20:41:12 +08:00
danfengliu
8a80f82bb5
Merge pull request #15328 from danfengliu/fix-nightly-ca-header-issue
Fix nightly ca header issue
2021-07-23 15:35:04 +08:00
孙世军
43da6467c0
Add a case of deleting an OIDC user (#15309)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-07-19 10:56:06 +08:00
danfengliu
32ea961f9f Fix nightly ca header issue
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-07-18 10:44:40 +08:00
danfengliu
8947a62dc4 Fix replication test issue in nightly
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-07-11 17:54:55 +08:00
danfengliu
646ca80187
Fix nightly test failures broken by UI change (#15233)
1. Fix break of byte statistic unit changing;
2. Fix OIDC login UI changing.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-07-02 10:19:43 +08:00
danfengliu
0bea79df93 Remove workround for docker20 adaptation
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-06-09 14:57:52 +08:00
danfengliu
be6ce16223 Add UI test for replicaiton flattening
1. Fix replication namespace level issue;
2. Add UI test for replicaiton flattening.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-05-31 14:14:50 +08:00
danfengliu
a40d609164 Remove customize case from common test
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-05-26 10:46:11 +08:00
danfengliu
344e51d2c6 Fix build base workflow issues
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-05-25 18:13:15 +08:00
danfengliu
3ef4dc17fc Fix notary trust directory issue and add login for each base image in makefile
1. Use root instead of ~ in notary parameter;
2. Fix tag immutability issue caused by GUI change;
3. Replace email domain name to harbor test;
4. Add login for each base image in makefile;
5. Add customize look test in nightly.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-05-20 18:23:13 +08:00
danfengliu
072f364d0b Add manifest and CNAB replication tests
1. Add manifest and CNAB replication tests;
2. Duplicate ORAS and sigularity tests from API test to nightly common
tests;
3. Optimize get dns code in CI;
4. Optimize E2E dockerfile;
5. Sample image size should be cover requirement for large size like 512M.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-05-10 21:47:48 +08:00
danfengliu
f0ebd17994 Add build base image step in build package git action workflow
Build base image step should be in build package workflow, and local base images build by new step should be removed since images have been pushed to docker hub.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-04-17 18:10:44 +08:00
danfengliu
fbf2409c78
Merge pull request #14472 from danfengliu/add-tests-for-manifest-replication
Add test for manifest replication
2021-04-02 17:39:14 +08:00
danfengliu
1801a3cee3 Add manifest and CNAB replication tests
1. Add manifest and CNAB replication tests;
2. Duplicate ORAS and sigularity tests from API test to nightly common
tests;
3. Optimize get dns code in CI;
4. Optimize E2E dockerfile;
5. Sample image size should be cover requirement for large size like 512M.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-04-02 15:39:30 +08:00
Wenkai Yin
3d7fd070c7 Refeactor replication policy APIs
Refeactor replication policy APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-17 10:57:31 +08:00
danfengliu
a2fc1bcfaf Fix quotas test issue in upgrade pipeline
1. Change way for quotas verification in upgrade pipeline, prepare specific size of image, then it's an known value for verifcation;
2. Add notary key rotate test;
3. For issue brought by docker 20, clean containerd cache is the only effective way, so both dockerd and containerd should be cache cleard and restarted;
4. Upgrade E2E Dockerfile for importing readable file size package, and other issues;
5. Uncomment project level robot account test in nightly.
6. Get DNS from local setting, and set it into docker deamon config file;

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-03-10 06:27:22 +00:00
danfengliu
7fb9dbd0fa Upgrade docker and containerd
1. Fix E2E quotas issue, push the same image but with different name;
2. Add checkpoint for robot account test;
3. Upgraded docker and containerd in E2E image;
4. Package base image sample(busybox) into E2E image, so in E2E
container, all local docker images can be cleaned up, once base image is needed for
building image, it can be loaded locally;
5. Adapt OIDC service of supporting LDAP user, and add OIDC group user
test;
6. Restart docker deamon before content trust test, both in API and UI
test;
7. Add retry for keyword "Add A Tag Immutability Rule";
8. Fix tag retention test issue, missing click angle icon, and enhance
checkpoint of dry run and real run;
9. Fix schedule test issue for wrong cron string;
10. Disable quotas verification, it's not stable for script defect;

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-02-24 15:43:11 +08:00
danfengliu
642bb26c39 Fix nightly issues caused by docker upgrade to 20
1. Local image should be removed, otherwise docker 20 will not tigger
get manifest request to harbor;
2. E2E image Dockerfile update;
3. Fix nighlty test issue of tag retention, add execution refesh to get
result;
4. Fix nighlty test keyword 'Create An New Project And Go Into
Project' issue that waiting long enough time for list display;
5. Add nightly test case, in GUI, scan result will show if cve id exist in allow list configuration;
6. Move proxy cache test to schdule pipeline, it will save some time for
db pipeline.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-28 11:44:32 +08:00
danfengliu
d4b7888098
Merge pull request #13882 from danfengliu/fix-nightly-keyword-issues
Add tests and fix nightly issues
2021-01-19 17:28:52 +08:00
Danfeng Liu (c)
ddf9d74135 Fix nightly keyword issues
1. Add retry for Get Statics Text keywords make this operation robust;
2. Replace image to one with slash in name for CNAB test, to cover more
condition;
3. Add replication rule deletion verfication in upgrade test;
4. Add non-admin user in pull operation for proxy cache test;
5. Add verification for quota display for upgrade test;
6. Add test for large size of image replication;
7. Add test checkpoint for system robot account py-test script;

Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2021-01-19 15:12:15 +08:00
Qian Deng
d06ce6efd5
Merge pull request #13922 from ninjadq/add_e2e_test_for_metrics
Add e2e test for harbor metrics
2021-01-18 11:04:11 +08:00
Ziming Zhang
5bff3e94a0 fix(immutability) swagger test case
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-11 03:21:12 +00:00
Ziming Zhang
be58c1d3ff fix(retention) swagger test case
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-11 03:21:12 +00:00
DQ
27a9f71e26 Add e2e test for harbor metrics
Signed-off-by: DQ <dengq@vmware.com>
2021-01-09 04:12:32 +08:00
Ziming Zhang
39fb500318 feat(retention) refactor to use go swagger api
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-08 07:09:28 +00:00
danfengliu
f57dc13081 Add py-test for system robot account
In this py-test, add a new system robot account, and verify all project privileges.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-08 12:01:53 +08:00
danfengliu
511bd86930
Merge pull request #13849 from danfengliu/add-gui-test-for-system-robot-account
Ad  nightly GUI test for system robot account
2021-01-06 16:59:05 +08:00
Danfeng Liu (c)
ae05e3fdb0 Add Nightly GUI test for system robot account
1. Add GUI test for system robot account feature;
2. Fix issue of failing to catch docker pull image exception in API test.

Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2021-01-06 15:47:04 +08:00
danfengliu
076ebb5ee8 Clear local images in nightly test
Remove local image right after pushing operation, then pull operation can be triggered by docker 20.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-04 10:25:02 +08:00
danfengliu
3659abc62a Migrate gitlab replication test to nightly
Add 2 gitlab pull/push replication test scripts in replication pipeline.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-11 18:04:46 +08:00
danfengliu
ad7581d9b3 Migrate gitlab replication test to nightly
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-09 12:00:38 +08:00
danfengliu
56a35437b4
Change image source in e2e pytest (#13640)
Change source of most of test image samples from docker-hub to local building ones, so it will cost less docker-hub pull requests.
And some of cases like push cnab, they have to use docker-hub, but image samples in cnab test will cost 17 quotas, in this PR, we
replace those samples, now cnab case will cost 6 quotas.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-04 02:28:29 -08:00
DQ
590212b485 Remove clair related code
- clair code in harbor core
- clair code in frontend
- clair code in robotcase

Signed-off-by: DQ <dengq@vmware.com>
2020-11-27 14:01:04 +08:00
danfengliu
2bf89e9356 Add docker-hub login before docker-hub pull
Due to docker-hub pull request rate limitation, we will use registed account to pull image
from docker-hub, therefore add docker-hub login in API tests.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-22 21:11:05 +08:00
danfengliu
a70753d382 Add p2p python test script
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-11 14:55:23 +08:00
danfengliu
1b127b16c5 Add API test scripts for tag immutability and scan signed image
1. Add 7 API python test scripts for tag immutability;
2. Swagger file has object struct defination issue on imuutability policy rule, because
   this policy is in shared with retention rule, therefore, they should be unified to
   the same  type of rule;
3. Fixed a timeout issue of action keyword, waiting for a long period;
4. Add API test scripts for scan signed image.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-03 10:10:06 +00:00
danfengliu
691168b8cf
Merge pull request #13359 from danfengliu/optimize-upgrade-pipeline-of-pull-push-image
Optimize upgrade pipeline script of push image
2020-10-29 14:31:30 +08:00
danfengliu
ab3e24a2e3 Add proxy cache python test scipts and upgrade containerd
Containerd in e2e image was using native one in ubuntu, it should be updated to the latest release.
And add proxy cache py-tests including pull image/manifest list by docker and ctr CLI.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-10-28 02:13:40 +00:00
danfengliu
cff4f529cd Optimize upgrade pipeline script of push image
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-10-26 15:16:18 +08:00
danfengliu
4bc7444602 Add UI test for LDAP admin group dn setting
1. Verify LDAP group admin in UI;
2. Update LDAP group admin need to verify the AdminRoleInAuth, get current user will return this field as true.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-10-14 11:05:09 +08:00
danfengliu
e4894017cd Fix typo in upgrade prepare scritp
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-09-28 10:36:22 +08:00
danfengliu
48510fde08
Merge branch 'master' into add-2.1-upgrade-pipeline-in-nightly 2020-09-22 13:48:22 +08:00
danfengliu
ef53edf553 Recovery nightly setup
1. Recovery nightly setup;
2. Add retry for replication rule selection.
3. Add retry for CNAB test.
4. Rerun keyword of push CNAB

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-09-22 02:41:50 +00:00
danfengliu
e2989e3b93 Add 2.0 & 2.1 upgrade pipeline in nightly
1. Add 2.0 upgrade pipeline in nightly;
2. Add 2.1 upgrade pipeline in nightly.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-09-22 02:14:40 +00:00
danfengliu
86fb6fdc65 Add checkpoint for LDAP group py-test
LDAP group has different role, user in group has the same role, as groups with different roles were added in project
member list, user should act like in different roles. for admin and dev role, there should be checkpoints to verify priviledges of each own.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-09-14 21:15:09 +08:00
danfengliu
931bde0df8 Fix issue of missing swagger package for python API test
1. Python API test can't import packages from swagger maker, the reason is that python install packages to site-packages instead of dist-packages,
   so the current solution is to copy pakages from site back to dist, I will try to find a way of using ENV variable to fix this;

2. Remove a python library no longer be used.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-09-02 15:18:23 +08:00