Commit Graph

965 Commits

Author SHA1 Message Date
Wenkai Yin(尹文开)
4c2e698af8
Merge pull request #14341 from heww/refactor-scanner-apis
refactor: generate scanner APIs by go-swagger
2021-03-04 17:48:04 +08:00
danfengliu
85c10c1c31 Remove python3.7 deployment in CI installtion phase.
Git action work node OS had python3.8 deployed already, so won't necessary to install python3.7 in CI.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-03-04 14:24:19 +08:00
He Weiwei
ef0bdf6954 refactor: generate scanner APIs by go-swagger
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-03-03 04:23:36 +00:00
He Weiwei
4b033c266a refactor: generate quota APIs by go-swagger
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-02-25 08:19:55 +00:00
danfengliu
7fb9dbd0fa Upgrade docker and containerd
1. Fix E2E quotas issue, push the same image but with different name;
2. Add checkpoint for robot account test;
3. Upgraded docker and containerd in E2E image;
4. Package base image sample(busybox) into E2E image, so in E2E
container, all local docker images can be cleaned up, once base image is needed for
building image, it can be loaded locally;
5. Adapt OIDC service of supporting LDAP user, and add OIDC group user
test;
6. Restart docker deamon before content trust test, both in API and UI
test;
7. Add retry for keyword "Add A Tag Immutability Rule";
8. Fix tag retention test issue, missing click angle icon, and enhance
checkpoint of dry run and real run;
9. Fix schedule test issue for wrong cron string;
10. Disable quotas verification, it's not stable for script defect;

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-02-24 15:43:11 +08:00
Will Sun
84f672bef5
Fix nightly failure for p2p case (#14261)
Signed-off-by: sshijun <sshijun@vmware.com>
2021-02-23 09:56:01 +08:00
Josh Soref
dfe360040b Spelling
* addition
* attribute
* auditing
* availability
* available
* bandwidth
* browser
* business
* cadence
* chartmuseum
* client
* column
* content
* demonstrate
* described
* endpoints
* facilitate
* github
* harbor
* information
* instance
* manual
* meaningful
* operation
* overridden
* password
* possible
* project
* refactor
* replication
* requires
* running
* scanned
* settings
* signup
* those
* unsigned
* vulnerability

--
Also removes trailing space from a filename

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2021-02-19 11:59:15 +08:00
danfengliu
f51bcd9fdf
Merge pull request #14092 from dirkmueller/zip_slip
Update oras to 0.9.0 to fix "zip slip" vulnerability
2021-02-07 16:15:11 +08:00
Wang Yan
885a268268
rename permission of robot account (#14150)
* rename permission of robot account

fixes #14060
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-02-04 15:32:44 +08:00
Qian Deng
6d78189dff
Merge pull request #14117 from ninjadq/fix_quota_metric_name
Chagne quota usage metric name
2021-01-29 10:36:55 +08:00
danfengliu
b0e54f5a33
Merge pull request #14041 from danfengliu/upgrade-containerd-in-e2e-dockerfile
Upgrade containerd in E2E Dockerfile
2021-01-29 09:42:41 +08:00
DQ
eb5e31a442 Chagne quota usage metric name
from `harbor_project_usage_byte` to `harbor_project_quota_usage_byte`

Signed-off-by: DQ <dengq@vmware.com>
2021-01-28 18:10:59 +08:00
Qian Deng
ebe05919be
Merge pull request #14078 from ninjadq/fix_self_registration_metric
Fix metric label related things
2021-01-28 17:27:53 +08:00
danfengliu
642bb26c39 Fix nightly issues caused by docker upgrade to 20
1. Local image should be removed, otherwise docker 20 will not tigger
get manifest request to harbor;
2. E2E image Dockerfile update;
3. Fix nighlty test issue of tag retention, add execution refesh to get
result;
4. Fix nighlty test keyword 'Create An New Project And Go Into
Project' issue that waiting long enough time for list display;
5. Add nightly test case, in GUI, scan result will show if cve id exist in allow list configuration;
6. Move proxy cache test to schdule pipeline, it will save some time for
db pipeline.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-28 11:44:32 +08:00
Dirk Mueller
3ddc44e28b Update oras to 0.9.0 to fix "zip slip" vulnerability
See https://github.com/deislabs/oras/security/advisories/GHSA-g5v4-5x39-vwhx
for details

Signed-off-by: Dirk Mueller <dirk@dmllr.de>
Signed-off-by: Dirk Mueller <dmueller@suse.com>
2021-01-26 20:36:52 +01:00
danfengliu
54b6884853 Upgrade python to v3 in git action CI host
The original python packaged: in git action host is V2, it should be upgraded to
V3.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-26 21:30:03 +08:00
DQ
66fd4a9728 change label project id to project name
project_name is more accurate

Signed-off-by: DQ <dengq@vmware.com>
2021-01-26 16:15:04 +08:00
DQ
25ea2b1c82 Use apt to install python in ci
To avoid upstream get_pip.py changes

Signed-off-by: DQ <dengq@vmware.com>
2021-01-25 16:54:09 +08:00
DQ
28ae77e5c6 Aggregate metrics
1. Add operation id in ctx in baseapi before prepare
2. add operation id for registry proxy request
3. use url for other request

Signed-off-by: DQ <dengq@vmware.com>
2021-01-25 09:59:10 +08:00
Wang Yan
dba229d0df
build third party binaries in CI (#14019)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-22 11:33:42 +08:00
danfengliu
d4b7888098
Merge pull request #13882 from danfengliu/fix-nightly-keyword-issues
Add tests and fix nightly issues
2021-01-19 17:28:52 +08:00
Danfeng Liu (c)
ddf9d74135 Fix nightly keyword issues
1. Add retry for Get Statics Text keywords make this operation robust;
2. Replace image to one with slash in name for CNAB test, to cover more
condition;
3. Add replication rule deletion verfication in upgrade test;
4. Add non-admin user in pull operation for proxy cache test;
5. Add verification for quota display for upgrade test;
6. Add test for large size of image replication;
7. Add test checkpoint for system robot account py-test script;

Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2021-01-19 15:12:15 +08:00
Qian Deng
d06ce6efd5
Merge pull request #13922 from ninjadq/add_e2e_test_for_metrics
Add e2e test for harbor metrics
2021-01-18 11:04:11 +08:00
Dirk Mueller
903707d6c2
Fix repoitory -> repository typo (#13794)
Signed-off-by: Dirk Mueller <dirk@dmllr.de>
2021-01-16 19:34:43 +08:00
Will Sun
b7c5fc0562
Modify i18n text for push permission (#14015)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-15 15:26:48 +08:00
danfengliu
a576ee8694 Pass git action secret tp push image function in build workflow
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-12 10:01:50 +08:00
danfengliu
5f713f22fd Push harbor build images to docker-hub in build workflow
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-11 18:24:57 +08:00
Ziming Zhang
5bff3e94a0 fix(immutability) swagger test case
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-11 03:21:12 +00:00
Ziming Zhang
be58c1d3ff fix(retention) swagger test case
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-11 03:21:12 +00:00
DQ
27a9f71e26 Add e2e test for harbor metrics
Signed-off-by: DQ <dengq@vmware.com>
2021-01-09 04:12:32 +08:00
Ziming Zhang
39fb500318 feat(retention) refactor to use go swagger api
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-01-08 07:09:28 +00:00
Will Sun
f566748c77
Fix nightly case by add to scan button (#13937)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-01-08 15:01:26 +08:00
danfengliu
f57dc13081 Add py-test for system robot account
In this py-test, add a new system robot account, and verify all project privileges.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-08 12:01:53 +08:00
danfengliu
511bd86930
Merge pull request #13849 from danfengliu/add-gui-test-for-system-robot-account
Ad  nightly GUI test for system robot account
2021-01-06 16:59:05 +08:00
danfengliu
b5607850af
Merge pull request #13834 from danfengliu/clear-local-useless-images-in-nightly
Clear local images in nightly test
2021-01-06 16:55:22 +08:00
Danfeng Liu (c)
ae05e3fdb0 Add Nightly GUI test for system robot account
1. Add GUI test for system robot account feature;
2. Fix issue of failing to catch docker pull image exception in API test.

Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2021-01-06 15:47:04 +08:00
danfengliu
076ebb5ee8 Clear local images in nightly test
Remove local image right after pushing operation, then pull operation can be triggered by docker 20.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-04 10:25:02 +08:00
Wang Yan
7a8a8fa104
upgrade go version to v1.15.6 (#13836)
Signed-off-by: wang yan <wangyan@vmware.com>
2020-12-23 18:53:09 +08:00
He Weiwei
ce6ed3eeb7 refactor(api): move scan all apis to go-swagger
Move scan all APIs from beego to go-swagger.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-21 02:17:02 +00:00
danfengliu
4f0842bd23 Add py-test for scan manifest list and CNAB bundle
Due to complicate logic of scan report in multi-level artifacts, should add scan tests
for into manifest list and CNAB bundle python test suit.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-18 16:37:53 +08:00
Wang Yan
01eb60c36b
add permission checking for gc operation (#13756)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-15 23:27:12 +08:00
Wenkai Yin(尹文开)
6569016d35
Merge pull request #13139 from wy65701436/migrate-gc
Migrate gc to task manager
2020-12-14 10:43:44 +08:00
danfengliu
3659abc62a Migrate gitlab replication test to nightly
Add 2 gitlab pull/push replication test scripts in replication pipeline.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-11 18:04:46 +08:00
wang yan
1bb79d402d update code per review comments
Signed-off-by: wang yan <wangyan@vmware.com>
2020-12-10 16:08:52 +08:00
danfengliu
ad7581d9b3 Migrate gitlab replication test to nightly
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-09 12:00:38 +08:00
danfengliu
56a35437b4
Change image source in e2e pytest (#13640)
Change source of most of test image samples from docker-hub to local building ones, so it will cost less docker-hub pull requests.
And some of cases like push cnab, they have to use docker-hub, but image samples in cnab test will cost 17 quotas, in this PR, we
replace those samples, now cnab case will cost 6 quotas.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-04 02:28:29 -08:00
danfengliu
ff50e2363d
Move proxy cache e2e test into a new CI job (#13690)
CI job timeout is 60 minutes, but proxy cache case timeout is 20 minutes, once
proxy cache case is timeout, it will reach CI job timeout, then cause CI job terminated
by timeout without running rest of test steps, expecially uploading useful harbor logs,
so move this test into a new clean job for saving time and debugging.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-04 02:27:59 -08:00
danfengliu
792a3c8c27
Merge pull request #13625 from danfengliu/fix-scan-all-py-test-script
Fix scan all python test script issue of missing test step
2020-12-02 09:17:10 +08:00
Wang Yan
732e9a21cd
updates on robot accounts (#13623)
* updates on robot accounts

1, add patch method to refresh secret of a robot
2, fix robot account update issue
3, add editable attribute to handle the version 1 robot account
4, add duration for robot account
5, hide secret for get/list robot account

Signed-off-by: wang yan <wangyan@vmware.com>

* update code per review comments

1, change expirate creation func to AddDate().
2, remove the scanner duration specification, use the default value.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-01 18:31:34 +08:00
danfengliu
b36a4cf3be Fix scan all python test script issue of missing test step
1. In scan all py-test, missing a important test step for checkin scan all status,
2. which indicate the ending of scan all job.
In swagger, ongoig field is missing in stats struct.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-27 20:27:37 +08:00