Commit Graph

10271 Commits

Author SHA1 Message Date
Wenkai Yin
294385c34d Refactor the replication execution
1. Use the task manager to manage the underlying execution/task
2. Use the pkg/scheduler to schedule the periodical job
3. Apply the new program model
4. Migration the old data into the new data model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-11-23 14:24:10 +08:00
Daniel Jiang
3536a5dfac
Merge pull request #13452 from ninjadq/fix_status_code_issue_on_metrics
Fix status code in metrics issue
2020-11-23 10:37:47 +08:00
danfengliu
d011a2e6ec
Merge pull request #13529 from danfengliu/add-docker-hub-login-before-docker-hub-pull
Add docker-hub login before docker-hub pull
2020-11-23 10:05:40 +08:00
Daniel Jiang
00c941a413
Merge pull request #13557 from heww/fix-issue-13556
fix: remove auto re-readable body in BeforeRequest
2020-11-23 01:08:52 +08:00
danfengliu
2bf89e9356 Add docker-hub login before docker-hub pull
Due to docker-hub pull request rate limitation, we will use registed account to pull image
from docker-hub, therefore add docker-hub login in API tests.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-22 21:11:05 +08:00
He Weiwei
f733f95300 fix: remove auto re-readable body in BeforeRequest
Remove nop close body in the BeforeRequest helper function.
Middleware must make the request body re-readable itself when it wants
read the body in the middleware.

Closes #13556

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-20 01:41:12 +00:00
Wang Yan
04c4354df9
add robot account version 2 controller (#13472)
the controller is for the enhanced robot account

Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-11-19 15:39:45 +08:00
Wenkai Yin(尹文开)
def782b6f8
Don't escape the name when doing the exact matching for tag (#13533)
Don't escape the name when doing the exact matching for tag
Fixes #13106

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-11-18 14:34:48 +08:00
stonezdj
e667121a34 Add more registry type to proxy cache
Includes: azure-acr, aws-ecr, google-gcr
Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-11-18 10:38:07 +08:00
DQ
a6edf51574 Fix status code in metrics issue
Stauts code can't get from request. It shoud be recorded by a delegator

Signed-off-by: DQ <dengq@vmware.com>
2020-11-17 19:53:06 +08:00
Will Sun
eca3de3489
Merge pull request #13494 from dirkmueller/lock_json_include
Include package.json/package-lock.json in portal image
2020-11-16 16:38:02 +08:00
stonezdj(Daojun Zhang)
fb549b2d9e
Merge pull request #13444 from wy65701436/robot2-self-mgr
add robot mgr
2020-11-16 11:33:33 +08:00
Christian Baumann
4530e9feee Fix api ldap group search by name
Signed-off-by: Christian Baumann <thechristschn@gmail.com>
2020-11-12 23:01:02 +00:00
sshijun
3d526e220c Improve adding replication rule page
Signed-off-by: sshijun <sshijun@vmware.com>
2020-11-12 18:17:26 +08:00
danfengliu
ac8bc94012
Merge pull request #13367 from danfengliu/add-p2p-py-test
Add p2p python test script
2020-11-12 17:00:21 +08:00
He Weiwei
83c07d6680
fix: ensure the role_id of role is correct (#13476)
Closes #13317

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-12 15:34:04 +08:00
He Weiwei
76f1afbe0d
refactor: remove core/promgr pkg (#13408)
* refactor: remove core/promgr pkg

Remove `core/promgr` package and use `controller/project` instead of it.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-12 15:33:13 +08:00
danfengliu
5d0e6d6dbf Add p2p python test script
Add p2p python test including create disstribution instance and project p2p preheat policy.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-12 05:41:35 +00:00
Dirk Mueller
12adc63a48 Include package.json/package-lock.json in portal image
This allows Trivy and other vulnerability scanners to correctly
determine the embedded dependencies in minified harbor-portal image.

Also simplify build process by reducing the number of layers in the
final stage container image

Signed-off-by: Dirk Mueller <dirk@dmllr.de>
2020-11-11 21:21:28 +01:00
danfengliu
a70753d382 Add p2p python test script
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-11 14:55:23 +08:00
Wang Yan
3550b5e5e9 add robot mgr
the robot account manager to handle the CRUD

Signed-off-by: wang yan <wangyan@vmware.com>
2020-11-11 13:47:03 +08:00
stonezdj(Daojun Zhang)
ca8cb87790
Merge pull request #13439 from wy65701436/robot2-mgr
add role permission manager for robot enhancement
2020-11-11 10:35:10 +08:00
Will Sun
d32527dfd7
Merge pull request #13470 from AllForNothing/tooltp
Fix tooltip bug for adding replication rule page
2020-11-10 20:43:12 +08:00
Wang Yan
9723655378 update code per review comments
1, rename table name to permission_policy
2, rename functions name

Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-11-10 18:11:31 +08:00
Wang Yan
ec15e320bf add role permission manager for robot enhancement
1, add two db tables of role permission and rbac policy
2, add manager of these two tables

Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-11-10 16:49:29 +08:00
He Weiwei
ebc3443da9
Merge pull request #13474 from heww/fix-issue-11892
fix: compute artifact size from db for schema1 manifest
2020-11-10 16:20:39 +08:00
Wenkai Yin(尹文开)
2467aad05d
Merge pull request #13473 from ywk253100/201109_mac
Downgrade the version of github.com/prometheus/client_golang to 1.7.1 to avoid the compile error in mac
2020-11-10 14:27:14 +08:00
Qian Deng
10f0654230
Merge pull request #13421 from ninjadq/remove_clair
Remove clair
2020-11-10 13:58:39 +08:00
DQ
c10a6325d8 Add deprecated msg for clair
Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
DQ
0c9faea294 Clean up Clair in prepare script
Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
DQ
8a584aff89 Clean up clair and clair-adapter in build scripts
1. Makefles
  2. Dockerfiles
  3. Installation script
  4. harbor.yml template

Signed-off-by: DQ <dengq@vmware.com>
2020-11-10 11:39:18 +08:00
danfengliu
dae17a890d
Merge pull request #13442 from danfengliu/Suppress-urllib3-warning-and-fix-nighlty-element-id-issue
Suppress urllib3 warning and fix nightly element locator issue
2020-11-10 11:02:16 +08:00
sshijun
bf99f6d5cd Fix tooltip bug for adding replication rule page
Signed-off-by: sshijun <sshijun@vmware.com>
2020-11-10 10:29:08 +08:00
danfengliu
444de8536a Suppress urllib3 warning and fix nightly element locator issue and remove switch scanner
1. Since upgrade python to 3.7, some urllib3 warnings come out during
   execution, because missing disconnection when using request by docker
   library, it's not affect script execution, therefore ignore them first.
2. Replication tests failed in nightly due to new UI code, some element
   locators are not valid anymore;
3. Remove switch scanner test steps, since Clair was removed.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-10 10:04:46 +08:00
Wenkai Yin
69ddb9c2a8 Downgrade the version of github.com/prometheus/client_golang to 1.7.1 to avoid the compile error in mac
The version 1.8.0 of "github.com/prometheus/client_golang" bumps up lib "golang.org/x/sys" which referenced by "github.com/docker/docker@1.4.2". The changes of "golang.org/x/sys" causes the compile error in mac

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-11-10 08:49:03 +08:00
He Weiwei
9c8377909b fix: compute artifact size from db for schema1 manifest
Closes #11892

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-09 12:32:07 +00:00
DQ
9152521b11 Fix: log container password expire
move chage command to base image

Signed-off-by: DQ <dengq@vmware.com>
2020-11-09 18:29:41 +08:00
Will Sun
039733b200
Merge pull request #13435 from AllForNothing/robot-prefix
[UI]add robot name prefix
2020-11-05 10:36:53 +08:00
He Weiwei
955431257c
Merge pull request #13422 from heww/disable-db-acess-for-ping
feat: skip middlewares require db for ping
2020-11-05 08:19:49 +08:00
danfengliu
5cc890c78f
Merge pull request #13431 from danfengliu/add-tag-immutability-py-tests
Add API test scripts for tag immutability and scan signed image
2020-11-04 19:24:26 +08:00
stonezdj(Daojun Zhang)
7c1ad11763
Merge pull request #13425 from wy65701436/robot-prefix
add robot name prefix
2020-11-04 11:48:42 +08:00
Steven Zou
af7000d277
Merge pull request #13417 from bitsf/fix_gitlab_replication_task_num
fix(replication) gitlab replication task num not correct
2020-11-04 11:33:57 +08:00
Will Sun
c42ffe0915
Merge pull request #13410 from AllForNothing/helm-hub
Add deprecated mark to Helm Hub
2020-11-04 11:03:45 +08:00
Wang Yan
62208dc3e4 fix code conflict
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-11-03 09:42:37 -08:00
danfengliu
1b127b16c5 Add API test scripts for tag immutability and scan signed image
1. Add 7 API python test scripts for tag immutability;
2. Swagger file has object struct defination issue on imuutability policy rule, because
   this policy is in shared with retention rule, therefore, they should be unified to
   the same  type of rule;
3. Fixed a timeout issue of action keyword, waiting for a long period;
4. Add API test scripts for scan signed image.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-03 10:10:06 +00:00
AllForNothing
ca93eb2218 Add robot name prefix to system setting UI
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-11-03 16:34:36 +08:00
Will Sun
e0617be15d
Merge pull request #13419 from AllForNothing/improve-ui
Improve UI according to the designer's commonts
2020-11-03 16:11:33 +08:00
Qian Deng
a9805fddd7
Merge pull request #13289 from ninjadq/metrics_for_harbor
Add metrics to Harbor Core
2020-11-03 15:49:16 +08:00
Wang Yan
5a22019e3d add robot name prefix
The system admin can set the prefix in configuration UI.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-11-02 23:12:57 -08:00
AllForNothing
6452d38925 Add deprecated mark to Helm Hub
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-11-03 15:04:15 +08:00