Commit Graph

2416 Commits

Author SHA1 Message Date
FangyuanCheng
609f6cbda2 Fix replication frontend issues
Signed-off-by: FangyuanCheng <fangyuanc@vmware.com>
2019-04-16 15:20:05 +08:00
Wenkai Yin
94cacf762a Add new registry type: LocalHarbor
The "LocalHarbor" is the type of registry where the replication service is running on

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-16 11:42:36 +08:00
Wenkai Yin
2f1d2257d5 Remove the namespace concept in replication
Update the replication logic to remove the "namespace"

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-16 08:36:42 +08:00
慕薇疯魔
027cf44e82 unit test for native adapter.
Signed-off-by: 慕薇疯魔 <kfanjian@gmail.com>
2019-04-15 23:51:20 +08:00
慕薇疯魔
c5045933a1 change to testable.
Signed-off-by: 慕薇疯魔 <kfanjian@gmail.com>
2019-04-15 20:39:50 +08:00
Wenkai Yin
7729bca23b
Merge pull request #7382 from wy65701436/replication_ng
Add event trigger to helm deletion replication
2019-04-15 20:36:10 +08:00
wang yan
7a373c2eed Add event trigger to helm upload/deletion replication
Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-15 19:02:33 +08:00
Wenkai Yin
d9a2b2b79a
Merge pull request #7380 from cd1989/anonymous-dockerhub
Support anonymous access to dockerhub
2019-04-15 18:41:02 +08:00
Wenkai Yin
24e7019b60
Merge pull request #7384 from kofj/fix_deadlock
fix deadlock when client not exist.
2019-04-15 18:40:10 +08:00
Yogi_Wang
4faea92b04 Release a test case about 'should filter data by keyword'
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-04-15 17:00:27 +08:00
FangyuanCheng
a95cf813fb Fix when editting the policy, the source registry isn’t shown
Signed-off-by: FangyuanCheng <fangyuanc@vmware.com>
2019-04-15 16:17:20 +08:00
慕薇疯魔
47d6e2583a fix deadlock when client not exist.
Signed-off-by: 慕薇疯魔 <kfanjian@gmail.com>
2019-04-15 16:12:42 +08:00
cd1989
21c7554644 Support anonymous access to dockerhub
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-14 16:44:24 +08:00
Wenkai Yin
00ecbd459e Fix buf in native docker registry adapter
Check the existence of repository if the name filter contains no * and ?

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-14 14:50:44 +08:00
Wenkai Yin
ba038eb883 Support replication all projects in Harbor
Support replication all projects in Harbor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-13 20:23:19 +08:00
Wenkai Yin
c222f18fa7 Update replication
1. Refine the health check of docker hub
2. Remove the GetNamespace method from adapter interface

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-13 15:20:06 +08:00
Daniel Jiang
f92bc8076d "Skip verify cert" to "verify cert"
This commit tweaks the attribute for auth proxy mode and OIDC auth mode.
To change it from "Skip verify cert" to "verify cert" so they are more
consistent with other modes.
Additionally it removes a workaround in `SearchUser` in auth proxy
authenticator.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-04-12 23:25:54 +08:00
Wenkai Yin
1d16e18dff Remove "ng" from source code
Remove "ng" from source code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-12 22:42:27 +08:00
Wenkai Yin
d4b6eede98
Merge pull request #7331 from cd1989/ng-operation-review
Correct some typos and do some small adjustments
2019-04-12 21:50:42 +08:00
cd1989
8ca5e17c58 Correct some typos and do some small adjustments
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-12 21:17:33 +08:00
慕薇疯魔
a555b09df0 1. Implement native docker registry adapter.
2. register the native adapter.

Signed-off-by: 慕薇疯魔 <kfanjian@gmail.com>
2019-04-12 21:08:23 +08:00
Wenkai Yin
f0f92791c7
Merge pull request #7333 from cd1989/check-status-when-add-registry
Check health status when add registry
2019-04-12 20:39:44 +08:00
Wenkai Yin
5ee30c7909
Merge pull request #7127 from cd1989/dockerhub-adaptor
Implement dockerhub adapter
2019-04-12 20:25:18 +08:00
cd1989
7f6f85aab3 Remove ValidResource method
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-12 19:47:27 +08:00
Wenkai Yin
c88eeef07c
Merge pull request #7360 from ywk253100/190412_case
Remove the E2E test cases for replication
2019-04-12 18:44:16 +08:00
Wenkai Yin
174a1f5bfa Remove the E2E test cases for replication
Remove the E2E test cases for replication

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-12 17:37:14 +08:00
cd1989
cc4f08cc04 Add valid method to adapter to valid resource
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-12 17:33:32 +08:00
cd1989
1231c533db Support name and tag filters
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-12 17:31:32 +08:00
cd1989
ca705275f2 Support pull based replication from Dockerhub
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-12 17:31:32 +08:00
cd1989
b876a576a6 Implement dockerhub adapter
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-12 17:31:32 +08:00
cd1989
a9fa22269c Check health status when add/update registry
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-12 17:29:05 +08:00
Wenkai Yin
271c5ab213
Merge pull request #7357 from ywk253100/190412_harbor_adapter
Add filter support when fetching resources for Harbor adapter
2019-04-12 15:51:58 +08:00
Wenkai Yin
7d4649fc6b
Merge pull request #7163 from yuanshuhan/replication_ng
huawei cloud registry adapter for replication
2019-04-12 15:32:23 +08:00
Wenkai Yin
04883d2b0d Support replication windows images
Skip the foreign layers when do the replication

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-12 15:28:41 +08:00
Wenkai Yin
2fe207b77d Add filter support when fetching resources for Harbor adapter
Add filter support when fetching resources for Harbor adapter

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-12 15:28:41 +08:00
Yogi_Wang
8b82e8925e Add Token Review Endpoint in configuration page
adding token review endpoint to the configuration page allows users to edit token review endpoint when logging in through http auth.

Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-04-12 14:29:39 +08:00
Yuan Lei
f8cb2d1e86 huawei cloud adapter for replication
Signed-off-by: Yuan Lei <yuanlei9@huawei.com>
2019-04-12 11:18:23 +08:00
Wenkai Yin
188d66d875
Merge pull request #7350 from ywk253100/190411_bugfix
Fix bug of replication
2019-04-12 08:22:59 +08:00
Wenkai Yin
bc0123662b Fix bug of replication
1. check the disable/enable status before starting the replication
2. process the support_namespace property

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-12 07:59:57 +08:00
Wenkai Yin
1f574e7d10
Merge pull request #7354 from wy65701436/replication_ng_namespace
Add api to get namespaces of registry
2019-04-11 23:44:24 +08:00
wang yan
117c36d52c Add api to get namespaces of registry
To query the namespace of the registry according to its ID.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-11 23:17:33 +08:00
Daniel Jiang
83a2246485
Merge pull request #7335 from reasonerjt/oidc-onboard-e2e
OIDC E2E flow + secret support
2019-04-11 18:13:22 +08:00
Wenkai Yin
a2fcb41b31 Fix bug in ping registry API
Fix bug in ping registry API: accept both ID and other properties

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-11 18:00:44 +08:00
jwangyangls
08ecb87220
Merge pull request #7336 from jwangyangls/loginPageChange
Change the login module
2019-04-11 17:00:34 +08:00
Daniel Jiang
763c5df010 Add UT
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-04-11 15:30:19 +08:00
Wenkai Yin
b73acde051 Support the migration for scheduled replication rule from previous version of Harbor
Support the migration for scheduled replication rule from previous version of Harbor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-11 13:14:32 +08:00
Yogi_Wang
94438608a7 Add Cli secret in Harbor user profile
user could copy the CLI secret when login through oidc, and using this cli secret as password when using docker/helm cli to access Harbor.

Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-04-11 12:52:15 +08:00
Wenkai Yin
5a047a7eb6 Update the adapter interface
Add ConvertResourceMetadata and PrepareForPush methods

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-10 23:37:11 +08:00
Daniel Jiang
0d18e6c82f Update according to comments
For more context see PR #7335

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-04-10 19:38:12 +08:00
Daniel Jiang
0a2343f542 Support secret for docker CLI
As CLI does not support oauth flow, we'll use secret for help OIDC user
to authenticate via CLI.
Add column to store secret and token, and add code to support
verify/refresh token associates with secret.  Such that when the user is
removed from OIDC provider the secret will no longer work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-04-10 19:38:11 +08:00
Daniel Jiang
08e00744be Fix misc bugs for e2e OIDC user onboard process
This commit adjust the code and fix some bugs to make onboard process
work.
Only thing missed is that the UI will need to initiate the redirection,
because the request of onboarding a user was sent via ajax call and didn't
handle the 302.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-04-10 19:38:11 +08:00
Yogi_Wang
5df1542c22 Change the login module
change the login module to fix the issue that the nav is being in page when we login harbor again

Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-04-10 14:27:51 +08:00
Meina Zhou
46f78906f7 modify filter to show all the filters in once
Signed-off-by: Meina Zhou <meinaz@vmware.com>
2019-04-10 11:15:40 +08:00
Wenkai Yin
7fcc391fed
Merge pull request #7322 from cd1989/update-with-props
Update only specific props in registry update
2019-04-10 09:44:51 +08:00
Wenkai Yin
2644c52926
Merge pull request #7302 from pureshine/registry-update
Fix replication ui related issues
2019-04-09 20:55:44 +08:00
Wenkai Yin
dba9822477
Merge pull request #7323 from zhoumeina/replication_ng
modify filter to show all the filters in once
2019-04-09 20:54:44 +08:00
cd1989
2ad8913e84 Update only specific props in registry update
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-09 19:57:21 +08:00
FangyuanCheng
e76c287a0c Fix replication ui related issues
Signed-off-by: FangyuanCheng <fangyuanc@vmware.com>
2019-04-09 17:44:42 +08:00
Wenkai Yin
5a65480594 Handle the policy from previous versions
Handle the policy from previous versions

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-09 17:43:07 +08:00
Meina Zhou
4dac7464cb modify filter to show all the filters in once
Signed-off-by: Meina Zhou <meinaz@vmware.com>
2019-04-09 17:28:36 +08:00
Wenkai Yin
580674f3da Merge remote-tracking branch 'upstream/master' into 190409_sync 2019-04-09 17:01:09 +08:00
Wenkai Yin
9e5e723855 Add unit test cases for Harbor adapter
Add unit test cases for Harbor adapter

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-09 13:04:04 +08:00
Wenkai Yin
855c0a2a6e
Merge pull request #7194 from stonezdj/remove_error_msg
Remove error message of saving system setting to db
2019-04-09 12:02:17 +08:00
Wenkai Yin
d72a53aa0c
Merge pull request #7318 from ywk253100/190408_upgrade
Upgrade the replication_job table
2019-04-08 22:43:40 +08:00
Wenkai Yin
4ffa0c3da0 Upgrade the replication_job table
This commit migrates the replication_job table, add one execution record and one task record for each job

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-08 22:23:53 +08:00
stonezdj
e8ab7156bc Remove error message of saving system setting to db
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-04-08 18:16:18 +08:00
Yogi_Wang
eec55f18eb Add OIDC onboard page redirect to Harbor url when we login from OIDC and add redirect page address when we login again after logout
1.when we are the first time we login harbor page when we login from OIDC  ,we add the url in onboard-add-username  page.
2.fix the part of issue that we cannot redirect to de origin page when we login

Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-04-08 16:30:32 +08:00
Wenkai Yin
ee6eb41dae
Merge pull request #7303 from zhoumeina/replication_ng
fix frontend issues
2019-04-08 15:50:21 +08:00
Wenkai Yin
7ff46acd25
Merge pull request #7289 from cd1989/add-registry-ping
Add registry ping API
2019-04-08 14:08:53 +08:00
cd1989
5a2d03593f Add helth check method to registry adapter
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-08 10:03:28 +08:00
Wenkai Yin
462aab5a1e
Merge pull request #7307 from cd1989/registry-healthchecker
Wait randomly before registry health checking
2019-04-08 07:11:49 +08:00
cd1989
f71a110bec Add registry ping API
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-06 17:00:52 +08:00
cd1989
07139684ce Wait randomly before registry health checking
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-05 20:46:29 +08:00
Wenkai Yin
e8fe2aa60c Upgrade the registry and replication policy tables
Upgrade the registry and replication tables in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-05 13:25:00 +08:00
cd1989
fe004e1bfc Init replication in core
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-04 21:58:31 +08:00
cd1989
2450dacecb Use policy controller in registry deletion
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-04 21:26:27 +08:00
Wenkai Yin
4116433de8
Merge pull request #7306 from ywk253100/190404_cleanup
Remove the useless replication code
2019-04-04 21:18:04 +08:00
Wenkai Yin
c2f702be2a Remove the useless replication code
This commit removes the useless replication code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-04 20:56:25 +08:00
Wenkai Yin
58a73de3e5
Merge pull request #7299 from ywk253100/190404_sync
Sync with master branch
2019-04-04 17:33:11 +08:00
Meina Zhou
0028dc0d62 fix frontend issues
Signed-off-by: Meina Zhou <meinaz@vmware.com>
2019-04-04 17:08:59 +08:00
Wenkai Yin
dc35be09c4
Merge pull request #7301 from ywk253100/190404_agent
Filter the events triggerred by replication
2019-04-04 16:50:23 +08:00
Wenkai Yin
02dd4e500a Clear up the table before doing dao test
Clear up the table before doing dao test

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-04 16:46:16 +08:00
Wenkai Yin
1c735a7464 Filter the events triggerred by replication
Filter the events triggerred by replication pull

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-04 16:03:38 +08:00
jwangyangls
ca846ad031
Merge pull request #7297 from jwangyangls/fix_oidc_ui_save
Fix issue of oidc configuration save button is always disabled.
2019-04-04 15:56:06 +08:00
Wenkai Yin
b66b1f341e Merge remote-tracking branch 'upstream/master' into 190404_sync 2019-04-04 14:55:09 +08:00
Wenkai Yin
78fc2846bd Stop a replication execution
Implement the function to support to stop the replication execution

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-04-04 14:51:37 +08:00
Yogi_Wang
311af5d217 Fix issue of oidc configuration save button is always disabled and incorrect character
when we login Harbor width OIDC ,we cannot click the save  button of oidc config page

Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-04-04 14:34:18 +08:00
jwangyangls
a27df46fb6
Merge pull request #7292 from jwangyangls/fix_login_robot_issue
Fix nightly test case failed when login harbor.
2019-04-04 12:14:43 +08:00
Yogi_Wang
4416e1810e fix login harbor issue with test case
Solve the problem that when we log on to the harbor page, the automation test will click on an incorrect button, which results in the test page being unable to log in.

Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-04-04 09:58:46 +08:00
Wenkai Yin
3ed8d87406
Merge pull request #7294 from pureshine/replication-ui
Fix cannot search for replication tasks
2019-04-04 09:52:04 +08:00
Wenkai Yin
48f02d0605
Merge pull request #7291 from cd1989/registry-with-empty-credential
Handle registry with empty credential
2019-04-03 21:35:32 +08:00
Yan
da0e20ec60
Add controller to onboard oidc user (#7286)
Signed-off-by: wang yan <wangyan@vmware.com>
2019-04-03 20:47:22 +08:00
cd1989
e2c86f8f59 Handle registry with empty credential
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-03 20:33:26 +08:00
cd1989
8968e82675 Allow edit registry description
Signed-off-by: cd1989 <chende@caicloud.io>
2019-04-03 19:47:14 +08:00
Fangyuan Cheng
89b72906f4
Merge branch 'master' into move-scan/gc 2019-04-03 16:07:35 +08:00
Meina Zhou
6b3f2bbf74 modify frontend replication UI to apply api changes. use dest registry as object
Signed-off-by: Meina Zhou <meinaz@vmware.com>
2019-04-03 15:45:42 +08:00
FangyuanCheng
8a1f3b7c00 Fix cannot search for replication tasks
Signed-off-by: FangyuanCheng <fangyuanc@vmware.com>
2019-04-03 15:38:18 +08:00
Wenkai Yin
0c171c642b
Merge pull request #7244 from wy65701436/oidc_user
add the dao funcs for OIDC onboard user
2019-04-03 15:23:05 +08:00
FangyuanCheng
82716d492d move vulnerability and gc ui
Signed-off-by: FangyuanCheng <fangyuanc@vmware.com>
2019-04-03 14:39:46 +08:00
jwangyangls
beeb8b53d7
Merge pull request #7281 from jwangyangls/oidc_step3_onboard_tpl
Add the OIDC onboard page
2019-04-03 14:11:25 +08:00