Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-11-25 19:56:09 +01:00
Code Issues Projects Releases Wiki Activity
12,228 Commits 53 Branches 317 Tags 354 MiB
6c394232b6
Commit Graph

1086 Commits

Author SHA1 Message Date
Daniel Jiang
64af09d52b Populate user groups during OIDC CLI secret verification 
This commit refactors the flow to populate user info and verify CLI
secret in OIDC authentication.

It will call the `userinfo` backend of OIDC backend and fallback to
using the ID token if userinfo is not supported by the backend.

It also makes sure the token will be persisted if it's refreshed during
this procedure.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-11-18 23:53:05 -08:00
He Weiwei
0c068d81f5
feat(vuln-severity): map negligible to none to match CVSS v3 ratings (#9885) 
BREAKING CHANGE: the value negligible of severity in project metadata will change to none in the responses of project APIs

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-11-18 14:36:51 +08:00
Wang Yan
6e03c8a54e
Merge pull request #9896 from heww/owner-check-for-project-member-robot-account 
fix(robot,project-member): check owner of member, robot when update, …
 2019-11-15 16:53:22 +08:00
Wang Yan
7b12ed14a1
Merge pull request #9852 from stonezdj/remove_tedious_msg 
Change log level to avoid tedious error in log
 2019-11-15 10:42:28 +08:00
He Weiwei
5bd1cfdbf2 fix(robot,project-member): check owner of member, robot when update, delete 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-11-14 07:00:44 +00:00
Wang Yan
29be93725d
Merge pull request #9860 from reasonerjt/authproxy-case-sensitive-master 
Authproxy case sensitive master
 2019-11-14 14:03:53 +08:00
Daniel Jiang
8933ab8074 Add configuration "case sensitive" to HTTP auth proxy 
This commit make case sensitivity configurable when the authentication
backend is auth proxy.
When the "http_authproxy_case_sensitive" is set to false, the name of
user/group will be converted to lower-case when onboarded to Harbor, so
as long as the authentication is successful there's no difference regardless
upper or lower case is used.  It will be mapped to one entry in Harbor's
User/Group table.
Similar to auth_mode, there is limitation that once there are users
onboarded to Harbor's DB this attribute is not configurable.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-11-13 15:00:05 +08:00
stonezdj
dc5cb3504c Change log level to avoid tedious error in log 
change from error to debug

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-11-13 11:15:00 +08:00
stonezdj
4d822e0a19 Fix review comments on PR9749 
Fix review comments on PR9749
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-11-12 13:05:11 +08:00
Wang Yan
407417ce7b
Merge pull request #9810 from stonezdj/bug9479 
Populate group from auth provider to Harbor when user login
 2019-11-11 19:52:31 +08:00
stonezdj
0c011ae717 Populate group from auth provider to Harbor DB when user login 
Fix #9749, change include LDAP auth, OIDC auth, HTTP auth

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-11-11 14:41:35 +08:00
Daniel Jiang
64dc5122e6 Add role list in project response 
This commit fixes #9771

It compares the roles to return the one with highest permission in the
response of `GET /api/projects`.
In addition to that, it adds the role list to the response, because a
user can have multiple roles in a project.
It also removes the togglable attribute as it's not used anywhere.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-11-11 14:03:50 +08:00
Daniel Jiang
7d04eab63c
Merge pull request #9593 from qyqcswill/code_improve 
promote code quality
 2019-11-08 18:28:46 +08:00
Steven Zou
ee31418e8e revoke scan permission from the developer role 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-11-06 17:57:48 +08:00
Steven Zou
ebc5d2482b do improvements to the scan all job 
- update scan all job to avoid sending too many HTTP requets
- update scan controller to support scan options
- update the db schema of the scan report to introduce requester
- introduce scan all metrics to report the overall progress of scan all job
- fix the status updating bug in scan report
- enhance the admin job status updats
- add duplicate checking before triggering generic admin job
- update the db scheme of admin job

fix #9705
fix #9722
fix #9670

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-11-05 15:12:07 +08:00
He Weiwei
ae8931e816 fix(policy-checker): add func to transform project severity to vuln.Severity 
The severity saved in db is lowercase but the severities in vuln pkg
begin with upper letter, this fix use func to transform project severity
value from db to vuln.Severity.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-10-31 14:11:44 +00:00
He Weiwei
3c80832341 fix(quota): order by quotas only on support resources 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-10-30 02:42:34 +00:00
Daniel Jiang
b17711abbf
Merge pull request #9592 from qyqcswill/code_clean 
remove useless code
 2019-10-29 15:08:59 +08:00
Steven Zou
5b2ab34e03 permission grant for scanner related actions are not correctly 
- add new endpoint for getting scanner candidates of specified project
- adjust the permission granting functions
- fix #9608

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-28 18:20:47 +08:00
Wenkai Yin(尹文开)
f007a62b04
Merge pull request #9588 from stonezdj/fix_ldap_group_sql 
Fix User Group Search SQL error
 2019-10-28 11:22:14 +08:00
hao.cheng
29e905271d promote code quality 
Signed-off-by: hao.cheng <hao.cheng@daocloud.io>
 2019-10-25 15:37:35 +08:00
hao.cheng
94bc8c2f5c remove useless code 
Signed-off-by: hao.cheng <hao.cheng@daocloud.io>
 2019-10-25 15:20:25 +08:00
stonezdj
f402db380b Fix User Group Search SQL error 
User Group Query SQL error in some cases

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-10-25 14:00:45 +08:00
Wang Yan
d18678a48d
Merge pull request #9506 from wy65701436/token-sevice 
Enable robot account to support scan pull case
 2019-10-24 19:52:33 +08:00
wang yan
71c769ec97 remvoe bypass to scanner pull 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-24 17:49:20 +08:00
wang yan
a6ad1b2db8 update code per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-23 20:05:51 +08:00
wang yan
2fa85aefca fix per comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-23 18:45:30 +08:00
wang yan
5996189bb0 update per comments and fix govet error 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-23 18:45:30 +08:00
wang yan
22b4ea0f89 Enable robot account bypass policy check 
1, the commit is for internal robot to bypass policy check, like vul and signature checking.
2, add a bool attribute into registry token, decode it in the harbor core and add the status into request context.
3, add a bool attribut for robot API controller, but API will not use it.y

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-23 18:45:30 +08:00
Wenkai Yin
9d896d4d72 Remove the health checker for Clair in health check API 
As we introduce the pluggable scanner, users can add the external scanners, so we remove the Clair from the health check API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-10-23 12:55:03 +08:00
stonezdj(Daojun Zhang)
4dcd323b4a
Merge pull request #9475 from wy65701436/immu-delete-repo 
add immutable match in the repository/tag delete api
 2019-10-22 17:28:15 +08:00
Wang Yan
fc106e218c
Merge pull request #9503 from heww/issue-9308 
fix(configuration): E notation support for int64 and quota types
 2019-10-22 11:50:06 +08:00
Wang Yan
3772ccc163
Merge pull request #9493 from stonezdj/remove_nested_group 
Remove nested group search
 2019-10-21 17:45:50 +08:00
He Weiwei
7c8f5426ed fix(configuration): E notation support for int64 and quota types 
Closes #9308

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-10-21 08:15:27 +00:00
wang yan
424f11e697 add immutable match in the repository/tag delete api 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-21 15:53:24 +08:00
stonezdj
b148ffe6a8 Remove the nested group search 
Remove the code change in #8378, because the previous code change caused issues: #9092, #9110, #9326

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-10-21 14:34:53 +08:00
He Weiwei
e254fe3095
fix(permissions): permissions checking for member and quota info (#9490) 
1. Only show project member info when has member list permission.
2. Only show quota info when has quota read permission.
3. Add quota read permission for all roles of project.
4. Refactor permission service in portoal.
5. Clear cache when clear session.

Closes #8697

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-10-21 14:03:52 +08:00
He Weiwei
bf6a14c9ad
feat(role): introduce a limited guest role (#9403) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-10-20 14:21:28 +08:00
Wenkai Yin(尹文开)
f98196e5ba
Merge pull request #9435 from reasonerjt/oidc-refresh-refine 
Update OIDC token refresh process
 2019-10-18 19:43:34 +08:00
Steven Zou
0f16913635 rebase: resolve the code confilcts with master 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-17 17:42:41 +08:00
Wenkai Yin(尹文开)
97ddff2ac8
Merge pull request #9434 from heww/clair-adapter 
build(clair): internal clair adapter when install with clair
 2019-10-17 16:06:10 +08:00
He Weiwei
8964a8697a build(clair): internal clair adapter when install with clair 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-10-17 12:00:51 +08:00
Daniel Jiang
f0cb16cb86 Update OIDC token refresh process 
1) Disassociate id token from user session

2) Some OIDC providers do not return id_token in the response of refresh
request:
https://openid.net/specs/openid-connect-core-1_0.html#RefreshTokenResponse
When validating the CLI secret it will not validate the id token,
instead it will check the expiration of the access token, and try to
refresh it.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-10-17 11:26:18 +08:00
Steven Zou
f18afc0a3f do changes to let the vul policy check compatiable with new framework 
- update the scan/scanner controller
- enhance the report summary generation
- do changes to the vulnerable handler
- remove the unused clair related code
- add more UT cases
- update the scan web hook event
- drop the unsed tables/index/triggers in sql schema

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-16 23:15:26 +08:00
stonezdj(Daojun Zhang)
2973ddcf6b
Merge pull request #9428 from stonezdj/disable_self_reg 
Update default self_registration=false
 2019-10-16 17:41:21 +08:00
stonezdj
3636a1afa5 Update default self_registration=false 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-10-16 01:33:48 -07:00
He Weiwei
d9a539807b perf(test): speed up TestAddBlobsToProject test in dao pkg 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-10-14 16:46:01 +00:00
Daniel Jiang
ee9e92b6dd
Merge pull request #9157 from phin1x/master 
Escape user dn in ldap group search filter
 2019-10-14 16:41:27 +08:00
Wenkai Yin(尹文开)
7d0505593f
Merge pull request #8556 from chlins/feat/image-replication-adapter-for-quay.io 
Feat/image replication adapter for quay.io
 2019-10-14 09:16:45 +08:00
Steven Zou
a86afd6ebc Merge branch 'master' into feature/pluggable_scanner_s3_merge 2019-10-12 15:18:06 +08:00
wang yan
6f6f113f0f refactor robot api 
1, add API controller for robot account, make it callable internally
2, add Manager to handler dao releate operation

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-10-11 17:26:18 +08:00
Steven Zou
58afd8e14b [stage3] support pluggable scanner 
- implement scan controller
- add scan resource and update role bindings
- update registration model and related interfaces

Signed-off-by: Steven Zou <szou@vmware.com>

- implement scan API to do scan/get report/get log
- update repository rest API to produce scan report summary
- update scan job hook handler
- update some UT cases

- update robot account making content
- hidden credential in the job log

Commnet scan related API test cases which will be re-activate later
fix #8985

fix the issues found by codacy

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-10-11 12:53:02 +08:00
chlins
4ab3b864ae feat: add image replication adapter for quay.io 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2019-10-11 10:00:07 +08:00
stonezdj(Daojun Zhang)
a2938c5d78
Merge pull request #9274 from wy65701436/immu-refatctor 
refactor immutable dao code to align the new structure under pkg
 2019-10-10 10:38:22 +08:00
He Weiwei
4ce72e37c4 fix(robot): robot account improvement for policies 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-09-27 03:07:58 +00:00
wang yan
7c4fd79b5c refactor immutable dao code to align the new structure under pkg 
1, add manager
2, move model dao to /pkg/dao

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-09-26 20:35:58 +08:00
stonezdj
cc22a175b9 Add immutable tag API 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-09-25 15:53:56 +08:00
stonezdj(Daojun Zhang)
ec559b0585
Merge pull request #9123 from stonezdj/immutable_tags 
Add DAO for immutable tags
 2019-09-23 21:46:07 +08:00
stonezdj
29d2bcce99 Add DAO for immutable tags 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-09-23 16:45:07 +08:00
Fabian
1467f4bbb1 Escape User DN 
Signed-off-by: Fabian Weber <fa.weber@enbw.com>
 2019-09-19 14:29:09 +02:00
Daniel Jiang
b21f9dc6f1 Support OIDC groups 
This commit enable project admin to add group as project member when
Harbor is configured against OIDC as AuthN backend.

It populates the information of groups from ID Token based on the claim
that is set in OIDC settings.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-09-19 17:49:31 +08:00
Daniel Jiang
f36efa4dcd Add groups claim to OIDC configuration 
This commit add the new setting "oidc_groups_claim" to Harbor's
configurations.
And add "group_claim" to OIDCSetting struct.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-09-16 15:54:14 +08:00
Daniel Jiang
753219834e
Merge pull request #8960 from ninjadq/upgrade_hash_alg_for_pswd 
Upgrade hash alg for pswd
 2019-09-12 11:22:39 +08:00
Wenkai Yin
089eb4c449 Add the port 8080 to the default URL of portal to avoid the health check API failure 
We changed the listenning port of portal from 80 to 8080 to run the process as non-root user, but the change didn't update the default URL of portal in source code, this causes the health check API fail.

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-09-11 10:45:55 +08:00
DQ
ea5c27fcd5 Enhance: Upgrade encrypt alg to sha256 
previous sha1 will still used for old password

Signed-off-by: DQ <dengq@vmware.com>
 2019-09-09 21:48:21 +08:00
Wenkai Yin
3b07be5a72 Check the status behind error when trying to update the scan schedule 
Check the status behind error when trying to update the scan schedule

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-09-09 13:31:10 +08:00
Wang Yan
2194834b41
Merge pull request #8910 from heww/foreign-layers 
fix(quota): correct size quota for image with foreign layers
 2019-09-03 00:29:24 +08:00
He Weiwei
f44b75f398 fix(quota): correct size quota for image with foreign layers 
1. Sync blobs from manifest for image with foreign layers.
2. Ignore size of foreign layers when compute size quota.
3. Fix repo info of artifact when upgrade from 1.8 version.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-09-02 14:10:58 +00:00
wang yan
d3f7d01a69 fix int out of range when to set usage in GC job 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-09-02 18:48:10 +08:00
wang yan
c28920c84f fix #8807 
Format the error of mount blob, return a http error so that the core can parse it.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-30 16:29:47 +08:00
wang yan
16b910e1cf fix(quota/sync) #8886 
The foreign layer won't be counted into project quota
NOTE: the foreign layer will be dumped from the registry in the migration

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-29 17:29:40 +08:00
Wang Yan
db5781bf78
Merge pull request #8860 from wy65701436/fix-quota-sync 
fix quota sync issues
 2019-08-29 13:45:38 +08:00
wang yan
5decb56369 update code per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-29 12:46:42 +08:00
Wenkai Yin(尹文开)
5da4286ef4 Hard delete project metadata (#8856) 
Hard delete project metadata

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-29 12:14:39 +08:00
wang yan
942e793f20 fix quota sync issues 
1, fix #8858, add retry to ping backend service
2, fix #8859, split the blobs data when larger then 65535

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-28 18:59:25 +08:00
He Weiwei
2c1c816941
fix(database): generate db url by url.URL for schema upgrade (#8852) 
Closes #7948

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-28 16:59:22 +08:00
Ziming
94138137d5
add valid for rule (#8846) 
Change-Id: I82215a0cf1ec32a253c8db9bfafe7e25b26c9ad9
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-28 16:58:49 +08:00
wang yan
19f543a025 fix sql in remove blob from project 
the project id is missing in the method, that makes GC to clean all of items,
and if quota will not compute twice for the existing manifest.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-28 00:38:11 +08:00
Wenkai Yin(尹文开)
7262cc4c1a
Merge pull request #8836 from wy65701436/update-quota-error 
Revise quota errors to make it more readable
 2019-08-27 11:34:36 +08:00
wang yan
f343b2ec45 Revise quota errors to make it more readable 
1, fix #8802, update the error formet
2, fix #8807, raise the real retag error to UI
3, fix #8832, raise the real chart error to chart client & ut

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-26 17:00:31 +08:00
wang yan
e3155e00d6 fix #8815 :add remove untagged blob record in table project_blob 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-26 15:57:19 +08:00
mmpei
d5f87063e4
Merge branch 'master' into official-wehook-events-20190811 2019-08-22 22:07:12 -05:00
wang yan
2d569192ab fix quota count size for same manifest in different repo 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-23 00:56:45 +08:00
Wenkai Yin(尹文开)
21f8290110
Merge pull request #8777 from heww/issue-8635 
fix(rbac): NewProjectNamespace in rbac only accept projectID
 2019-08-22 17:52:27 +08:00
Wenkai Yin(尹文开)
6198ed2634
Merge pull request #8758 from heww/issue-8681 
refactor(quota,middleware): skip overflow error when subtract resources
 2019-08-22 13:54:01 +08:00
He Weiwei
8effdc6f18 fix(rbac): NewProjectNamespace in rbac only accept projectID 
Closes #8635

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-21 19:37:28 +00:00
wang yan
e91ded65cb fix quota size usage in gc job, issue #https://github.com/goharbor/harbor/issues/8699 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-21 16:51:31 +08:00
Wang Yan
4bccb17236
Merge pull request #8749 from heww/issue-8493 
fix(quota-driver): owner name of project quota reference object
 2019-08-21 13:47:17 +08:00
He Weiwei
c22bf2539e refactor(quota,middleware): skip overflow error when subtract resources 
1. Skip overflow error when subtract resources
2. Take up resources before handle request and put it back when handle
failed for add action in quota interceptor
3. Free resources only after handle success for subtract action in quota
interceptor

Closes #8681

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-20 14:41:55 +00:00
He Weiwei
8eb17be13c fix(quota-driver): owner name of project quota reference object 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-20 07:03:11 +00:00
stonezdj
7c7b6d2710 Normalize LDAP filter for user filter and group filter 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-08-20 10:55:30 +08:00
Daniel Jiang
f10fb67d6d
Merge pull request #8662 from stonezdj/email_sec2 
Set default email to null if not provided
 2019-08-20 09:01:50 +08:00
He Weiwei
75772aae11
refactor(quota): new error types for quota checking (#8726) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-19 19:00:29 +08:00
stonezdj
5fa8eb7854 Set default email to null if not provided 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-08-19 15:20:44 +08:00
Daniel Jiang
b3abd0316b
Merge pull request #8713 from reasonerjt/fix-8702 
Avoid overwriting system CVE whitelist by mistake
 2019-08-19 01:42:58 +08:00
Daniel Jiang
504202ecfd
Merge pull request #8378 from Typositoire/ldap/nested-groups 
Search for LDAP_MATCHING_RULE_IN_CHAIN groups
 2019-08-18 16:07:16 +08:00
Wang Yan
7a41d89ac8 Add quota sync api toi to sync quota data with backend storage 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-16 14:55:46 +08:00
Daniel Jiang
30bb2ddcdf Avoid overwriting system CVE whitelist by mistake 
Fixes #8702
Also enforce the code to mitigate the potential risk.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-08-16 13:28:16 +08:00
Qian Deng
89aed1a1ea
Merge pull request #8672 from ywk253100/190815_content_length 
Set content length when pushing blobs
 2019-08-15 12:45:35 +08:00
Wenkai Yin
b94a99dded Set content length when pushing blobs 
Set content length when pushing blobs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-15 10:52:08 +08:00
Wang Yan
bf0b5a3fd0
Merge pull request #8663 from wy65701436/fix-quota-api 
Fix quota switch fail to get project size
 2019-08-15 10:49:49 +08:00
wang yan
a947a4259d Fix quota switch fail to get project size 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-14 22:32:32 +08:00
He Weiwei
98e1f68468 feat(configuration,db): connection pool configs for db 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-14 14:30:34 +08:00
wang yan
9e0addee55 Enable usage sync when switch quota setting 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-14 12:47:12 +08:00
wang yan
76c52c2332 append commit to fix core compile error introduced by pr #8606 
Signed-off-by: wang yan <wangyan@vmware.com>

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-14 00:22:55 +08:00
Steven Zou
1adc3a9469
Merge pull request #8606 from ywk253100/190807_stuck 
Fix replication tasks stuck in "InProgress" issue
 2019-08-13 15:59:20 +08:00
stonezdj(Daojun Zhang)
3e0191be5a
Merge pull request #8621 from stonezdj/project_sort 
Sort project by name
 2019-08-13 14:13:29 +08:00
He Weiwei
c1cea42089 feat(quota,middleware): enable or disable quota per project by config 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-12 00:02:26 +00:00
peimingming
222c47142a Add chart and scanning event for webhook 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2019-08-11 18:01:07 +08:00
stonezdj
65dc665717 Sort project by name 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-08-09 16:22:55 +08:00
Wang Yan
54a39c7159
Merge pull request #8597 from heww/size-quota 
refactor(quota,middleware): implement size quota by quota interceptor
 2019-08-09 15:44:33 +08:00
He Weiwei
e62c29123d refactor(quota,middleware): implement size quota by quota interceptor 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-08 23:55:54 +00:00
Wang Yan
9cbcc93e8a
Merge pull request #8602 from goharbor/webhook-dev-20190807 
Add feature webhook implementation
 2019-08-08 16:01:39 +08:00
Wenkai Yin
8777c07d47 Fix replication tasks stuck in "InProgress" issue 
Fix replication tasks stuck in "InProgress" issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-08 15:42:42 +08:00
Yann David
6435f32bc5
Prevent duplicated entries 
Signed-off-by: Yann David <davidyann88@gmail.com>
 2019-08-07 13:16:43 -04:00
guanxiatao
e7fafd1941 webhook policy, job, event support 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2019-08-07 20:30:26 +08:00
cd1989
870d7115c4 Refactor code to extract a common task runner 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-07 17:14:10 +08:00
cd1989
e2e540233b Use context for concurrency control 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-07 17:14:10 +08:00
cd1989
1f541c890c Improve performance for other registry adapters 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-07 17:14:10 +08:00
Wenkai Yin(尹文开)
6c0c75743e
Merge pull request #8571 from ywk253100/190806_retention_time 
Populate pull/push time properties to the returning data when listing tags
 2019-08-07 12:41:23 +08:00
Wang Yan
305242e993
Merge pull request #8573 from stonezdj/change_trace_level 
Change trace level of missing configure metadata
 2019-08-07 12:41:00 +08:00
Wenkai Yin
216ef269b3 Populate pull/push time properties to the returning data when listing tags 
Populate pull/push time properties to the returning data when listing tags

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-07 11:47:05 +08:00
stonezdj
05f9920e62 Change trace level of missing metadata 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-08-06 14:09:54 +08:00
Daniel Jiang
eec4fc2798 Remove clair notifier 
The way Harbor handles notification is problematic.
It currently triggers rescan, which will cause problem when there are
lot of images in the registry.
Such as #7316
This commit removes the notifier and we need to revisit the notification
to figure out how to map the notification to a particular image if need
the notification mechanism in future.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-08-06 01:58:15 +08:00
stonezdj(Daojun Zhang)
12fb643f0a
Merge pull request #8557 from stonezdj/merge_user_group_roles 
Merge user roles and group roles
 2019-08-05 17:07:35 +08:00
stonezdj
35a49568ce Merge user roles and group roles 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-08-05 15:10:06 +08:00
Steven Zou
97c812a1e8
Merge pull request #8359 from nlowe/bugfix/logging-line-call-outside-repo-root 
Fix logger line() call if built outside of the repo root
 2019-08-05 14:49:06 +08:00
He Weiwei
9778954852 feat(quota,middleware): image count quota support 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-01 14:48:59 +08:00
He Weiwei
8cc9314984
feat(helm-chart,quota): count quota support for helm chart (#8439) 
* feat(helm-chart,quota): count quota support for helm chart

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-31 16:48:40 +08:00
Wang Yan
0a92e61d97
Merge pull request #8485 from wy65701436/internal-reg-quota 
add internal reg request handler chain
 2019-07-30 20:47:21 +08:00
wang yan
4410cc93f9 add internal reg request handler chain 
this is for internal registry api call, the request should be intercpeted by quota middlerwares, like retag and delete.
Note: The api developer has to know that if the internal registry call in your api, please consider to use
NewRepositoryClientForLocal() to init the repository client, which can handle quota change.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-30 19:39:56 +08:00
Wenkai Yin(尹文开)
9e6b022ce1
Merge pull request #8425 from ywk253100/190726_acr 
Fix #8319, got error when replicating image with Azure container registry
 2019-07-30 15:19:12 +08:00
Wenkai Yin
4dac036013 Fix #8319, got error when replicating image with Azure container registry 
Fix #8319, got error when replicating image with Azure container registry

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-30 12:58:22 +08:00
Daniel Jiang
2211be7a80
Merge pull request #8446 from reasonerjt/group-perm-merge 
Update GetRolesByGroupID
 2019-07-29 19:11:51 +08:00
Daniel Jiang
37b7ab6174 Update GetRolesByGroupID 
This commit fixes #8432
When querying the role of group ID, all matched roles should be returned
instead of the minimal role ID.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-29 11:24:35 +08:00
wang yan
a23ff4e448 Update pull time in artifact table for docker image pull 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-28 12:30:20 +08:00
Wang Yan
b9ea3731f7
Merge pull request #8350 from wy65701436/blob-flow-dev 
Add size middleware to support quota
 2019-07-26 01:25:40 +08:00
Wang Yan
1dfc47d24e Add size middleware to support quota 
[Add]:
1, size middleware for quota size
2, count middleware for quota artifact count

[Support]:
1, put, patch, mount blob
2, put manifest

[Refactor]:
1, Add handle response for middlerware
2, Remove the modifyResponse for registry proxy
3, Use the custom response writer to recored status

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-26 00:28:36 +08:00
He Weiwei
f3a2280033
Merge pull request #8384 from heww/quota-apis 
feat(quota,api): APIs for quotas
 2019-07-25 15:19:46 +08:00
He Weiwei
e625f2aa11 feat(quota,api): APIs for quotas 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-25 13:40:26 +08:00
wang yan
4763864dae merge with latest master code with quota feature branch 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-24 08:47:05 -07:00
Steven Zou
c44747fd3c merge code from master and fix conflicts 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-24 17:27:37 +08:00
Ziming
43c2af9857 map retention with policy (#8313) 
Signed-off-by: Ziming Zhang <zziming@vmware.com>

Implement the API and controller of tag retention
 - API handler
 - retention controller
 - dao
 2019-07-24 17:22:26 +08:00
He Weiwei
ce58c58c01 feat(quota,api): quota support for create project API 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-24 01:02:51 +08:00
Yann David
51eb8bc60f
Search for LDAP_MATCHING_RULE_IN_CHAIN groups 
Signed-off-by: Yann David <davidyann88@gmail.com>
 2019-07-23 12:19:56 -04:00
Wenkai Yin
7362fae7cc Implement a common scheduler 
Implement a common scheduler that can be used globally

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-23 17:20:31 +08:00
wang yan
2292954a31 Merge branch 'master' of https://github.com/goharbor/harbor into project-quota-dev 2019-07-22 15:46:09 +08:00
Wang Yan
e8565a4539
Merge pull request #8335 from reasonerjt/add-oidc-ping-api 
Add API to ping OIDC endpoint
 2019-07-22 14:30:24 +08:00
Wang Yan
834e604ec0
Merge pull request #8246 from ninjadq/fix_chart_museum_500_error 
Fix: Internal server error with messy code when chartmuseum not work
 2019-07-22 11:07:55 +08:00
Nathan Lowe
b4e169db26
Fix logger line() call if built outside of the repo root 
If harbor is built (or `go test`'d) in a different folder than the repo
root, the call to common/utils/log/line(...) will panic with an index
out of range runtime error because the separator can't find `harbor/src`
in the path.

Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-07-21 22:30:17 -04:00
Wenkai Yin
5f1d2bd644 Fix package import cycle issue 
Fix package import cycle issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-19 13:50:55 +08:00
He Weiwei
9c9b8d3a6d Merge branch 'master' into project-quota-dev 2019-07-19 10:02:51 +08:00
Daniel Jiang
96e2e0b145 Add API to ping OIDC endpoint 
This commit adds an API to help admin verify the OIDC endpoint is a
valid one.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-18 19:32:12 +08:00
stonezdj
13772b859e Fix OnBoardGroup issue 
Signed-off-by: stonezdj <stonezdj@gmail.com>

Fix issue when adding a HTTP user group to a project member, returns HTTP 500 error.
 2019-07-18 19:19:09 +08:00
Steven Zou
746d082e2e Merge branch 'master' into feature/tag_retention 2019-07-18 10:40:49 +08:00
Wenkai Yin(尹文开)
a64e089773
Merge pull request #8210 from stonezdj/http_group_dao2 
Add HTTP group support
 2019-07-17 15:22:36 +08:00
DQ
af58195a29 Fix: Internal server error with messy code when chartmuseum not work 
log err when doesn't get data from chart museum

Signed-off-by: DQ <dengq@vmware.com>
 2019-07-17 15:14:50 +08:00
Ziming Zhang
815901ea33 fix 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I3f2d3c7f1e32b4983c31c23d9753f04239e3c82f
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-07-16 19:24:40 +08:00
stonezdj
bb2ae7c093 Add HTTP group feature 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-07-16 15:38:46 +08:00
Ziming Zhang
c22c38994a retention api 
Change-Id: I70f2c34d6bb96ecf4cb5359e2b1ab2dbb99fdbf9
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-07-16 15:06:37 +08:00
Wang Yan
8ac6bdbbb0 Add quota workflow for quota 
1, apply count for manifest if it's a new image
2, insert data for artifact and artifact_blob

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-16 14:48:05 +08:00
wang yan
f066d986b9 merge with latest master code 2019-07-11 20:21:15 +08:00
Wang Yan
b98ca7bf0b
Merge pull request #8237 from wy65701436/redis-locker 
add redis lock
 2019-07-11 20:10:16 +08:00
wang yan
ef14f0cf35 add redis lock, it will be used to lock digest in the quota scenario 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-11 19:24:24 +08:00
Wenkai Yin(尹文开)
3bebf7bc64
Merge pull request #8238 from reasonerjt/project-cve-whitelist 
Enable project level CVE whitelist
 2019-07-10 14:41:01 +08:00
Wang Yan
155b0b0acd
Merge pull request #8175 from heww/quota-manager 
Add manager for quota
 2019-07-10 11:03:57 +08:00
wang yan
6d0271ee5c Merge branch 'master' of https://github.com/goharbor/harbor into project-quota-dev 2019-07-10 10:57:10 +08:00
He Weiwei
41ba410bb2 Manager for quota 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-09 13:59:48 +08:00
wang yan
24c3753581 add dao of artifact 
Signed-off-by: wang yan <wangyan@vmware.com>

Add dao for quota

Signed-off-by: He Weiwei <hweiwei@vmware.com>

fix govet

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-08 23:42:50 +08:00
Daniel Jiang
8f5f0031c7 Enable project level CVE whitelist 
This commit update the project API to support "reuse_sys_cve_whitelist"
setting in project metadata and "cve_whitelist" in project request.
Also modify the interceptor to support project level CVE whitelist if
the reuse flag is false.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-08 18:55:54 +08:00
Daniel Jiang
c296f0ddfb
Merge pull request #8176 from stonezdj/http_group 
Refactor LDAP usergroup
 2019-07-08 09:54:31 +08:00
stonezdj
c0ed55445d Refactor LDAP group 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-07-05 14:44:18 +08:00
Daniel Jiang
88a5572f8e Reload OIDC provider older than 3 seconds 
This commit make sure the OIDC is more actively recreated, to mitigate
the problem in #8177

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-04 14:55:34 +08:00
He Weiwei
4fedfa6580 Add dao for quota 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-04 11:53:26 +08:00
Daniel Jiang
5d887ad0d8
Merge pull request #8179 from reasonerjt/interceptor-use-whitelist 
Apply CVE white list in interceptor
 2019-07-03 15:12:33 +08:00
Daniel Jiang
bba4b2a6a4 Apply CVE white list in interceptor 
Interceptor will filter the vulnerability in whitelist while calculating
the serverity of an image and determine whether or not to block client
form pulling it.

It will use the system level whitelist in this commit, another commit
will switch to project level whitelist based on setting in a project.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-03 14:13:00 +08:00
He Weiwei
720dcc72bd Fix read permission of project member read api 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-02 14:40:46 +08:00
Ziming
af548e915e
Merge branch 'master' into replication_gcr_1.9 2019-06-27 11:27:33 +08:00
Steven Zou
5521b7b7ad
Merge pull request #7915 from bitsf/replication_ecr_1.9 
aws driver for replication
 2019-06-27 11:24:54 +08:00
Daniel Jiang
4aca812ff2 API for system level vulnerability whitelist 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-06-26 23:35:40 +08:00
Ziming Zhang
072bdd101b aws driver for replication 
Change-Id: I8792ffce2eaa5975359bb6159a1ba7b85926a925
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-06-25 19:11:27 +08:00
Ziming Zhang
e387c63242 gcr driver for replication 
Change-Id: I5a6626950d3878bfa9726b332e68bee59159269f
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-06-25 18:08:10 +08:00
wang yan
a4b202d656 remove the id in the post body when to create a robot account 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-06-11 10:47:56 +08:00
wang yan
056cfc7e31 Return account id when to issue a robot 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-22 10:39:26 +08:00
wang yan
2068732eef add validation for robot account registration 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-15 15:03:35 +08:00
Wang Yan
774a9f8d75
Remove unused configure item cfg_expiration (#7744) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-09 22:07:18 +08:00
Daniel Jiang
cbbf2ea973 Redirect regular user to OIDC login page (#7717) 
When the auth mode is OIDC, when a user login via Harbor's login form.
If the user does not exist or the user is onboarded via OIDC, he will be
redirected to the OIDC login page.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-09 10:53:40 +08:00
Wang Yan
095f7b2ff7
add scan all and gc schedule migration (#7628) 
* add scan all and gc schedule migration

Signed-off-by: wang yan <wangyan@vmware.com>

* Fix gofmt errors

Signed-off-by: wang yan <wangyan@vmware.com>

* Update code according to review comments

Signed-off-by: wang yan <wangyan@vmware.com>

* remove convertschedule return name just return value

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-08 19:11:33 +08:00
Daniel Jiang
4118769088 print more sectors of file path in logger 
This would help as we have more and more source files having duplicated
names.
Fixes #7202

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-08 15:49:19 +08:00
Daniel Jiang
c16b44d30b Make sure panic is not thrown when refresh token 
Fixes #7695

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-07 20:30:07 +08:00
Wenkai Yin
d74624d306 Iterate all paginations when listing projects and repositories (#7660) 
Iterate all paginations when listing projects and repositories

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-05-07 13:34:48 +08:00
Wenkai Yin
e64a71d809
Merge pull request #7594 from wy65701436/fix-gc-log 
Fix get log issue of Periodic job
 2019-04-30 10:19:17 +08:00
He Weiwei
37a4f1c982 Remove push+pull action (#7571) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-04-29 15:37:10 +08:00
wang yan
02c7cbeec2 Fix get log issue of Periodic job 
Use the latest error or success execution as the periodic job log

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-29 15:30:05 +08:00
Wenkai Yin
c53d73775a
Merge pull request #7590 from reasonerjt/oidc-wrong-secret-err 
Return more details for error in exchange token
 2019-04-29 14:22:37 +08:00
Wang Yan
c26f655bce
add periodic job UUID to upstream job id and use execution log as the… (#7530) 
* add periodic job UUID to upstream job id and use execution log as the periodic log

Signed-off-by: wang yan <wangyan@vmware.com>

* add comments to fix codacy

Signed-off-by: wang yan <wangyan@vmware.com>

* Update code per comments

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-28 15:09:56 +08:00
Wenkai Yin
7af679af7e
Merge pull request #7567 from reasonerjt/oidc-google-refresh-token 
Persist the new token in DB after login
 2019-04-28 14:12:25 +08:00
Daniel Jiang
15626fcae0 Return more details for error in exchange token 
This commit update the response off OIDC callback when there's error in exchange token.
Additionally add comments to clarify that by default 500 error will not
contain any details.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-28 13:41:53 +08:00
Wenkai Yin
2a463016a9 Upgrade the distribution and notary library (#7516) 
* Return 404 when the log of task doesn't exist

Return 404 when the log of task doesn't exist

Signed-off-by: Wenkai Yin <yinw@vmware.com>

* Upgrade the distribution and notary library

Upgrade the distribution library to 2.7.1, the notary library to 0.6.1

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-28 12:00:26 +08:00
Daniel Jiang
473fed5689 Persist the new token in DB after login 
This commit make sure the token is persist to DB after every time after
a user logs in via OIDC provider, to make sure the secret is usable for
the OIDC providers that don't provide refresh token.

It also updates the authorize URL for google to make sure the refresh
token will be returned.

Also some misc refinement included, including add comment to the
OIDC onboarded user, preset the username in onboard dialog.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-27 23:03:59 +08:00
Wenkai Yin
6511417ba6
Merge pull request #7495 from stonezdj/const_debts 
Replace string with const in metadatalist.go
 2019-04-25 17:41:04 +08:00
stonezdj
504eab56c3 Replace string with const in metadatalist.go 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-04-25 17:01:43 +08:00
Wenkai Yin
7160e411cc
Merge pull request #7498 from ywk253100/190423_docker_hub 
Support replicate public repositories from Docker Hub
 2019-04-24 17:17:23 +08:00
Wenkai Yin
66087aac82
Merge pull request #7493 from stonezdj/tech_debts 
Remove adminserver in sourcecode
 2019-04-24 16:24:59 +08:00
Steven Zou
9bd2de3e35
Merge pull request #7452 from steven-zou/fix_issues_for_jobservice 
Fix issues for jobservice
 2019-04-24 16:15:43 +08:00
Wenkai Yin
5629bf8546 Support replicate public repositories from Docker Hub 
Support replicate the public repositories from Docker Hub without providing the credential

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-24 16:15:31 +08:00
Steven Zou
9bcbe2907b fix go vet issues in the code 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-04-24 07:31:37 +08:00
stonezdj
d7798a12d2 Remove adminserver in sourcecode 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-04-23 15:05:29 +08:00
wang yan
1b4c75af25 Add event into upload ctx 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-23 10:57:31 +08:00
wang yan
df6e0600c9 Fix chart upload issue on event based 
Use chart API to load the uploaded chart file to get the name and version

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-23 10:57:31 +08:00
Steven Zou
3937c8b0dc Merge branch 'master' into fix_issues_for_jobservice 2019-04-22 19:26:51 +08:00
Daniel Jiang
1fdc2e6ba9 Provide API to generate CLI secret 
This commit provide an API to allow a user that is onboarded via OIDC
authn update his CLI secret.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-22 13:34:12 +08:00
Steven Zou
8e734407c0 Merge branch 'master' into fix_issues_for_jobservice 2019-04-19 21:15:21 +08:00
stonezdj(Daojun Zhang)
36d13e8243
Merge pull request #7328 from stonezdj/debts 
Fix issue 6450 Test LDAP server error without save configuration
 2019-04-19 16:51:57 +08:00
Steven Zou
f8feaa192e add get scheduled and periodic executions APIs 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-04-19 13:54:23 +08:00
Wenkai Yin
059b75e97c
Merge pull request #7392 from reasonerjt/oidc-logout 
Handle OIDC user invalidation from OIDC provider.
 2019-04-19 12:46:36 +08:00
Daniel Jiang
239b33c5fb Handle OIDC user invalidation from OIDC provider. 
Ths commmit ensures that when user's token is invalidated OIDC provider, he
cannot access protected resource in Harbor with the user info in his session.
We share the code path with secret verification b/c the refresh token
can be used only once, so it has to be stored in one place.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-19 01:27:31 +08:00
Steven Zou
16f97326ad
Merge pull request #7433 from goharbor/replication_ng 
Merge the replication ng branch to master
 2019-04-18 16:35:45 +08:00
Steven Zou
1f481e492c Refactor job servcie primary logic to fix related bugs 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-04-18 16:02:49 +08:00
stonezdj
41a574e55c Fix issue 6450 Test LDAP server error without save configuration 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-04-18 14:24:21 +08:00
wang yan
ba76550d14 Disable throw internal error to UI 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-18 00:04:19 +08:00
wang yan
e017294f71 merge with master latest 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-17 17:52:39 +08:00
Wang Yan
a6af9e9972
Support well-formatted error returned from the REST APIs. (#6957) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-17 16:43:06 +08:00
wang yan
7a373c2eed Add event trigger to helm upload/deletion replication 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-15 19:02:33 +08:00
Wenkai Yin
c222f18fa7 Update replication 
1. Refine the health check of docker hub
2. Remove the GetNamespace method from adapter interface

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-13 15:20:06 +08:00
Daniel Jiang
f92bc8076d "Skip verify cert" to "verify cert" 
This commit tweaks the attribute for auth proxy mode and OIDC auth mode.
To change it from "Skip verify cert" to "verify cert" so they are more
consistent with other modes.
Additionally it removes a workaround in `SearchUser` in auth proxy
authenticator.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-12 23:25:54 +08:00
Daniel Jiang
763c5df010 Add UT 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-11 15:30:19 +08:00
Wenkai Yin
b73acde051 Support the migration for scheduled replication rule from previous version of Harbor 
Support the migration for scheduled replication rule from previous version of Harbor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-11 13:14:32 +08:00
Daniel Jiang
0d18e6c82f Update according to comments 
For more context see PR #7335

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-10 19:38:12 +08:00
Daniel Jiang
0a2343f542 Support secret for docker CLI 
As CLI does not support oauth flow, we'll use secret for help OIDC user
to authenticate via CLI.
Add column to store secret and token, and add code to support
verify/refresh token associates with secret.  Such that when the user is
removed from OIDC provider the secret will no longer work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-10 19:38:11 +08:00
Daniel Jiang
08e00744be Fix misc bugs for e2e OIDC user onboard process 
This commit adjust the code and fix some bugs to make onboard process
work.
Only thing missed is that the UI will need to initiate the redirection,
because the request of onboarding a user was sent via ajax call and didn't
handle the 302.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-10 19:38:11 +08:00
Wenkai Yin
580674f3da Merge remote-tracking branch 'upstream/master' into 190409_sync 2019-04-09 17:01:09 +08:00
Wenkai Yin
855c0a2a6e
Merge pull request #7194 from stonezdj/remove_error_msg 
Remove error message of saving system setting to db
 2019-04-09 12:02:17 +08:00
Wenkai Yin
d72a53aa0c
Merge pull request #7318 from ywk253100/190408_upgrade 
Upgrade the replication_job table
 2019-04-08 22:43:40 +08:00
Wenkai Yin
4ffa0c3da0 Upgrade the replication_job table 
This commit migrates the replication_job table, add one execution record and one task record for each job

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-08 22:23:53 +08:00
stonezdj
e8ab7156bc Remove error message of saving system setting to db 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-04-08 18:16:18 +08:00
cd1989
5a2d03593f Add helth check method to registry adapter 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-04-08 10:03:28 +08:00
Wenkai Yin
e8fe2aa60c Upgrade the registry and replication policy tables 
Upgrade the registry and replication tables in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-05 13:25:00 +08:00
Wenkai Yin
4116433de8
Merge pull request #7306 from ywk253100/190404_cleanup 
Remove the useless replication code
 2019-04-04 21:18:04 +08:00
Wenkai Yin
c2f702be2a Remove the useless replication code 
This commit removes the useless replication code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-04 20:56:25 +08:00
Wenkai Yin
b66b1f341e Merge remote-tracking branch 'upstream/master' into 190404_sync 2019-04-04 14:55:09 +08:00
Yan
da0e20ec60
Add controller to onboard oidc user (#7286) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-03 20:47:22 +08:00
wang yan
dcf1d704e6 fix dao UT issue and refine the error of onboard OIDC user 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-03 14:05:18 +08:00
wang yan
41018041f7 remove oidc controller and add more UTs 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-03 09:54:21 +08:00
Yan
0de5999f52 add the controller for ocdi onboard user 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-03 09:52:22 +08:00
Wenkai Yin
74efee569e Update the registry client to support pulling public images from docker hub without login 
Only add the authentication info when the username is provided to support pulling public images from docker hub without login

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-01 19:15:07 +08:00
Daniel Jiang
587acd33ad Add callback controller for OIDC 
This commit add callback controller to handle the redirection from
successful OIDC authentication.
For E2E case this requires callback controller to kick off onboard
process, which will be covered in subsequent commits.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-01 12:35:31 +08:00
Wenkai Yin
8c7b63bac2
Merge pull request #7248 from ywk253100/190326_event 
Add event based trigger and scheduled trigger
 2019-03-29 14:58:09 +08:00
Wenkai Yin
4f8e283e8e Add event based trigger and scheduled trigger 
This commit implements the event based trigger and scheduled trigger in replilcation

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-29 13:48:34 +08:00
Daniel Jiang
9ce98f4acd Add controller to handle oidc login 
The controller will redirect user to the OIDC login page based on
configuration.
Additionally this commit add some basic code to wrap `oauth2` package
and `provider` in `go-oidc`, and fixed an issue in UT to make
InMemoryDriver for config management thread-safe.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-03-28 11:29:05 +08:00
Yan
03709e4ec1
add authn proxy (#7199) 
* add authn proxy docker login support

User could use the web hook token issued by k8s api server to login to harbor.
The username should add a specific prefix.

Signed-off-by: wang yan <wangyan@vmware.com>

* update code per review comments

Signed-off-by: wang yan <wangyan@vmware.com>

* Add UT for auth proxy modifier

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-03-27 12:37:54 +08:00
Wenkai Yin
017bba8dc1 Merge remote-tracking branch 'upstream/master' into 190327_sync 2019-03-27 11:43:51 +08:00
Daniel Jiang
49aae76205 Onbard settings for OIDC provider (#7204) 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-03-25 12:24:39 +08:00
Yan
8d3946a0e2
Refactor scan all api (#7120) 
* Refactor scan all api

This commit is to let scan all api using admin job to handle schedule
management. After the PR, GC and scan all share unified code path.

Signed-off-by: wang yan <wangyan@vmware.com>

* update admin job api code according to review comments

Signed-off-by: wang yan <wangyan@vmware.com>

* Update test code and comments per review

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-03-22 17:52:21 +08:00
Wenkai Yin
49cf50adb1 Merge remote-tracking branch 'upstream/master' into 190324_sync 
Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-22 15:55:52 +08:00
Meina Zhou
130e132f86 Merge branch 'master' into replication_ng 
Signed-off-by: Meina Zhou <meinaz@vmware.com>
 2019-03-21 14:16:33 +08:00
He Weiwei
79235fffd1 Fix pagination for users and users search apis 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-03-21 13:03:30 +08:00
Wenkai Yin
fb394c2c7a Replicate helm charts 
This commit provides the capability for Harbor to replicate helm charts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-20 00:35:15 +08:00
stonezdj(Daojun Zhang)
7060747d5b ldap_url and ldap_base_dn not exist in user config (#7115) 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-03-15 15:20:12 +08:00
wang yan
73d68903d6 update robot account return attribute 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-03-14 13:57:50 +08:00
Wenkai Yin
258b22a9a5 Fix bug in replication 
This commit fixes bugs found in the implement of replciation NG

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-10 20:57:59 +08:00
Wenkai Yin
cabef73980 Add Harbor adapter for replication 
Implement the replication adapter for Harbor registry

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-13 21:39:39 +08:00
peimingming
4efad287ce Add execution and hooks 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2019-03-13 09:35:01 +08:00
Wenkai Yin
772367498f Merge remote-tracking branch 'upstream/master' into 190311_sync 2019-03-11 20:34:49 +08:00
Frank Kung
5bd5d59a4f 1. Define ng persist replication policy model. 
2. Add ng replication policy CURD methods.
3. Implement ng policy manger.

Signed-off-by: Frank Kung <kfanjian@gmail.com>
Signed-off-by: 慕薇疯魔 <kfanjian@gmail.com>
 2019-03-11 11:13:10 +08:00
Wenkai Yin
ec2a7f9239 Implement replication operation API 
This commit implements the replication operation related APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-08 10:06:33 +08:00
stonezdj(Daojun Zhang)
f7745baf30
Merge pull request #6599 from stonezdj/pr6161 
Add new parameter ldap_group_membership_attribute (PR#6161)
 2019-03-07 13:26:26 +08:00
De Chen
2bc2a44db8
Merge branch 'replication_ng' into registries-management 2019-03-05 16:22:34 +08:00
cd1989
b00098d492 Add unit tests and fix CI 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-03-05 15:37:36 +08:00
stonezdj
4dfee0c1f0 Remove verify_remote_cert 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-03-05 14:04:10 +08:00
stonezdj
cf134bc80e Add new parameter ldap_group_membership_attribute 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-03-03 10:03:22 +08:00
Daniel Jiang
321874c815 Move Settings of HTTP auth proxy (#7047) 
Previously the settings of HTTP authproxy were set in environment
variable.
This commit move them to the configuration API

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-03-01 14:11:14 +08:00
cd1989
8732a20709 Rewrite registry manager with new interface 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-02-27 11:54:04 +08:00
cd1989
6bdf3053a7 Implement registries manager 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-02-27 11:54:04 +08:00
Wenkai Yin
95888b3dc2
Merge branch 'replication_ng' into 190130_transfer_repo 2019-02-27 11:00:42 +08:00
wang yan
91aa67a541 Update expiration variable name to expiresat/tokenduration 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-02-25 11:55:42 +08:00
wang yan
36a778b482 Update expiration schema to bigint and default unit to minute 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-02-22 18:42:43 +08:00
wang yan
47a09b5891 add expiration of robot account 
This commit is to make the expiration of robot account configurable

1, The expiration could be set by system admin in the configuation page or
by /api/config with robot_token_expiration=60, the default value is 30 days.
2, The expiration could be shown in the robot account infor both on UI and API.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-02-22 18:42:34 +08:00
stonezdj(Daojun Zhang)
4cb49e5388
Merge pull request #6963 from stonezdj/remove_container 
Remove everything of adminserver container
 2019-02-22 18:27:43 +08:00
stonezdj
0cba36d79f Remove everything of adminserver 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-02-22 16:34:39 +08:00
Nguyen Quang Huy
eda6c47b3e add signoff for DCO gate (#6981) 
Some variable name, function name is colliding with builtin function.

Signed-off-by: Nguyen Quang Huy <huynq0911@gmail.com>
 2019-02-22 15:00:18 +08:00
Daniel Jiang
321adc8362
Merge pull request #6941 from ywk253100/190213_replication_policy 
Fix #6698: cannot create a same name replication policy after deleting it
 2019-02-21 16:03:55 +08:00
stonezdj
7a5fbf718f Revise code with review comments 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-02-18 15:20:54 +08:00
stonezdj
36e1c13a43 fix ut error in systeminfo_test.go 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-02-18 14:06:19 +08:00
stonezdj
1ae5126bb4 Refactor adminserver stage 3: replace config api and change ut settings 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-02-18 14:06:19 +08:00
Wenkai Yin
f0f2e77fb4 Implement the repository transfer 
This commit implements the Transfer interface for resource repository

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-02-15 17:49:35 +08:00
Wenkai Yin
530ba1d27b Fix #6698 
This commit fixes the issue #6698: cannot create a same name replication policy after deleting it

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-02-15 15:17:48 +08:00
Daniel Jiang
81639e2110
Merge pull request #6865 from wy65701436/remove-token 
Remove the token attribute from robot table
 2019-02-13 19:23:06 +08:00
Yan
e9556a4cec
Add post response for robot account API (#6906) 
This commit is to do:
1, Add post response on creating robot account
2, Lower-case the attribute of response

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-02-13 14:40:04 +08:00
He Weiwei
1c4b9aa346 Protect API using rbac 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-02-01 18:55:06 +08:00
wang yan
5d6a28d73e Remove the token attribute for robot table 
This commit is to remove the token attribute as harbor doesn't store the token in DB.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-30 23:56:23 +08:00
wang yan
f4f4535304 Fix action and resouce of RBAC change 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-29 17:05:15 +08:00
Daniel Jiang
bf663df0e7
Merge pull request #6820 from wy65701436/robot-service 
Add robot account authn & authz implementation
 2019-01-29 16:08:25 +08:00
He Weiwei
6e95b98108 Standard actions for rbac 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-29 11:59:11 +08:00
He Weiwei
1da0a66fe5
Merge pull request #6781 from heww/user-permissions-api 
Implement api for get current user permissions
 2019-01-29 01:58:51 +08:00
He Weiwei
0ab7c93e16 Replace casbin builtin keyMatch2 with custom match func 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-29 01:26:38 +08:00
wang yan
2d7ea9c383 update codes per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-28 21:26:06 +08:00
He Weiwei
8b5e68073d Implement api for get current user permissions 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-28 18:06:52 +08:00
Yan
71f37fb820 * Add robot account authn & authz implementation. 
This commit is to add the jwt token service, and do the authn & authz for robot account.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-28 17:39:57 +08:00
Daniel Jiang
20db0e737b Provide HTTP authenticator 
An HTTP authenticator verifies the credentials by sending a POST request
to an HTTP endpoint.  After successful authentication he will be
onboarded to Harbor's local DB and assigned a role in a project.

This commit provides the initial implementation.
Currently one limitation is that we don't have clear definition about
how we would "search" a user via this HTTP authenticator, a flag for
"alway onboard" is provided to skip the search, otherwise, a user has
to login first before he can be assigned a role in Harbor.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-01-28 15:43:44 +08:00
He Weiwei
3f8e06a8bc Support master role for project member create and update apis (#6780) 
* Support master role for project member create and update apis

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* Fix description for role_id in swagger.yaml

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-23 14:56:23 +08:00
He Weiwei
ae061482ae Add Can method to securty.Context interface (#6779) 
* Add Can method to securty.Context interface

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* Improve mockSecurityContext Can method

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-23 14:32:37 +08:00
wang yan
903e15235e Update validation and error message per comments 2019-01-17 15:33:05 +08:00
wang yan
4cde11892a update the conflict check with DB unique constrain error message 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-17 13:13:55 +08:00
Yan
1af0f3c3b9 Add API implementation of robot account 
Add API implementation of robot account

1. POST /api/project/pid/robots
2, GET /api/project/pid/robots/id?
3, PUT /api/project/pid/robots/id
4, DELETE /api/project/pid/robots/id

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-17 13:13:55 +08:00
He Weiwei
8dab10bbed
Merge pull request #6765 from heww/rename-ram 
Rename ram to rbac
 2019-01-17 11:50:14 +08:00
He Weiwei
bacfe64979 Rename ram to rbac 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-16 18:20:30 +08:00
Wenkai Yin
f8d9653419
Merge pull request #6737 from ywk253100/190109_health_check 
Implement the unified health check API
 2019-01-16 18:14:14 +08:00
He Weiwei
76bee7a9fc
Merge pull request #6710 from heww/security-by-ram 
Implement current security interfaces using ram
 2019-01-16 17:47:13 +08:00
Wenkai Yin
be4455ec1b Implement the unified health check API 
The commit implements an unified health check API for all Harbor services

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-01-16 17:21:04 +08:00
He Weiwei
ebd26c0105 Implement current security interfaces using ram 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-16 16:08:17 +08:00
Steven Zou
464bdf71cd
Merge pull request #6727 from wy65701436/robot-dao 
Add dao of robot account
 2019-01-14 19:34:23 +08:00
wang yan
d349c256e8 add support for query nil 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-11 16:19:42 +08:00
Daniel Jiang
a1d4bfd332
Merge pull request #6344 from reasonerjt/bump-up-golang 
Bump up golang to 1.11.2
 2019-01-11 16:15:59 +08:00
Daniel Jiang
5d59d6fab8 Bump up golang to 1.11.2 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-01-11 14:44:32 +08:00
wang yan
6bd6fbd4ad Add fuzzy match and delete funt per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-11 14:26:49 +08:00
wang yan
c6ae1388ec Add dao of robot account 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-10 14:51:33 +08:00
Daniel Jiang
80af81154c
Merge pull request #6702 from wy65701436/robot-db-scheme 
Add DB table for robot account
 2019-01-10 14:25:58 +08:00
wang yan
db09f9f101 Update token length and upper case the sql key words 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-09 10:00:54 +08:00
wang yan
362a0638d0 Add DB table for robot account 
This commit is to add DB scheme for robot account and update the db orm releated.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-08 18:46:16 +08:00
Daniel Jiang
b5788f0695
Merge pull request #6671 from heww/ram 
Add ram pkg
 2019-01-08 15:39:36 +08:00
He Weiwei
79f786ecbe Add ram pkg 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-04 13:17:13 +08:00
Wenkai Yin
75d45ebd9d
Merge pull request #6547 from cd1989/retag-input-validation 
Validate repo and tag names in retag
 2019-01-03 17:45:44 +08:00
cd1989
c117a23133 Validate repo and tag names in retag 
Signed-off-by: cd1989 <chende@caicloud.io>
 2018-12-24 16:49:39 +08:00
Daniel Jiang
93c0a18b06
Merge pull request #6537 from stonezdj/ref_admin_driver 
Refactor config settings stage2
 2018-12-21 15:12:56 +08:00
stonezdj
2446878f6b Refactor config settings stage2 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2018-12-21 10:59:11 +08:00
Wenkai Yin
b28bca7af4
Merge pull request #6541 from salkin/proxy-transport 
Add support for http proxy in transport
 2018-12-18 15:46:29 +08:00
Niklas Wik
138bc69f0f Add support for http proxy in transport 
Signed-off-by: Niklas Wik <niklas.wik@nokia.com>
 2018-12-17 10:35:27 +02:00
stonezdj(Daojun Zhang)
13511d74ed Refactor config settings encrypt + metadata (#6387) 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2018-12-12 12:14:33 +08:00
Wenkai Yin
f7a28ee2a2 Remove the duplicate http error struct (#6516) 
There are two different types to represent http error in the current code. This commit updates the codes to keep only one.

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2018-12-12 11:51:19 +08:00
cd1989
caf07a96fe Give meaningful messages when retag forbided 
Signed-off-by: cd1989 <chende@caicloud.io>
 2018-12-06 16:25:21 +08:00
Daniel Jiang
ae240df031 Remove the Scan all in-memory marker (#6399) 
Previously there was a in-memory marker to prevent user from frequently
calling the "scan all" API.  This has become problematic in HA
deployment, and is no longer needed after enhancement in jobservice.

This commit removes the marker for "scan all" api, however, we need to
review the mechanism and rework to make it stateless.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2018-12-02 15:40:50 +08:00
Steven Zou
ec2ad4d0b8
Merge pull request #6093 from cd1989/replication-record-id 
Add op uuid to image replication
 2018-11-30 14:54:43 +08:00
Wenkai Yin
9d5cf57373 Check the existence of name when creating replication rule and fix bugs in testing library (#6381) 
1. Fix #5102 by checking the existence of name when creating/editing replication rule
2. Add unique constraint to the name of replication policy and target
3. Fix bugs of testing library

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2018-11-30 13:32:20 +08:00
peimingming
238dbc0347 Add UT and review comments and issue fix (#6144) 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2018-11-28 17:43:14 +08:00
peimingming
c67fdc40f5 Support store job log in DB (#6144) 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2018-11-28 15:09:29 +08:00
Steven Zou
e6d4c024ee Update README of job service to reflect latest updates 
Signed-off-by: Steven Zou <szou@vmware.com>
 2018-11-08 10:35:12 +08:00
Steven Zou
7b106d06c5 Build logger framework to support configurable loggers/sweepers/getters 
Signed-off-by: Steven Zou <szou@vmware.com>
 2018-11-06 09:31:31 +08:00
Daniel Jiang
39b4d011c7 Not submit scan all job when core container starts 
Fixes #6115

As for the change in migration sql file, in 1.7 we'll switch to
jobservice for scheduling "scan all" job.  To avoid inconsistency,
this item will be reset and user will need to configure the policy again.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2018-10-25 19:01:52 +08:00
Steven Zou
3b76a960e1
Merge pull request #6039 from stonezdj/refact_5996 
Refactor capacity
 2018-10-24 10:50:11 +08:00
陈德
1ffd9d8fba Add op uuid to image replication 
Signed-off-by: 陈德 <chende@caicloud.io>
 2018-10-21 23:55:57 +08:00
Steven Zou
db24cbe25a
Merge pull request #5779 from cd1989/images-retag 
Merge Images retag
 2018-10-19 11:04:48 +08:00
Daniel Jiang
fe2e58e1a0 Ignore duplication error when inserting config 
This commit mitigates the situation when more then one adminserver is
deployed and there may be duplication error when they try to initialize
the configuration to DB.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2018-10-19 10:23:00 +08:00
陈德
a1b4729aa7 Add more unit tests 
Signed-off-by: 陈德 <chende@caicloud.io>
 2018-10-18 00:26:25 +08:00
stonezdj
0278981523 Change admin server to core in jobservice 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2018-10-16 19:23:12 +08:00
stonezdj(Daojun Zhang)
b764033fc9
Merge pull request #6007 from stonezdj/refact_5998 
Change admin server to core in jobservice
 2018-10-15 17:52:24 +08:00
stonezdj
79bac7a64e Change admin server to core in jobservice 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2018-10-15 14:56:18 +08:00
Daniel Jiang
00c8344c13 Remove the local scheduler 
This is no longer needed after moving the "scan all" to job-service.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2018-10-15 14:14:11 +08:00
Wenkai Yin
83147b1982
Merge pull request #6003 from wy65701436/fix-gc-bug 
Fix gc api issues
 2018-10-11 10:26:38 +08:00
Daniel Jiang
1188bd89b9 Use secure transport to access HTTP endpoint 
In various parts of the code, we used insecure transport in http Client
when we assume the endpoint is http.  This causes complaints form
security scanner.  We should use secure transport in such cases.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2018-10-10 17:51:02 +08:00