Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-11-18 16:25:16 +01:00
Code Issues Projects Releases Wiki Activity
8,230 Commits 54 Branches 316 Tags 351 MiB
a087ba02e3
Commit Graph

585 Commits

Author SHA1 Message Date
Ziming Zhang
3b62addc76 check rule conflict 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I8af648f22aa3fe76024240d322a7fdae560e3fbd
 2019-08-07 15:38:21 +08:00
ethan
f463b7c449 project_test.go: many typos "respose" fixup 
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
 2019-08-07 13:39:05 +08:00
Wenkai Yin
216ef269b3 Populate pull/push time properties to the returning data when listing tags 
Populate pull/push time properties to the returning data when listing tags

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-07 11:47:05 +08:00
Wenkai Yin(尹文开)
556e07f0c4
Merge pull request #8551 from nlowe/feat/retention/GH-8548-Pulled-in-the-last-N-Days 
Retention: New Evaluator: Pulled within the last N Days
 2019-08-07 11:37:46 +08:00
Qian Deng
635066da92
Merge pull request #8466 from reasonerjt/remove-clair-notification 
Remove clair notifier
 2019-08-07 10:53:35 +08:00
Nathan Lowe
9a7df265ce
Retention: New Evaluator: Pulled within the last N Days 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-08-06 22:28:28 -04:00
Wenkai Yin(尹文开)
ec4fa753d7
Merge pull request #8552 from nlowe/feat/retention/GH-8549-Pushed-in-the-last-N-Days 
Retention: New Evaluator: Retain Images pushed within N days
 2019-08-07 09:55:05 +08:00
ethan
edd102c3b7
core/main.go: logging message correction 
Signed-off-by: ethan <guangming.wang@daocloud.io>
 2019-08-06 21:56:18 +08:00
Ziming Zhang
498a813299 retain nothing rule 
Change-Id: I4e7a4ecb40fe39b80e41a6d9bf8b5fb3968a41af
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-06 15:10:27 +08:00
Nathan Lowe
756352d271
Add metadata for daysps to the API for the UI 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-08-05 20:08:34 -04:00
Daniel Jiang
eec4fc2798 Remove clair notifier 
The way Harbor handles notification is problematic.
It currently triggers rescan, which will cause problem when there are
lot of images in the registry.
Such as #7316
This commit removes the notifier and we need to revisit the notification
to figure out how to map the notification to a particular image if need
the notification mechanism in future.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-08-06 01:58:15 +08:00
Ziming Zhang
3a80123a82 1. remove rule always 
2. update i18n

Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I0e2394f0a6b283d0efef7a44ed2d4afb9745eabd
 2019-08-05 17:59:00 +08:00
Ziming Zhang
026aee75d9 add task retain num 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I2f8b89454fe3bb9b56af237048c9e2b90783f434
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-05 17:57:09 +08:00
Ziming Zhang
decffdd6a4 add total page num for tag retention 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I58a250dbb643f6949c1e34aa3a84a01dc3e0b285
 2019-08-02 14:43:52 +08:00
He Weiwei
9778954852 feat(quota,middleware): image count quota support 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-01 14:48:59 +08:00
He Weiwei
8cc9314984
feat(helm-chart,quota): count quota support for helm chart (#8439) 
* feat(helm-chart,quota): count quota support for helm chart

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-31 16:48:40 +08:00
wang yan
4410cc93f9 add internal reg request handler chain 
this is for internal registry api call, the request should be intercpeted by quota middlerwares, like retag and delete.
Note: The api developer has to know that if the internal registry call in your api, please consider to use
NewRepositoryClientForLocal() to init the repository client, which can handle quota change.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-30 19:39:56 +08:00
Ziming
a5c31d7160
Merge pull request #8456 from ywk253100/190729_scheduler 
Register the scheduler job and fix bugs
 2019-07-30 08:57:10 +08:00
Steven Zou
4bf7f7b3e4
Merge pull request #8445 from steven-zou/fix/tag_retention 
refactor index registering processes
 2019-07-30 07:58:14 +08:00
Wenkai Yin
94c1cf8ac9 Register the scheduler job and fix bugs 
1. Register the scheduler job
2. Fix retention bugs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-30 07:37:47 +08:00
Wenkai Yin(尹文开)
c99f12fe33
Merge pull request #8444 from bitsf/tag_retention_exec_status 
get execution status on the fly
 2019-07-29 17:08:37 +08:00
Wang Yan
ca20281b35
Merge pull request #8438 from wy65701436/fix-quota-bug 
Fix logic error in quota middlewware on handling failure response
 2019-07-29 16:02:57 +08:00
Ziming Zhang
ba47b4c00f get execution status on the fly 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Iefcf8946d7a2c7a27bc22bd326ee9723b4b79c66
 2019-07-29 14:48:39 +08:00
Steven Zou
6479a22a08 refacor index registering processes 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-29 14:01:47 +08:00
wang yan
1975f4a71d Fix logic error in quota middlewware on handling failure response 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-28 12:37:58 +08:00
wang yan
a23ff4e448 Update pull time in artifact table for docker image pull 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-28 12:30:20 +08:00
Ziming Zhang
83b045f5ec add ut for tag retention controller 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I1469ee13675537ec389a068e4bc29e457b402fa4
 2019-07-26 17:55:11 +08:00
Wenkai Yin(尹文开)
d45674960f
Merge pull request #8417 from goharbor/project-quota-dev 
Add feature project quota dev
 2019-07-26 15:41:09 +08:00
wang yan
54101180ae Update log level for init proxy 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-26 13:39:06 +08:00
He Weiwei
c566a48880 fix(api,project): fix repo, chart count missing in project summary 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-26 13:15:44 +08:00
wang yan
676b922c95 merge with latest master code with quota branch 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-26 01:32:45 +08:00
Wang Yan
b9ea3731f7
Merge pull request #8350 from wy65701436/blob-flow-dev 
Add size middleware to support quota
 2019-07-26 01:25:40 +08:00
Wang Yan
1dfc47d24e Add size middleware to support quota 
[Add]:
1, size middleware for quota size
2, count middleware for quota artifact count

[Support]:
1, put, patch, mount blob
2, put manifest

[Refactor]:
1, Add handle response for middlerware
2, Remove the modifyResponse for registry proxy
3, Use the custom response writer to recored status

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-26 00:28:36 +08:00
Wenkai Yin
2e9521ad45 Support to stop one execution of retention 
Support to stop one execution of retention

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-25 16:48:00 +08:00
He Weiwei
f3a2280033
Merge pull request #8384 from heww/quota-apis 
feat(quota,api): APIs for quotas
 2019-07-25 15:19:46 +08:00
He Weiwei
e625f2aa11 feat(quota,api): APIs for quotas 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-25 13:40:26 +08:00
Steven Zou
584dcd8571 support dry run of retention 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-25 12:53:19 +08:00
wang yan
4763864dae merge with latest master code with quota feature branch 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-24 08:47:05 -07:00
Steven Zou
c44747fd3c merge code from master and fix conflicts 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-24 17:27:37 +08:00
Wenkai Yin(尹文开)
cd411f6588
Merge pull request #8385 from ywk253100/190724_task_hook 
Implement the webhook handler for retention task
 2019-07-24 17:22:57 +08:00
Ziming
43c2af9857 map retention with policy (#8313) 
Signed-off-by: Ziming Zhang <zziming@vmware.com>

Implement the API and controller of tag retention
 - API handler
 - retention controller
 - dao
 2019-07-24 17:22:26 +08:00
Wenkai Yin
e2808f12cf Implement the webhook handler for retention task 
Implement the webhook handler for retention task

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-24 14:32:05 +08:00
He Weiwei
ce58c58c01 feat(quota,api): quota support for create project API 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-24 01:02:51 +08:00
Wenkai Yin
7362fae7cc Implement a common scheduler 
Implement a common scheduler that can be used globally

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-23 17:20:31 +08:00
wang yan
2292954a31 Merge branch 'master' of https://github.com/goharbor/harbor into project-quota-dev 2019-07-22 15:46:09 +08:00
Wang Yan
e8565a4539
Merge pull request #8335 from reasonerjt/add-oidc-ping-api 
Add API to ping OIDC endpoint
 2019-07-22 14:30:24 +08:00
Wang Yan
834e604ec0
Merge pull request #8246 from ninjadq/fix_chart_museum_500_error 
Fix: Internal server error with messy code when chartmuseum not work
 2019-07-22 11:07:55 +08:00
Steven Zou
deb021eb48 rebase and fix name conflicts among packages and vars 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-19 22:37:54 +08:00
Wenkai Yin
ddbde72d2e Move the pkg manager/controller declaration to the api package 
Move the pkg manager/controller declaration to the api package to avoid the dependency cycle

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-19 17:21:58 +08:00
Steven Zou
f0ea62caa9 Merge branch 'feature/tag_retention' into feature/tag_retention-performer 2019-07-19 16:00:13 +08:00
Steven Zou
c2b5d95e02 implement retain action performer 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-19 15:59:21 +08:00
Wenkai Yin
5f1d2bd644 Fix package import cycle issue 
Fix package import cycle issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-19 13:50:55 +08:00
He Weiwei
9c9b8d3a6d Merge branch 'master' into project-quota-dev 2019-07-19 10:02:51 +08:00
Daniel Jiang
96e2e0b145 Add API to ping OIDC endpoint 
This commit adds an API to help admin verify the OIDC endpoint is a
valid one.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-18 19:32:12 +08:00
stonezdj
13772b859e Fix OnBoardGroup issue 
Signed-off-by: stonezdj <stonezdj@gmail.com>

Fix issue when adding a HTTP user group to a project member, returns HTTP 500 error.
 2019-07-18 19:19:09 +08:00
Steven Zou
746d082e2e Merge branch 'master' into feature/tag_retention 2019-07-18 10:40:49 +08:00
Wenkai Yin(尹文开)
a64e089773
Merge pull request #8210 from stonezdj/http_group_dao2 
Add HTTP group support
 2019-07-17 15:22:36 +08:00
DQ
af58195a29 Fix: Internal server error with messy code when chartmuseum not work 
log err when doesn't get data from chart museum

Signed-off-by: DQ <dengq@vmware.com>
 2019-07-17 15:14:50 +08:00
Wenkai Yin
d6c6231e08 Implement the retention client 
Implement the retention client

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-17 14:04:29 +08:00
stonezdj
bb2ae7c093 Add HTTP group feature 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-07-16 15:38:46 +08:00
Ziming Zhang
c22c38994a retention api 
Change-Id: I70f2c34d6bb96ecf4cb5359e2b1ab2dbb99fdbf9
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-07-16 15:06:37 +08:00
Wang Yan
8ac6bdbbb0 Add quota workflow for quota 
1, apply count for manifest if it's a new image
2, insert data for artifact and artifact_blob

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-16 14:48:05 +08:00
wang yan
f066d986b9 merge with latest master code 2019-07-11 20:21:15 +08:00
Wenkai Yin
91b050a01b Implement the launcher 
The commit implements the launcher for tag retention

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-11 17:18:46 +08:00
Wenkai Yin(尹文开)
3bebf7bc64
Merge pull request #8238 from reasonerjt/project-cve-whitelist 
Enable project level CVE whitelist
 2019-07-10 14:41:01 +08:00
wang yan
6d0271ee5c Merge branch 'master' of https://github.com/goharbor/harbor into project-quota-dev 2019-07-10 10:57:10 +08:00
wang yan
7b38389898 update codes per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>

fix middlewares per review comments
1, add scheme1 and scheme2 check
2, change MustCompile to Compile

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-09 11:08:19 +08:00
wang yan
b3c5137a2f add copyright and fix codecy 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-09 11:08:19 +08:00
wang yan
57821b1b4c Refactor interceptors code with chain 
1, add a blob inteceptors for quota usage
2, add a manifest inteceptors for quota usage

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-09 11:08:11 +08:00
Daniel Jiang
8f5f0031c7 Enable project level CVE whitelist 
This commit update the project API to support "reuse_sys_cve_whitelist"
setting in project metadata and "cve_whitelist" in project request.
Also modify the interceptor to support project level CVE whitelist if
the reuse flag is false.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-08 18:55:54 +08:00
Daniel Jiang
c296f0ddfb
Merge pull request #8176 from stonezdj/http_group 
Refactor LDAP usergroup
 2019-07-08 09:54:31 +08:00
stonezdj
c0ed55445d Refactor LDAP group 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-07-05 14:44:18 +08:00
Daniel Jiang
8a9d352f54 Handle helm push in OIDC filter 
Fixes #8130
Enable OIDC filter to handle requests to /api/chartrepo/*

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-03 17:29:37 +08:00
Daniel Jiang
5d887ad0d8
Merge pull request #8179 from reasonerjt/interceptor-use-whitelist 
Apply CVE white list in interceptor
 2019-07-03 15:12:33 +08:00
Daniel Jiang
bba4b2a6a4 Apply CVE white list in interceptor 
Interceptor will filter the vulnerability in whitelist while calculating
the serverity of an image and determine whether or not to block client
form pulling it.

It will use the system level whitelist in this commit, another commit
will switch to project level whitelist based on setting in a project.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-03 14:13:00 +08:00
Daniel Jiang
6f166bc02c
Merge pull request #8154 from markpeek/markpeek-registry-health-check 
Switch registry health check to a 200 response url
 2019-07-03 10:29:35 +08:00
He Weiwei
720dcc72bd Fix read permission of project member read api 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-02 14:40:46 +08:00
mmpei
5dfc3f2402 Optimize fetch manifest loop when handling notification 
Signed-off-by: mmpei <peimingming@corp.netease.com>
 2019-07-01 17:54:52 +08:00
Steven Zou
5521b7b7ad
Merge pull request #7915 from bitsf/replication_ecr_1.9 
aws driver for replication
 2019-06-27 11:24:54 +08:00
Wenkai Yin(尹文开)
fce920bbee
Merge pull request #8075 from reasonerjt/sys-vuln-whitelist-api 
API for system level vulnerability whitelist
 2019-06-27 10:53:09 +08:00
Mark Peek
3cae31da54 Switch registry health check to a 200 response url 
The health check for the registry was using "/v2" which returned an
unauthorized response and put additional errors in the logs. Switch
to using "/" which returns an OK response with reduced logging.

Signed-off-by: Mark Peek <markpeek@vmware.com>
 2019-06-26 14:23:08 -07:00
Daniel Jiang
4aca812ff2 API for system level vulnerability whitelist 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-06-26 23:35:40 +08:00
Ziming Zhang
072bdd101b aws driver for replication 
Change-Id: I8792ffce2eaa5975359bb6159a1ba7b85926a925
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-06-25 19:11:27 +08:00
guanxiatao
b40ee6edc9 Set Event.Type value to event.EventTypeImageDelete instead of event.EventTypeImagePush when deleting 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2019-06-20 09:39:45 +08:00
Steven Zou
9bac5e602d
Merge pull request #8030 from ywk253100/190605_replication_bugfix 
Fix replication bug
 2019-06-13 19:12:29 +08:00
Wenkai Yin
5fef7585c7 Fix replication bug 
Fixes #7875, fixes #7968

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-06-13 17:17:29 +08:00
Daniel Jiang
c928924fa5
Merge pull request #8003 from reasonerjt/bump-up-go112 
Bump up to go 1.12.5 enable go.mod
 2019-06-11 13:21:32 +08:00
wang yan
a4b202d656 remove the id in the post body when to create a robot account 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-06-11 10:47:56 +08:00
Daniel Jiang
737eaa396c Bump up to go 1.12.5 enable go.mod 
This commit bumps up the version of Go to compile the code to v1.12.5,
and shifts to go.mod for managing depedency.
Some code from "harbor/tests" to "harbor/src/testing" to avoid depedency
loop of modules.

Note that in short term we will still vendor the dependency.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-06-11 00:42:20 +08:00
Qian Deng
592e40bacf
Merge pull request #7859 from wy65701436/return-robot-id 
Return account id when to issue a robot
 2019-05-29 10:55:20 +08:00
wang yan
15ca9bfb81 Add ensure settings when to search user in auth proxy 
This issue is regresssion that introduced by f92bc8076d (diff-42381e6df5f17ebd3d9165a325d5d8f4), the a.ensure() was removed from SearchUser(), which leads to the alwaysonboard cannot be updated.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-22 17:16:16 +08:00
wang yan
056cfc7e31 Return account id when to issue a robot 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-22 10:39:26 +08:00
wang yan
2068732eef add validation for robot account registration 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-15 15:03:35 +08:00
Wang Yan
3be14b7997
fix issue7793: ping ldap server is always success (#7795) 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-05-15 11:35:57 +08:00
Wenkai Yin
e399ffca54
Merge pull request #7799 from reasonerjt/oidc-onboard-user-name 
Set the real name of OIDC user when onboarding
 2019-05-15 11:35:42 +08:00
Daniel Jiang
5f11dbe675 Set the real name of OIDC user when onboarding 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-15 00:44:51 +08:00
stonezdj
99c0a5a498 fix issue7793: ping ldap server is always success 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-05-14 19:20:54 +08:00
Qian Deng
f607c5177d Fix frontend failure caused by absolute path 
Fix failures because front downlowd chart using relative path

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-14 13:22:06 +08:00
Qian Deng
cd6c5a9f10 Enable absolute url in helm chart 
assign public_url to chart-url
remove namespace merge in index.yaml

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-14 12:56:20 +08:00
Daniel Jiang
e963ee88c7 Update login controller to return 403 for redirection 
As the UI cannot handle 302, update the login controller to return 403
and put the redirection URL in a json response body.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-13 18:16:06 +08:00
Daniel Jiang
d81afe274c Add filter to handle request with ID token (#7759) 
This commit allows request with a valid ID token to access the API.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-10 16:58:25 +08:00
Wenkai Yin
8348c1fa4b
Merge pull request #7635 from heww/validation-failed-status-code 
Return 400 status code for validation failed
 2019-05-10 14:22:05 +08:00
He Weiwei
58cbaaace8 Return 400 status code for validation failed 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-05-10 10:35:25 +08:00
Wang Yan
774a9f8d75
Remove unused configure item cfg_expiration (#7744) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-09 22:07:18 +08:00
wang yan
5c12c7713d Return 200 when to post an none schedule 
1, Throw the error of job service to UI when to create job schedule.
2, Return 200 when to save none without schedule.

Fixed #7675

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-09 13:32:46 +08:00
Daniel Jiang
cbbf2ea973 Redirect regular user to OIDC login page (#7717) 
When the auth mode is OIDC, when a user login via Harbor's login form.
If the user does not exist or the user is onboarded via OIDC, he will be
redirected to the OIDC login page.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-09 10:53:40 +08:00
Wang Yan
095f7b2ff7
add scan all and gc schedule migration (#7628) 
* add scan all and gc schedule migration

Signed-off-by: wang yan <wangyan@vmware.com>

* Fix gofmt errors

Signed-off-by: wang yan <wangyan@vmware.com>

* Update code according to review comments

Signed-off-by: wang yan <wangyan@vmware.com>

* remove convertschedule return name just return value

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-08 19:11:33 +08:00
Daniel Jiang
58aed3dde9
Merge pull request #7707 from renmaosheng/harbor-6838 
Don't display password when calling api/users API.
 2019-05-08 12:50:39 +08:00
Steven Ren
8311ff729a Don't display password when calling api/users API. 
This change fixes github issue 6838

Signed-off-by: Steven Ren <stevenr@stevenr-a01.vmware.com>
 2019-05-07 18:40:36 +08:00
wang yan
ab08a576e4 add multiple manifest intercepetor handler 
1, Add a interceptor to block request to upload manifest list
2, Discard notiification without tag.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-07 16:10:09 +08:00
Wenkai Yin
d27a6c0335 Fix a few bugs of replication (#7619) 
1. handle the public/private property when creating the projects
2. extend the length of access_secret
3. update the task status by using orm functions

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-05-07 10:47:14 +08:00
stonezdj(Daojun Zhang)
86bfd7a733 fix issue7637: /api/systeminfo cannot return 500 when DB is down (#7650) 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-05-07 10:36:10 +08:00
Qian Deng
210081604c
Merge pull request #7648 from reasonerjt/rm-index-controller 
Remove IndexController
 2019-05-06 18:29:14 +08:00
Daniel Jiang
45210f7c40 Fix condition for OIDC security filter (#7645) 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-06 16:30:09 +08:00
Daniel Jiang
28871b78ae Remove IndexController 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-06 14:04:06 +08:00
Daniel Jiang
6c3df3c8ce Disable CA download when hosted via HTTP 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-06 11:14:39 +08:00
Wenkai Yin
12d58370ad
Merge pull request #7503 from ywk253100/190424_stop_execution 
Check the task status of execution whose status is running when deleting the policy
 2019-04-30 11:28:49 +08:00
wang yan
02c7cbeec2 Fix get log issue of Periodic job 
Use the latest error or success execution as the periodic job log

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-29 15:30:05 +08:00
Wenkai Yin
7e4c227318 Check the task status of execution whose status is running when deleting the policy 
Check the task status of execution whose status is running when deleting the policy

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-29 15:04:41 +08:00
Daniel Jiang
56c7d55c73
Merge pull request #7593 from reasonerjt/ext-url-systeminfo 
Add Ext URL to response of systeminfo API
 2019-04-29 14:51:40 +08:00
Wenkai Yin
c53d73775a
Merge pull request #7590 from reasonerjt/oidc-wrong-secret-err 
Return more details for error in exchange token
 2019-04-29 14:22:37 +08:00
Daniel Jiang
02cf75c142 Add Ext URL to response of systeminfo API 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-29 13:31:52 +08:00
wang yan
2b99e148d9 Add gc parameters when to update gc schedule 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-28 18:17:08 +08:00
Wang Yan
c26f655bce
add periodic job UUID to upstream job id and use execution log as the… (#7530) 
* add periodic job UUID to upstream job id and use execution log as the periodic log

Signed-off-by: wang yan <wangyan@vmware.com>

* add comments to fix codacy

Signed-off-by: wang yan <wangyan@vmware.com>

* Update code per comments

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-28 15:09:56 +08:00
Wenkai Yin
7af679af7e
Merge pull request #7567 from reasonerjt/oidc-google-refresh-token 
Persist the new token in DB after login
 2019-04-28 14:12:25 +08:00
Daniel Jiang
80176cc354 Check whether user is nil in Prepare() of users API (#7507) 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-28 14:06:35 +08:00
Daniel Jiang
15626fcae0 Return more details for error in exchange token 
This commit update the response off OIDC callback when there's error in exchange token.
Additionally add comments to clarify that by default 500 error will not
contain any details.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-28 13:41:53 +08:00
Wenkai Yin
2a463016a9 Upgrade the distribution and notary library (#7516) 
* Return 404 when the log of task doesn't exist

Return 404 when the log of task doesn't exist

Signed-off-by: Wenkai Yin <yinw@vmware.com>

* Upgrade the distribution and notary library

Upgrade the distribution library to 2.7.1, the notary library to 0.6.1

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-28 12:00:26 +08:00
Daniel Jiang
473fed5689 Persist the new token in DB after login 
This commit make sure the token is persist to DB after every time after
a user logs in via OIDC provider, to make sure the secret is usable for
the OIDC providers that don't provide refresh token.

It also updates the authorize URL for google to make sure the refresh
token will be returned.

Also some misc refinement included, including add comment to the
OIDC onboarded user, preset the username in onboard dialog.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-27 23:03:59 +08:00
Daniel Jiang
b9f5f1027c
Merge pull request #7504 from reasonerjt/reload-auth-proxy-cert-verify 
Update Transport of HTTP cient in auth proxy client
 2019-04-26 23:24:33 +08:00
Daniel Jiang
07d15a8553 Update Transport of HTTP cient in auth proxy client 
This commit ensures that the TLS config of the HTTP client for auth
proxy is updated when the configuration is changed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-26 18:39:28 +08:00
Wenkai Yin
6511417ba6
Merge pull request #7495 from stonezdj/const_debts 
Replace string with const in metadatalist.go
 2019-04-25 17:41:04 +08:00
stonezdj
504eab56c3 Replace string with const in metadatalist.go 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-04-25 17:01:43 +08:00
Wenkai Yin
66087aac82
Merge pull request #7493 from stonezdj/tech_debts 
Remove adminserver in sourcecode
 2019-04-24 16:24:59 +08:00
Steven Zou
9bd2de3e35
Merge pull request #7452 from steven-zou/fix_issues_for_jobservice 
Fix issues for jobservice
 2019-04-24 16:15:43 +08:00
Wenkai Yin
d8310cc708 Fix replication bugs (#7470) 
1. Only return the event based trigger for local Harbor
2. Valid the trigger pattern and cron string when creating/updating policies
3. Set the schema as "http" if it isn't specified when creating/updating registries

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-23 19:34:29 +08:00
Steven Zou
823d9c04a9
Merge pull request #7497 from wy65701436/fix-replc-500 
refine chart clint http response
 2019-04-23 19:30:36 +08:00
wang yan
0d563fda9c refine chart clint http response 
Chart client eats the http error if not status ok, after refactor, the
real http response will be catched in core api.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-23 18:22:23 +08:00
Steven Zou
69d9a28860
Merge pull request #7482 from wy65701436/chart-upload 
Fix chart upload issue on event based
 2019-04-23 17:33:08 +08:00
stonezdj(Daojun Zhang)
e4506604e2 fix error message (#7459) 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-04-23 15:53:34 +08:00
stonezdj
d7798a12d2 Remove adminserver in sourcecode 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-04-23 15:05:29 +08:00
wang yan
a3763466b3 Update err message to general information 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-23 10:57:31 +08:00
wang yan
1b4c75af25 Add event into upload ctx 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-23 10:57:31 +08:00
wang yan
df6e0600c9 Fix chart upload issue on event based 
Use chart API to load the uploaded chart file to get the name and version

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-23 10:57:31 +08:00
Steven Zou
3937c8b0dc Merge branch 'master' into fix_issues_for_jobservice 2019-04-22 19:26:51 +08:00
Daniel Jiang
1fdc2e6ba9 Provide API to generate CLI secret 
This commit provide an API to allow a user that is onboarded via OIDC
authn update his CLI secret.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-22 13:34:12 +08:00
Steven Zou
8e734407c0 Merge branch 'master' into fix_issues_for_jobservice 2019-04-19 21:15:21 +08:00
Steven Zou
e92164c886
Merge pull request #7442 from ywk253100/190418_replication_bug_fix 
Fix bug in replication
 2019-04-19 17:04:34 +08:00
stonezdj(Daojun Zhang)
36d13e8243
Merge pull request #7328 from stonezdj/debts 
Fix issue 6450 Test LDAP server error without save configuration
 2019-04-19 16:51:57 +08:00
Daniel Jiang
6b45b5ef7c
Merge pull request #7451 from reasonerjt/oidc-logout 
Skip verifying OIDC token for local user
 2019-04-19 14:55:26 +08:00
Steven Zou
f8feaa192e add get scheduled and periodic executions APIs 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-04-19 13:54:23 +08:00
Wenkai Yin
cf5cd5902f Fix bug in replication 
1. Fix bug when creating the namespace
2. Keep the same logic for hiding access secret
3. Filter only push mode policies for event trigger

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-19 13:34:04 +08:00
Daniel Jiang
5292aea89e Skip verifying OIDC token for local user 
If a user does not have OIDC meta data in DB, it means he's not
onboarded via OIDC authn, hence, we should not check the token.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-19 13:07:06 +08:00
Wenkai Yin
059b75e97c
Merge pull request #7392 from reasonerjt/oidc-logout 
Handle OIDC user invalidation from OIDC provider.
 2019-04-19 12:46:36 +08:00
Daniel Jiang
239b33c5fb Handle OIDC user invalidation from OIDC provider. 
Ths commmit ensures that when user's token is invalidated OIDC provider, he
cannot access protected resource in Harbor with the user info in his session.
We share the code path with secret verification b/c the refresh token
can be used only once, so it has to be stored in one place.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-19 01:27:31 +08:00
Steven Zou
1f481e492c Refactor job servcie primary logic to fix related bugs 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-04-18 16:02:49 +08:00
stonezdj
41a574e55c Fix issue 6450 Test LDAP server error without save configuration 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-04-18 14:24:21 +08:00
Wenkai Yin
a5cc228781
Merge pull request #7420 from ywk253100/190417_revert_local_harbor 
Update the migration sql
 2019-04-17 19:58:31 +08:00
wang yan
ddec7bd645 fix error handlering in job notification 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-17 18:19:01 +08:00
wang yan
e017294f71 merge with master latest 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-17 17:52:39 +08:00
Wenkai Yin
547c2337de Update the migration sql 
1. Update the migration sql
2. Rename the ResourceRepository from repository to image

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-17 17:18:03 +08:00
Wang Yan
a6af9e9972
Support well-formatted error returned from the REST APIs. (#6957) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-17 16:43:06 +08:00
Wenkai Yin
6e0d892963 Support creating project with service account 
This commit introduces a solution to workaround the restriction of project creation API: only normal users can create projects

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-17 14:50:32 +08:00
Wenkai Yin
3f7884d9d2 Revert "Add new registry type: LocalHarbor" 
This reverts commit 94cacf762a.

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-17 14:50:24 +08:00
Wenkai Yin
94cacf762a Add new registry type: LocalHarbor 
The "LocalHarbor" is the type of registry where the replication service is running on

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-16 11:42:36 +08:00
Wenkai Yin
2f1d2257d5 Remove the namespace concept in replication 
Update the replication logic to remove the "namespace"

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-16 08:36:42 +08:00
wang yan
7a373c2eed Add event trigger to helm upload/deletion replication 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-15 19:02:33 +08:00
Wenkai Yin
ba038eb883 Support replication all projects in Harbor 
Support replication all projects in Harbor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-13 20:23:19 +08:00
Wenkai Yin
c222f18fa7 Update replication 
1. Refine the health check of docker hub
2. Remove the GetNamespace method from adapter interface

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-13 15:20:06 +08:00
Daniel Jiang
f92bc8076d "Skip verify cert" to "verify cert" 
This commit tweaks the attribute for auth proxy mode and OIDC auth mode.
To change it from "Skip verify cert" to "verify cert" so they are more
consistent with other modes.
Additionally it removes a workaround in `SearchUser` in auth proxy
authenticator.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-12 23:25:54 +08:00
Wenkai Yin
1d16e18dff Remove "ng" from source code 
Remove "ng" from source code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-12 22:42:27 +08:00
cd1989
8ca5e17c58 Correct some typos and do some small adjustments 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-04-12 21:17:33 +08:00
cd1989
a9fa22269c Check health status when add/update registry 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-04-12 17:29:05 +08:00
Wenkai Yin
188d66d875
Merge pull request #7350 from ywk253100/190411_bugfix 
Fix bug of replication
 2019-04-12 08:22:59 +08:00
Wenkai Yin
bc0123662b Fix bug of replication 
1. check the disable/enable status before starting the replication
2. process the support_namespace property

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-12 07:59:57 +08:00
Wenkai Yin
1f574e7d10
Merge pull request #7354 from wy65701436/replication_ng_namespace 
Add api to get namespaces of registry
 2019-04-11 23:44:24 +08:00
wang yan
117c36d52c Add api to get namespaces of registry 
To query the namespace of the registry according to its ID.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-11 23:17:33 +08:00
Wenkai Yin
a2fcb41b31 Fix bug in ping registry API 
Fix bug in ping registry API: accept both ID and other properties

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-11 18:00:44 +08:00
Daniel Jiang
763c5df010 Add UT 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-11 15:30:19 +08:00
Wenkai Yin
b73acde051 Support the migration for scheduled replication rule from previous version of Harbor 
Support the migration for scheduled replication rule from previous version of Harbor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-11 13:14:32 +08:00
Wenkai Yin
5a047a7eb6 Update the adapter interface 
Add ConvertResourceMetadata and PrepareForPush methods

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-10 23:37:11 +08:00
Daniel Jiang
0d18e6c82f Update according to comments 
For more context see PR #7335

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-10 19:38:12 +08:00
Daniel Jiang
0a2343f542 Support secret for docker CLI 
As CLI does not support oauth flow, we'll use secret for help OIDC user
to authenticate via CLI.
Add column to store secret and token, and add code to support
verify/refresh token associates with secret.  Such that when the user is
removed from OIDC provider the secret will no longer work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-10 19:38:11 +08:00
Daniel Jiang
08e00744be Fix misc bugs for e2e OIDC user onboard process 
This commit adjust the code and fix some bugs to make onboard process
work.
Only thing missed is that the UI will need to initiate the redirection,
because the request of onboarding a user was sent via ajax call and didn't
handle the 302.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-10 19:38:11 +08:00
Wenkai Yin
5a65480594 Handle the policy from previous versions 
Handle the policy from previous versions

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-09 17:43:07 +08:00
Wenkai Yin
7ff46acd25
Merge pull request #7289 from cd1989/add-registry-ping 
Add registry ping API
 2019-04-08 14:08:53 +08:00
cd1989
5a2d03593f Add helth check method to registry adapter 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-04-08 10:03:28 +08:00
cd1989
f71a110bec Add registry ping API 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-04-06 17:00:52 +08:00
cd1989
07139684ce Wait randomly before registry health checking 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-04-05 20:46:29 +08:00
cd1989
fe004e1bfc Init replication in core 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-04-04 21:58:31 +08:00
cd1989
2450dacecb Use policy controller in registry deletion 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-04-04 21:26:27 +08:00
Wenkai Yin
4116433de8
Merge pull request #7306 from ywk253100/190404_cleanup 
Remove the useless replication code
 2019-04-04 21:18:04 +08:00
Wenkai Yin
c2f702be2a Remove the useless replication code 
This commit removes the useless replication code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-04 20:56:25 +08:00
Wenkai Yin
58a73de3e5
Merge pull request #7299 from ywk253100/190404_sync 
Sync with master branch
 2019-04-04 17:33:11 +08:00
Wenkai Yin
1c735a7464 Filter the events triggerred by replication 
Filter the events triggerred by replication pull

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-04 16:03:38 +08:00
Wenkai Yin
b66b1f341e Merge remote-tracking branch 'upstream/master' into 190404_sync 2019-04-04 14:55:09 +08:00
Wenkai Yin
48f02d0605
Merge pull request #7291 from cd1989/registry-with-empty-credential 
Handle registry with empty credential
 2019-04-03 21:35:32 +08:00
Yan
da0e20ec60
Add controller to onboard oidc user (#7286) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-03 20:47:22 +08:00
cd1989
e2c86f8f59 Handle registry with empty credential 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-04-03 20:33:26 +08:00
cd1989
8968e82675 Allow edit registry description 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-04-03 19:47:14 +08:00
wang yan
dcf1d704e6 fix dao UT issue and refine the error of onboard OIDC user 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-03 14:05:18 +08:00
wang yan
41018041f7 remove oidc controller and add more UTs 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-03 09:54:21 +08:00
Yan
0de5999f52 add the controller for ocdi onboard user 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-04-03 09:52:22 +08:00
Wenkai Yin
5219073c49 Call Harbor API to delete the images in Harbor adapter 
Call Harbor API to delete the images in Harbor adapter to avoid the inconsistent between the different versions of Harbor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-02 17:02:15 +08:00
Wenkai Yin
4484bca756 Fix replication related issues 
1. Add operation property for tasks
2. Add trigger property for executions
3. Update the getting registry info API to allow passing 0 as ID to get the info of local Harbor registry

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-04-02 14:26:17 +08:00
Daniel Jiang
587acd33ad Add callback controller for OIDC 
This commit add callback controller to handle the redirection from
successful OIDC authentication.
For E2E case this requires callback controller to kick off onboard
process, which will be covered in subsequent commits.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-04-01 12:35:31 +08:00
Wenkai Yin
71b706e60a Update the replication API 
1. Add getting execution by ID API
2. Return registry detail info in listing policies API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-30 20:46:02 +08:00
Wenkai Yin
8c7b63bac2
Merge pull request #7248 from ywk253100/190326_event 
Add event based trigger and scheduled trigger
 2019-03-29 14:58:09 +08:00
Wenkai Yin
4f8e283e8e Add event based trigger and scheduled trigger 
This commit implements the event based trigger and scheduled trigger in replilcation

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-29 13:48:34 +08:00
Daniel Jiang
9ce98f4acd Add controller to handle oidc login 
The controller will redirect user to the OIDC login page based on
configuration.
Additionally this commit add some basic code to wrap `oauth2` package
and `provider` in `go-oidc`, and fixed an issue in UT to make
InMemoryDriver for config management thread-safe.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-03-28 11:29:05 +08:00
Wenkai Yin
0e9bdbc09c
Merge pull request #7225 from ywk253100/190326_structure 
Adjust the file structure of replication package
 2019-03-27 13:08:05 +08:00
Yan
03709e4ec1
add authn proxy (#7199) 
* add authn proxy docker login support

User could use the web hook token issued by k8s api server to login to harbor.
The username should add a specific prefix.

Signed-off-by: wang yan <wangyan@vmware.com>

* update code per review comments

Signed-off-by: wang yan <wangyan@vmware.com>

* Add UT for auth proxy modifier

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-03-27 12:37:54 +08:00
Wenkai Yin
017bba8dc1 Merge remote-tracking branch 'upstream/master' into 190327_sync 2019-03-27 11:43:51 +08:00
Wenkai Yin
de4eb0369a Adjust the file structure of replication package 
Move the scheduler, execution, hook and flow package into operation

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-26 14:59:58 +08:00
Daniel Jiang
49aae76205 Onbard settings for OIDC provider (#7204) 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-03-25 12:24:39 +08:00
Wenkai Yin
b37f4018a6 Update the registry adapter interface 
This commit adds the Info() method to the registry adapter interface

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-23 07:49:59 +08:00
Wenkai Yin
679b0d3d6a Convert job status to replication task status 
This commits converts job status to task status

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-22 18:16:27 +08:00
Yan
8d3946a0e2
Refactor scan all api (#7120) 
* Refactor scan all api

This commit is to let scan all api using admin job to handle schedule
management. After the PR, GC and scan all share unified code path.

Signed-off-by: wang yan <wangyan@vmware.com>

* update admin job api code according to review comments

Signed-off-by: wang yan <wangyan@vmware.com>

* Update test code and comments per review

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-03-22 17:52:21 +08:00
Wenkai Yin
49cf50adb1 Merge remote-tracking branch 'upstream/master' into 190324_sync 
Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-22 15:55:52 +08:00
Wenkai Yin
791aecddfa
Merge pull request #7210 from ywk253100/190321_delete 
Add support for replicating the delation of resource
 2019-03-23 20:08:13 +08:00
Wenkai Yin
1120368c9c Add support for replicating the delation of resource 
This commit refines the replication flows and provides the support for replicating resource deletion

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-22 12:37:39 +08:00
Meina Zhou
130e132f86 Merge branch 'master' into replication_ng 
Signed-off-by: Meina Zhou <meinaz@vmware.com>
 2019-03-21 14:16:33 +08:00
Wenkai Yin
c65d5e6669 Update listing/getting replication adapter API 
This commit updates the listing/getting replication adapter API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-20 05:30:15 +08:00
Daniel Jiang
a73b499988 Expose HTTP auth proxy infor in systeminfo API (#7164) 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-03-19 15:54:40 +08:00
Wenkai Yin
258b22a9a5 Fix bug in replication 
This commit fixes bugs found in the implement of replciation NG

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-10 20:57:59 +08:00
Wenkai Yin
cabef73980 Add Harbor adapter for replication 
Implement the replication adapter for Harbor registry

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-13 21:39:39 +08:00
peimingming
4efad287ce Add execution and hooks 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2019-03-13 09:35:01 +08:00
Wenkai Yin
772367498f Merge remote-tracking branch 'upstream/master' into 190311_sync 2019-03-11 20:34:49 +08:00
Wenkai Yin
d1f4c20e64 Implement replication policy management API 
This commit implements the replication policy management API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-09 01:32:51 +08:00
Wenkai Yin
ec2a7f9239 Implement replication operation API 
This commit implements the replication operation related APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-08 10:06:33 +08:00
Wenkai Yin
db7a709aad
Merge pull request #7063 from heww/users-search 
Add users search API
 2019-03-07 15:58:26 +08:00
stonezdj(Daojun Zhang)
f7745baf30
Merge pull request #6599 from stonezdj/pr6161 
Add new parameter ldap_group_membership_attribute (PR#6161)
 2019-03-07 13:26:26 +08:00
He Weiwei
20556aebd2 Add users search API 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-03-07 12:47:26 +08:00
Wenkai Yin
7f49151115 Implement replication adapter API 
This commit implements the replication adapter API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-03-06 15:01:33 +08:00
Daniel Jiang
002094dbbb
Merge pull request #7075 from wy65701436/cron-str 
update gc api to support raw cron string
 2019-03-06 13:36:52 +08:00
Qian Deng
b68f09cf41 Fix: global search not work when chart enabled 
Global search result data does not contain the chart info when chart is empty

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-03-06 10:37:41 +08:00
wang yan
e373167546 update gc api to support raw cron string 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-03-05 16:31:35 +08:00
cd1989
b00098d492 Add unit tests and fix CI 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-03-05 15:37:36 +08:00
stonezdj(Daojun Zhang)
dffb971366
Merge pull request #7055 from stonezdj/bug7038 
Remove verify_remote_cert
 2019-03-05 14:54:02 +08:00
stonezdj
4dfee0c1f0 Remove verify_remote_cert 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-03-05 14:04:10 +08:00
Wenkai Yin
77688c90b9
Merge pull request #7061 from stonezdj/bug6767 
Search local DB first when adding a project member with username
 2019-03-05 12:56:19 +08:00
Mia ZHOU
76a07eb5fe
Merge pull request #7059 from ninjadq/fix_global_search_issue 
Fix global search issue
 2019-03-05 08:41:30 +08:00
Qian Deng
4ba012ab8e Fix: global search not work issue 
Both Frontend and Backend should not send chart data when chartmusuem not enabled

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-03-04 17:44:34 +08:00
stonezdj
3fdc0fd9ba Search local DB first when adding a project member with username 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-03-04 11:22:37 +08:00
stonezdj
cf134bc80e Add new parameter ldap_group_membership_attribute 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-03-03 10:03:22 +08:00
Daniel Jiang
321874c815 Move Settings of HTTP auth proxy (#7047) 
Previously the settings of HTTP authproxy were set in environment
variable.
This commit move them to the configuration API

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-03-01 14:11:14 +08:00
cd1989
8732a20709 Rewrite registry manager with new interface 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-02-27 11:54:04 +08:00
cd1989
6bdf3053a7 Implement registries manager 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-02-27 11:54:04 +08:00
wang yan
91aa67a541 Update expiration variable name to expiresat/tokenduration 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-02-25 11:55:42 +08:00
wang yan
36a778b482 Update expiration schema to bigint and default unit to minute 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-02-22 18:42:43 +08:00
wang yan
47a09b5891 add expiration of robot account 
This commit is to make the expiration of robot account configurable

1, The expiration could be set by system admin in the configuation page or
by /api/config with robot_token_expiration=60, the default value is 30 days.
2, The expiration could be shown in the robot account infor both on UI and API.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-02-22 18:42:34 +08:00
Nguyen Quang Huy
eda6c47b3e add signoff for DCO gate (#6981) 
Some variable name, function name is colliding with builtin function.

Signed-off-by: Nguyen Quang Huy <huynq0911@gmail.com>
 2019-02-22 15:00:18 +08:00
stonezdj
7a5fbf718f Revise code with review comments 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-02-18 15:20:54 +08:00
stonezdj
880051c08a Add load for user settings in core/config/config.go 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-02-18 14:06:19 +08:00
stonezdj
36e1c13a43 fix ut error in systeminfo_test.go 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-02-18 14:06:19 +08:00
stonezdj
1ae5126bb4 Refactor adminserver stage 3: replace config api and change ut settings 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-02-18 14:06:19 +08:00
He Weiwei
1c4b9aa346 Protect API using rbac 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-02-01 18:55:06 +08:00
Daniel Jiang
bf663df0e7
Merge pull request #6820 from wy65701436/robot-service 
Add robot account authn & authz implementation
 2019-01-29 16:08:25 +08:00
He Weiwei
6e95b98108 Standard actions for rbac 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-29 11:59:11 +08:00
He Weiwei
1da0a66fe5
Merge pull request #6781 from heww/user-permissions-api 
Implement api for get current user permissions
 2019-01-29 01:58:51 +08:00
wang yan
2d7ea9c383 update codes per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-28 21:26:06 +08:00
He Weiwei
8b5e68073d Implement api for get current user permissions 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-28 18:06:52 +08:00
Yan
71f37fb820 * Add robot account authn & authz implementation. 
This commit is to add the jwt token service, and do the authn & authz for robot account.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-28 17:39:57 +08:00
Daniel Jiang
20db0e737b Provide HTTP authenticator 
An HTTP authenticator verifies the credentials by sending a POST request
to an HTTP endpoint.  After successful authentication he will be
onboarded to Harbor's local DB and assigned a role in a project.

This commit provides the initial implementation.
Currently one limitation is that we don't have clear definition about
how we would "search" a user via this HTTP authenticator, a flag for
"alway onboard" is provided to skip the search, otherwise, a user has
to login first before he can be assigned a role in Harbor.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-01-28 15:43:44 +08:00
He Weiwei
3f8e06a8bc Support master role for project member create and update apis (#6780) 
* Support master role for project member create and update apis

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* Fix description for role_id in swagger.yaml

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-23 14:56:23 +08:00
He Weiwei
ae061482ae Add Can method to securty.Context interface (#6779) 
* Add Can method to securty.Context interface

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* Improve mockSecurityContext Can method

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-01-23 14:32:37 +08:00
wang yan
903e15235e Update validation and error message per comments 2019-01-17 15:33:05 +08:00
wang yan
4cde11892a update the conflict check with DB unique constrain error message 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-17 13:13:55 +08:00
Yan
1af0f3c3b9 Add API implementation of robot account 
Add API implementation of robot account

1. POST /api/project/pid/robots
2, GET /api/project/pid/robots/id?
3, PUT /api/project/pid/robots/id
4, DELETE /api/project/pid/robots/id

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-01-17 13:13:55 +08:00
Wenkai Yin
f8d9653419
Merge pull request #6737 from ywk253100/190109_health_check 
Implement the unified health check API
 2019-01-16 18:14:14 +08:00
Wenkai Yin
be4455ec1b Implement the unified health check API 
The commit implements an unified health check API for all Harbor services

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-01-16 17:21:04 +08:00
Daniel Jiang
5d59d6fab8 Bump up golang to 1.11.2 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-01-11 14:44:32 +08:00
Jan-Otto Kröpke
8b65e4f424
Remove user agent check for docker notifications 
Fixes #5729

Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>
 2019-01-03 21:17:29 +01:00
Wenkai Yin
75d45ebd9d
Merge pull request #6547 from cd1989/retag-input-validation 
Validate repo and tag names in retag
 2019-01-03 17:45:44 +08:00
cd1989
c117a23133 Validate repo and tag names in retag 
Signed-off-by: cd1989 <chende@caicloud.io>
 2018-12-24 16:49:39 +08:00
He Weiwei
f403e50234
Merge pull request #6577 from heww/master 
Include os version in image tag detail page
 2018-12-20 18:18:44 +08:00
He Weiwei
e7f09643bd Include os version in image tag detail page 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2018-12-20 15:12:19 +08:00
Wenkai Yin
b28bca7af4
Merge pull request #6541 from salkin/proxy-transport 
Add support for http proxy in transport
 2018-12-18 15:46:29 +08:00
Niklas Wik
138bc69f0f Add support for http proxy in transport 
Signed-off-by: Niklas Wik <niklas.wik@nokia.com>
 2018-12-17 10:35:27 +02:00
Wenkai Yin
f7a28ee2a2 Remove the duplicate http error struct (#6516) 
There are two different types to represent http error in the current code. This commit updates the codes to keep only one.

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2018-12-12 11:51:19 +08:00
Wenkai Yin
bcd6947fcc
Merge pull request #6470 from cd1989/retag-problem 
Give meaningful error messages when retag is forbidden
 2018-12-06 18:47:58 +08:00
De Chen
60d65a9d86 Block retag requests in read-only mode (#6457) 
Signed-off-by: cd1989 <chende@caicloud.io>
 2018-12-06 18:35:22 +08:00
cd1989
caf07a96fe Give meaningful messages when retag forbided 
Signed-off-by: cd1989 <chende@caicloud.io>
 2018-12-06 16:25:21 +08:00
Wenkai Yin
746d58ceb4 Return the error message when changing password with wrong old password (#6466) 
Return a meaningful error message when changing password but the a wrong old password is provided to render on UI

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2018-12-06 13:29:30 +08:00
Daniel Jiang
29d5b5da72 Return 409 when there is a scan all job running (#6460) 
* Return 409 when user trigger another "scan all"

This commit fixes #6418, that when multiple "scan all" jobs are
triggered, the API should not return 500.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>

* Update swagger to add 409 to scanAll API

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2018-12-06 11:46:38 +08:00
Wenkai Yin
538082ceb6 Remove the permission checking for getcert API (#6436) 
The Harbor root cert can be downloaded by all users now, so the permission checking is not needed anymore

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2018-12-05 12:15:23 +08:00
Daniel Jiang
ae240df031 Remove the Scan all in-memory marker (#6399) 
Previously there was a in-memory marker to prevent user from frequently
calling the "scan all" API.  This has become problematic in HA
deployment, and is no longer needed after enhancement in jobservice.

This commit removes the marker for "scan all" api, however, we need to
review the mechanism and rework to make it stateless.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2018-12-02 15:40:50 +08:00
Wenkai Yin
a81346a4ba
Merge pull request #6386 from heww/session 
change session cookie name to sid
 2018-11-30 16:13:45 +08:00
Steven Zou
ec2ad4d0b8
Merge pull request #6093 from cd1989/replication-record-id 
Add op uuid to image replication
 2018-11-30 14:54:43 +08:00
Wenkai Yin
9d5cf57373 Check the existence of name when creating replication rule and fix bugs in testing library (#6381) 
1. Fix #5102 by checking the existence of name when creating/editing replication rule
2. Add unique constraint to the name of replication policy and target
3. Fix bugs of testing library

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2018-11-30 13:32:20 +08:00
stonezdj
3b165d41d4 Fix LDAP search error 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2018-11-29 18:37:23 +08:00
Steven Zou
68b1b98f0a
Merge pull request #6375 from steven-zou/fix_global_search_502_issue 
Fix global search 502 issue happened when chart repo is not enabled
 2018-11-29 16:29:08 +08:00
Steven Zou
e7ffaecca5 Fix global search 502 issue happened when chart repo is not enabled 
Signed-off-by: Steven Zou <szou@vmware.com>
 2018-11-29 15:53:09 +08:00
He Weiwei
00a3948fff change session cookie name to sid 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2018-11-29 15:18:15 +08:00
陈德
f72c7766ae Fix status code for permission check in retag, use 403 
Signed-off-by: 陈德 <chende@caicloud.io>
 2018-11-28 19:48:25 +08:00
Daniel Jiang
abe728325b Wait for manifest in notification handler 
There's an issue in registry 2.6.x, that when the webhook is sent the
manifest of the image may not be written.
For details: https://github.com/docker/distribution/issues/2625

This will cause issue in "scan on push" or replication.
This commit mitigates the issue by adding retries in notification
handler.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2018-11-06 19:23:54 +08:00
Qian Deng
35f3346948
Merge pull request #6155 from mmpei/6086-UserNameLength 
Support longer UserName
 2018-11-06 10:50:50 +08:00
Steven Zou
1b1ab60802
Merge pull request #6152 from mmpei/5178-jibSupporting 
5178 jib supporting
 2018-10-29 16:34:13 +08:00
peimingming
d3a617efd6 Support longer UserName 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2018-10-29 15:59:17 +08:00
mmpei
a209519b0e add support jib 
Signed-off-by: mmpei <peimingming1986@126.com>
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2018-10-26 15:32:10 +08:00