Commit Graph

2766 Commits

Author SHA1 Message Date
Wang Yan
31c3413ddf
Merge pull request #8288 from wy65701436/mf-workflow-base
Add quota workflow for manifest
2019-07-17 11:25:30 +08:00
Steven Zou
0b2f94b0dd
Merge pull request #8256 from goharbor/feat/retention/GH-6661-retain-newer-than-n-days
Retention: Implement Evaluator: Retain if created less than x days ago
2019-07-17 00:04:40 +08:00
Steven Zou
3243d5bce1
Merge pull request #8255 from goharbor/feat/retention/GH-6660-keep-most-recent-n
Retention: Implement Filter: Keep Latest K
2019-07-17 00:04:02 +08:00
Ziming Zhang
815901ea33 fix
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I3f2d3c7f1e32b4983c31c23d9753f04239e3c82f
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-07-16 19:24:40 +08:00
stonezdj
bb2ae7c093 Add HTTP group feature
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-07-16 15:38:46 +08:00
Shijun Sun (c)
140a3da238 modify UT for cve-whitelist UI
Signed-off-by: Shijun Sun (c) <sshijun@vmware.com>
2019-07-16 15:24:15 +08:00
Ziming Zhang
c22c38994a retention api
Change-Id: I70f2c34d6bb96ecf4cb5359e2b1ab2dbb99fdbf9
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-07-16 15:06:37 +08:00
wang yan
105518fe12 Merge branch 'master' of https://github.com/goharbor/harbor into project-quota-dev 2019-07-16 14:51:37 +08:00
Wang Yan
8ac6bdbbb0 Add quota workflow for quota
1, apply count for manifest if it's a new image
2, insert data for artifact and artifact_blob

Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-16 14:48:05 +08:00
Shijun Sun (c)
274ab7bcff add CVE-Whitelist
Signed-off-by: Shijun Sun (c) <sshijun@vmware.com>
2019-07-15 19:47:46 +08:00
Nathan Lowe
d7e6b1b621
Sort artifacts before processing and port tests to github.com/stretchr/testify/suite
Signed-off-by: Nathan Lowe <public@nlowe.me>
2019-07-14 22:45:36 -04:00
Nathan Lowe
f03ed9b14c
Port tests to github.com/stretchr/testify/suite
Signed-off-by: Nathan Lowe <public@nlowe.me>
2019-07-14 22:38:26 -04:00
chlins
fc713a1763 refactor: fix unit tests
Signed-off-by: chlins <chlins.zhang@gmail.com>
2019-07-12 11:33:31 +08:00
chlins
000fe3d084 refactor: adjust some logic in huawei adapter
Signed-off-by: chlins <chlins.zhang@gmail.com>
2019-07-12 11:33:30 +08:00
wang yan
f066d986b9 merge with latest master code 2019-07-11 20:21:15 +08:00
Wang Yan
b98ca7bf0b
Merge pull request #8237 from wy65701436/redis-locker
add redis lock
2019-07-11 20:10:16 +08:00
wang yan
ef14f0cf35 add redis lock, it will be used to lock digest in the quota scenario
Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-11 19:24:24 +08:00
Wenkai Yin(尹文开)
44ad142d86
Merge pull request #8260 from ywk253100/190709_launcher
Implement the launcher
2019-07-11 18:21:04 +08:00
Wenkai Yin
91b050a01b Implement the launcher
The commit implements the launcher for tag retention

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-07-11 17:18:46 +08:00
Steven Zou
69ff8ff8ba
Merge pull request #8277 from steven-zou/feature/tag_retentio_more_evaluators
add more rule evaluators
2019-07-11 14:09:29 +08:00
Steven Zou
07250469ac add more rule evaluators
Signed-off-by: Steven Zou <szou@vmware.com>
2019-07-11 10:50:10 +08:00
Steven Zou
52ca6de22a refactor the processor interface
Signed-off-by: Steven Zou <szou@vmware.com>
2019-07-11 00:23:19 +08:00
Wenkai Yin(尹文开)
3bebf7bc64
Merge pull request #8238 from reasonerjt/project-cve-whitelist
Enable project level CVE whitelist
2019-07-10 14:41:01 +08:00
jwangyangls
432f08b5b5
Merge pull request #8241 from jwangyangls/fixReplicationLabelFilterbug
Fix bug when no labels
2019-07-10 14:38:51 +08:00
jwangyangls
c73fa851ff
Merge pull request #8221 from jwangyangls/addGroupInhttpMode
Add user group when http auth mode
2019-07-10 14:38:31 +08:00
Steven Zou
9756195496 implement retention job
Signed-off-by: Steven Zou <szou@vmware.com>
2019-07-10 14:01:17 +08:00
Yogi_Wang
6ef82d4db9 Fix bug when no labels
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-07-10 13:02:34 +08:00
Nathan Lowe
8b174825c1
Retention: Implement Evaluator: Retain if created less than x days ago
Signed-off-by: Nathan Lowe <public@nlowe.me>
2019-07-09 23:46:26 -04:00
Nathan Lowe
4ecbe749e4
Retention: Implement Filter: Keep Latest K
Signed-off-by: Nathan Lowe <public@nlowe.me>
2019-07-09 23:24:56 -04:00
Wang Yan
155b0b0acd
Merge pull request #8175 from heww/quota-manager
Add manager for quota
2019-07-10 11:03:57 +08:00
wang yan
6d0271ee5c Merge branch 'master' of https://github.com/goharbor/harbor into project-quota-dev 2019-07-10 10:57:10 +08:00
Steven Zou
185f4f3861 refine or.processor implementation and also add UT case
Signed-off-by: Steven Zou <szou@vmware.com>
2019-07-09 22:10:09 +08:00
Steven Zou
3409065438 implement policy builder
Signed-off-by: Steven Zou <szou@vmware.com>
2019-07-09 17:43:19 +08:00
Steven Zou
c36afcd07d add index for rule,selector and performer
Signed-off-by: Steven Zou <szou@vmware.com>
2019-07-09 15:10:53 +08:00
He Weiwei
41ba410bb2 Manager for quota
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-07-09 13:59:48 +08:00
Wang Yan
7fd0bbcbd2
Merge pull request #8230 from wy65701436/dao-artifact
add dao of artifact
2019-07-09 13:44:05 +08:00
wang yan
7b38389898 update codes per review comments
Signed-off-by: wang yan <wangyan@vmware.com>

fix middlewares per review comments
1, add scheme1 and scheme2 check
2, change MustCompile to Compile

Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-09 11:08:19 +08:00
wang yan
b3c5137a2f add copyright and fix codecy
Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-09 11:08:19 +08:00
wang yan
57821b1b4c Refactor interceptors code with chain
1, add a blob inteceptors for quota usage
2, add a manifest inteceptors for quota usage

Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-09 11:08:11 +08:00
Steven Zou
24ee32d7d1 define the base interfaces for the tag retention feature
Signed-off-by: Steven Zou <szou@vmware.com>
2019-07-09 09:40:59 +08:00
wang yan
24c3753581 add dao of artifact
Signed-off-by: wang yan <wangyan@vmware.com>

Add dao for quota

Signed-off-by: He Weiwei <hweiwei@vmware.com>

fix govet

Signed-off-by: wang yan <wangyan@vmware.com>
2019-07-08 23:42:50 +08:00
Daniel Jiang
8f5f0031c7 Enable project level CVE whitelist
This commit update the project API to support "reuse_sys_cve_whitelist"
setting in project metadata and "cve_whitelist" in project request.
Also modify the interceptor to support project level CVE whitelist if
the reuse flag is false.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-07-08 18:55:54 +08:00
Yogi_Wang
c57087574a Add user group when http auth mode
add the pages  in project and system

Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-07-08 17:28:38 +08:00
Ziming Zhang
0691a5ad9c fix replication dockerhub tls skip tls
Change-Id: I9584d89cf9bda2b782b0fd556450a81be0fc7510
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-07-08 08:17:26 +00:00
Wenkai Yin(尹文开)
5f9420a5a7
Merge pull request #8190 from ywk253100/190701_replication
Merge Default ImageRegistry into the native adapter to reduce the duplicate code
2019-07-08 12:47:55 +08:00
Daniel Jiang
c296f0ddfb
Merge pull request #8176 from stonezdj/http_group
Refactor LDAP usergroup
2019-07-08 09:54:31 +08:00
stonezdj
c0ed55445d Refactor LDAP group
Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-07-05 14:44:18 +08:00
Wenkai Yin(尹文开)
0f28fe42fd
Merge pull request #8167 from ywk253100/190622_replication
Support v1 signed media type when do the replication
2019-07-05 14:33:22 +08:00
Wenkai Yin(尹文开)
c01bedb740
Merge pull request #8220 from reasonerjt/oidc-rotation-fix
Reload OIDC provider older than 3 seconds
2019-07-05 10:12:33 +08:00
Daniel Jiang
a75bc027de
Merge pull request #8213 from reasonerjt/oidc-helm-push
Handle helm push in OIDC filter
2019-07-04 15:56:50 +08:00
Daniel Jiang
88a5572f8e Reload OIDC provider older than 3 seconds
This commit make sure the OIDC is more actively recreated, to mitigate
the problem in #8177

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-07-04 14:55:34 +08:00
He Weiwei
4fedfa6580 Add dao for quota
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-07-04 11:53:26 +08:00
Steven Zou
63e2ce7606
Merge pull request #8209 from steven-zou/fix/enqueuer_schedule_slot
use separate key for the last periodic enqueue data
2019-07-03 21:48:21 +08:00
Steven Zou
8fc693d843
Merge pull request #8208 from steven-zou/fix/enqueuer_ut
fix failure ut case of job service
2019-07-03 17:32:13 +08:00
Daniel Jiang
8a9d352f54 Handle helm push in OIDC filter
Fixes #8130
Enable OIDC filter to handle requests to /api/chartrepo/*

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-07-03 17:29:37 +08:00
Steven Zou
b88159e747 use separate key for the last periodic enqueue data
Signed-off-by: Steven Zou <szou@vmware.com>
2019-07-03 17:08:23 +08:00
Daniel Jiang
5d887ad0d8
Merge pull request #8179 from reasonerjt/interceptor-use-whitelist
Apply CVE white list in interceptor
2019-07-03 15:12:33 +08:00
Steven Zou
1002e3ce17 fix failure ut case of job service
Signed-off-by: Steven Zou <szou@vmware.com>
2019-07-03 15:10:41 +08:00
Daniel Jiang
bba4b2a6a4 Apply CVE white list in interceptor
Interceptor will filter the vulnerability in whitelist while calculating
the serverity of an image and determine whether or not to block client
form pulling it.

It will use the system level whitelist in this commit, another commit
will switch to project level whitelist based on setting in a project.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-07-03 14:13:00 +08:00
Steven Zou
8bb18e73d2
Merge pull request #8159 from ywk253100/190627_label_filter
Update replication label filter
2019-07-03 13:08:54 +08:00
Daniel Jiang
6f166bc02c
Merge pull request #8154 from markpeek/markpeek-registry-health-check
Switch registry health check to a 200 response url
2019-07-03 10:29:35 +08:00
He Weiwei
720dcc72bd Fix read permission of project member read api
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-07-02 14:40:46 +08:00
jwangyangls
02c5823915
Merge pull request #8182 from jwangyangls/fixVulnerabilityWidth
Fix the different width on vulnerability
2019-07-02 11:56:48 +08:00
jwangyangls
e87381d671
Merge pull request #8178 from jwangyangls/replicationAddLabelFilter
Add label filter in replication Ng
2019-07-02 11:55:55 +08:00
jwangyangls
1916f3c078
Merge pull request #8165 from AllForNothing/filter-ui
fix  filter ui bug : half filter is covered when there is one or two records in a datagrid
2019-07-02 11:07:07 +08:00
Yogi_Wang
9c07caa1a6 Add label filter in replication Ng
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-07-02 10:53:17 +08:00
Wenkai Yin
8768a5678c Merge Default ImageRegistry into the native adapter to reduce the duplicate code
Merge Default ImageRegistry into the native adapter to reduce the duplicate code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-07-01 19:21:12 +08:00
Wenkai Yin(尹文开)
174cfd5de5
Merge pull request #8172 from cd1989/azure-acr-adapter
Implement azure acr adapter
2019-07-01 19:06:19 +08:00
Yogi_Wang
c3572028b9 Fix the different width on vulnerability
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-07-01 17:26:59 +08:00
cd1989
c305103e05 Add unit test for azure adapter
Signed-off-by: cd1989 <chende@caicloud.io>
2019-07-01 14:31:12 +08:00
cd1989
2097e928d0 Implement azure acr adapter
Signed-off-by: cd1989 <chende@caicloud.io>
2019-06-29 18:34:48 +08:00
AllForNothing
322c79f348 fix filter ui bug #7700
Signed-off-by: sshijun <sshijun@vmware.com>
2019-06-28 15:48:24 +08:00
Wenkai Yin
6ba2ace0a6 Update replication label filter
Support specify multiple labels in one label filter

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-06-28 15:03:58 +08:00
Wenkai Yin
d2a938812f Support v1 signed media type when do the replication
Support v1 signed media type when do the replicatio

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-06-28 13:24:28 +08:00
Wenkai Yin
943dfd32fc Clean up the dao test for replication
Clean up the dao test for replication

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-06-28 13:18:26 +08:00
Ziming
a69cea9952
Merge pull request #8028 from bitsf/replication_gcr_ui_1.9
ui change: gcr/ecr driver for replication
2019-06-27 14:16:34 +08:00
Ziming
50180f0d7a
Merge pull request #7946 from bitsf/replication_gcr_1.9
gcr driver for replication
2019-06-27 14:16:07 +08:00
Will Sun
108b9284a5
Merge pull request #8150 from AllForNothing/robot-account-helm-ui
UI modification for creating robot account
2019-06-27 13:23:39 +08:00
Ziming
af548e915e
Merge branch 'master' into replication_gcr_1.9 2019-06-27 11:27:33 +08:00
Steven Zou
5521b7b7ad
Merge pull request #7915 from bitsf/replication_ecr_1.9
aws driver for replication
2019-06-27 11:24:54 +08:00
Wenkai Yin(尹文开)
fce920bbee
Merge pull request #8075 from reasonerjt/sys-vuln-whitelist-api
API for system level vulnerability whitelist
2019-06-27 10:53:09 +08:00
Mark Peek
3cae31da54 Switch registry health check to a 200 response url
The health check for the registry was using "/v2" which returned an
unauthorized response and put additional errors in the logs. Switch
to using "/" which returns an OK response with reduced logging.

Signed-off-by: Mark Peek <markpeek@vmware.com>
2019-06-26 14:23:08 -07:00
Daniel Jiang
4aca812ff2 API for system level vulnerability whitelist
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-06-26 23:35:40 +08:00
sshijun
42cf09fad6 UI modification for creating robot account
Signed-off-by: sshijun <sshijun@vmware.com>
2019-06-26 19:40:02 +08:00
Ziming Zhang
b9dbe429be ui change:gcr/ecr driver for replication
Change-Id: I52aa3a35da898d2501b09ff90b5a65169a6d54ed
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-06-26 09:52:32 +08:00
Ziming Zhang
072bdd101b aws driver for replication
Change-Id: I8792ffce2eaa5975359bb6159a1ba7b85926a925
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-06-25 19:11:27 +08:00
Ziming Zhang
e387c63242 gcr driver for replication
Change-Id: I5a6626950d3878bfa9726b332e68bee59159269f
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-06-25 18:08:10 +08:00
Wenkai Yin(尹文开)
f1becf8100
Merge pull request #8078 from tedgxt/optimize_image_delete_event_type
Set Event.Type value to event.EventTypeImageDelete instead of event.EventTypeImagePush when deleting
2019-06-25 16:59:38 +08:00
Wenkai Yin(尹文开)
8da61464b5
Merge pull request #8032 from ywk253100/190529_filter
Support label filter for replication
2019-06-25 16:59:25 +08:00
Will Sun
8ea9e44103
Merge pull request #8119 from AllForNothing/dev_ssj
fix bug: Save button not enabled when all the required fields are filled on Safari, Edge , Internet Explorer
2019-06-24 15:34:17 +08:00
Steven Zou
7c86c98687
Merge pull request #8049 from steven-zou/fix_js_ut_cases
fix failure UT case of job service
2019-06-21 14:55:50 +08:00
sshijun
412c6389d1 fix bug #7829 2019-06-21 14:53:22 +08:00
Steven Zou
b9f45cd30a fix failure UT case of job service
Signed-off-by: Steven Zou <szou@vmware.com>
2019-06-21 13:31:27 +08:00
Yogi_Wang
db4793d318 Fix issue click members and logs tab in project detail page portal will redirect to projects list page when call API failed
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-06-20 14:42:51 +08:00
guanxiatao
b40ee6edc9 Set Event.Type value to event.EventTypeImageDelete instead of event.EventTypeImagePush when deleting
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2019-06-20 09:39:45 +08:00
Wenkai Yin
684df243b2 Support label filter for replication
Only the system level labels are supported to be used as the replication policy filter

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-06-19 20:46:44 +08:00
Steven Zou
89ad59b0d4
Merge pull request #8026 from steven-zou/fix_bug_#7988
add migration process for rdb as well as UT cases
2019-06-17 10:58:42 +08:00
Steven Zou
9bac5e602d
Merge pull request #8030 from ywk253100/190605_replication_bugfix
Fix replication bug
2019-06-13 19:12:29 +08:00
Wenkai Yin
5fef7585c7 Fix replication bug
Fixes #7875, fixes #7968

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-06-13 17:17:29 +08:00
Steven Zou
2bc1402357 add migration process for rdb as well as UT cases
fix issue #7988

Signed-off-by: Steven Zou <szou@vmware.com>
2019-06-12 12:29:51 +08:00