Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-11-18 16:25:16 +01:00
Code Issues Projects Releases Wiki Activity
8,230 Commits 54 Branches 316 Tags 351 MiB
a087ba02e3
Commit Graph

585 Commits

Author SHA1 Message Date
Daniel Jiang
b21f9dc6f1 Support OIDC groups 
This commit enable project admin to add group as project member when
Harbor is configured against OIDC as AuthN backend.

It populates the information of groups from ID Token based on the claim
that is set in OIDC settings.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-09-19 17:49:31 +08:00
Steven Zou
4c4897aef1
Merge pull request #9134 from steven-zou/feature/pluggable_scanners 
support pluggable scanner
 2019-09-19 16:08:24 +08:00
Steven Zou
e324a4d623 support pluggable scanner 
- add DAO layer for scanner registration
- add CURD manager for scanner registration
- add API controller for plug scanner
- add REST APIs for CURD of plug scanner
- add migration sql:0011_1.10.0
- add scan interface definition (no implementations)
- add related UT cases with testify

fix #8979 #8990

Signed-off-by: Steven Zou <szou@vmware.com>
 2019-09-18 21:56:45 +08:00
wang yan
5498b5719b remove filter redeclared as imported package name in base.go 
It's introduced by https://github.com/goharbor/harbor/pull/8976

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-09-17 16:36:59 +08:00
Wang Yan
f77ce4aa3a
Merge pull request #8976 from ninjadq/add_auth_for_project_head 
Fix: Add authenticate to projects head
 2019-09-17 14:02:45 +08:00
Daniel Jiang
f36efa4dcd Add groups claim to OIDC configuration 
This commit add the new setting "oidc_groups_claim" to Harbor's
configurations.
And add "group_claim" to OIDCSetting struct.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-09-16 15:54:14 +08:00
Daniel Jiang
753219834e
Merge pull request #8960 from ninjadq/upgrade_hash_alg_for_pswd 
Upgrade hash alg for pswd
 2019-09-12 11:22:39 +08:00
DQ
ea5c27fcd5 Enhance: Upgrade encrypt alg to sha256 
previous sha1 will still used for old password

Signed-off-by: DQ <dengq@vmware.com>
 2019-09-09 21:48:21 +08:00
Wenkai Yin
3b07be5a72 Check the status behind error when trying to update the scan schedule 
Check the status behind error when trying to update the scan schedule

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-09-09 13:31:10 +08:00
Ziming Zhang
722e45b20b add swagger for tag retention 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I0f3ed8085e231868de74c273ba85946826181d5b
 2019-09-06 17:27:20 +08:00
DQ
4ac145b45b Fix: Add authenticate to projects head 
Project head only allow authenticated user to call.

Signed-off-by: DQ <dengq@vmware.com>
 2019-09-06 14:50:47 +08:00
gklp
dee3defeff refactored 
Signed-off-by: gklp <gokalpkuscu@gmail.com>
 2019-09-03 21:46:52 +03:00
wang yan
d8e17b122e add read only for quota switcher 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-09-03 14:08:39 +08:00
Wang Yan
2194834b41
Merge pull request #8910 from heww/foreign-layers 
fix(quota): correct size quota for image with foreign layers
 2019-09-03 00:29:24 +08:00
He Weiwei
f44b75f398 fix(quota): correct size quota for image with foreign layers 
1. Sync blobs from manifest for image with foreign layers.
2. Ignore size of foreign layers when compute size quota.
3. Fix repo info of artifact when upgrade from 1.8 version.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-09-02 14:10:58 +00:00
Wenkai Yin(尹文开)
7d151946e0
Merge pull request #8917 from reasonerjt/fix-create-admin-user 
Disallow creating an admin user when registration
 2019-09-02 18:43:41 +08:00
Wenkai Yin(尹文开)
d762e0730a
Merge pull request #8843 from heww/fix-chart-builder 
fix(chart,quota): fix path regexp of chart creation and deletion
 2019-09-02 15:59:18 +08:00
Iradier, AlvaroJose
bf9ac08c89 Handle error on OIDC callback 
If wrong OIDC scopes are defined, or there are some configuration errors, the OIDC callback query string might contain "error=..." with an error message. Intercept this case and show an error to the user instead of trying to exchange the token with a missing "code" parameter.

Signed-off-by: Iradier, AlvaroJose <AlvaroJose.Iradier@adidas.com>

Change error variable name

Signed-off-by: Iradier, AlvaroJose <AlvaroJose.Iradier@adidas.com>
 2019-08-30 13:44:12 +02:00
Wenkai Yin(尹文开)
8c67f71b5e
Merge pull request #8902 from wy65701436/fix-8888 
fix #8888
 2019-08-30 18:09:56 +08:00
wang yan
dd9f028fe0 fix #8888 
the image may has the same blobs as the references, which causes the artifact & blobs
can not be inserted by unique constraint

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-30 15:30:05 +08:00
Wang Yan
240b718508
Merge pull request #8887 from wy65701436/fix-8886 
fix(quota/sync) #8886
 2019-08-30 15:06:35 +08:00
wang yan
16b910e1cf fix(quota/sync) #8886 
The foreign layer won't be counted into project quota
NOTE: the foreign layer will be dumped from the registry in the migration

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-29 17:29:40 +08:00
Ziming Zhang
8fb6e2f65b verify permission of GetRetentionExecTaskLog 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I4bf4ddf3d3ed6f07a4618e242e2f3774996716d6
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-29 15:01:53 +08:00
Wang Yan
db5781bf78
Merge pull request #8860 from wy65701436/fix-quota-sync 
fix quota sync issues
 2019-08-29 13:45:38 +08:00
wang yan
5decb56369 update code per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-29 12:46:42 +08:00
wang yan
942e793f20 fix quota sync issues 
1, fix #8858, add retry to ping backend service
2, fix #8859, split the blobs data when larger then 65535

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-28 18:59:25 +08:00
Daniel Jiang
b6db8a8a10 Disallow creating an admin user when registration 
This commit enhance the `POST /api/users` API to block request from non-admin to create
admin user.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-08-28 18:30:27 +08:00
Qian Deng
ed54b1da45
Merge pull request #8857 from wy65701436/fix-syne-readonly 
fix quota sync read only setting
 2019-08-28 17:04:02 +08:00
Ziming
94138137d5
add valid for rule (#8846) 
Change-Id: I82215a0cf1ec32a253c8db9bfafe7e25b26c9ad9
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-28 16:58:49 +08:00
wang yan
19a37282c1 fix quota sync read only setting 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-28 14:53:01 +08:00
Wang Yan
87893abc5e
Merge pull request #8829 from ywk253100/190822_retry_status 
Add status revision to handle retrying in replication task
 2019-08-28 10:55:13 +08:00
Qian Deng
ea33561d95
Merge pull request #8747 from 928234269/patch1 
fixt typo "an user" to "a user"
 2019-08-27 18:26:11 +08:00
Wang Yan
be1e702d9d
Merge pull request #8787 from cd1989/core-hunging 
Fix core hung when stop problem
 2019-08-27 15:56:21 +08:00
Wenkai Yin
7924f37d86 Add status revision to handle retrying in replication task 
Add status revision to handle retrying in replication task

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-27 14:17:11 +08:00
He Weiwei
1d8eac8cf4 fix(chart,quota): fix path regexp of chart creation and deletion 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-27 01:04:10 +00:00
wang yan
f343b2ec45 Revise quota errors to make it more readable 
1, fix #8802, update the error formet
2, fix #8807, raise the real retag error to UI
3, fix #8832, raise the real chart error to chart client & ut

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-26 17:00:31 +08:00
Wang Yan
e7488e37b6
Merge pull request #8788 from bitsf/tag_retention_validate_model 
add tag retention model test
 2019-08-23 13:57:51 +08:00
Ziming Zhang
39db65e90f add tag retention model test 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I3f8b06b994024fe6052b0dee87600ed932edaaff
 2019-08-23 13:24:28 +08:00
mmpei
d5f87063e4
Merge branch 'master' into official-wehook-events-20190811 2019-08-22 22:07:12 -05:00
Wang Yan
35e786e54c
Merge pull request #8794 from ywk253100/190822_retry_status 
Add status revision to retention task to handle retrying
 2019-08-23 10:54:35 +08:00
peimingming
599d12a04d Fix bugs by comments for webhook 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2019-08-23 09:44:51 +08:00
cd1989
4b59346423 Exist core when signal received 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-23 09:34:41 +08:00
Wang Yan
299032d602
Merge pull request #8771 from wy65701436/fix-manifest-dup 
fix quota count size for same manifest in different repo
 2019-08-23 08:37:03 +08:00
wang yan
2d569192ab fix quota count size for same manifest in different repo 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-23 00:56:45 +08:00
Wenkai Yin
661470e7bc Add status revision to retention task to handle retrying 
Add status revision to retention task to handle retrying

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-22 20:04:25 +08:00
wang yan
83a3274a96 fix #8688 
use the docker defined error to avoid retry pushing on quota overflow

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-22 19:19:43 +08:00
Wenkai Yin(尹文开)
21f8290110
Merge pull request #8777 from heww/issue-8635 
fix(rbac): NewProjectNamespace in rbac only accept projectID
 2019-08-22 17:52:27 +08:00
wang yan
dfa4301b21 fix deleted project error when to migrate quota 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-22 14:51:26 +08:00
Wenkai Yin(尹文开)
6198ed2634
Merge pull request #8758 from heww/issue-8681 
refactor(quota,middleware): skip overflow error when subtract resources
 2019-08-22 13:54:01 +08:00
He Weiwei
8effdc6f18 fix(rbac): NewProjectNamespace in rbac only accept projectID 
Closes #8635

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-21 19:37:28 +00:00
Ziming Zhang
06e9467b06 1. remove rule none 
2. change rule orders
3. remove laber selector

Change-Id: Idc18a27cb0267f5f5c80a04b381e4a5dc6998508
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-21 20:24:33 +08:00
Wang Yan
c1ad0518c9
Merge pull request #8582 from ethan-daocloud/patch-1 
core/main.go: logging message correction
 2019-08-21 13:49:00 +08:00
928234269
7a4fde5ec9
fixt typo "an user" to "a user" 
Signed-off-by: 928234269 <longfei.shang@daocloud.io>
 2019-08-21 10:20:35 +08:00
He Weiwei
c22bf2539e refactor(quota,middleware): skip overflow error when subtract resources 
1. Skip overflow error when subtract resources
2. Take up resources before handle request and put it back when handle
failed for add action in quota interceptor
3. Free resources only after handle success for subtract action in quota
interceptor

Closes #8681

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-20 14:41:55 +00:00
Steven Zou
217252a097
Merge pull request #8675 from ywk253100/190814_retention_task 
Handle the retention task status updating in concurrency
 2019-08-20 17:07:21 +08:00
Steven Zou
f6067d5474
Merge pull request #8708 from bitsf/tag_retention_rule_conflict 
fix rule conflict, id error
 2019-08-20 16:45:21 +08:00
Wang Yan
92fa01d6cd
Merge pull request #8733 from ywk253100/190819_label 
Fix bug of listing tags filtered by label
 2019-08-20 14:17:41 +08:00
Wenkai Yin(尹文开)
29ab93ad9c
Merge pull request #8714 from ywk253100/190816_scheduler 
Fix bug found in scheduler
 2019-08-20 14:11:04 +08:00
Wenkai Yin(尹文开)
0086d1b211
Merge pull request #8586 from ethan-daocloud/patch-2 
project_test.go: many typos "respose" fixup
 2019-08-20 13:54:37 +08:00
Wang Yan
f930786050
Merge pull request #8725 from heww/issue-8701 
fix(quota): fix computeResources method of qutoa interceptor
 2019-08-20 11:29:59 +08:00
Daniel Jiang
f10fb67d6d
Merge pull request #8662 from stonezdj/email_sec2 
Set default email to null if not provided
 2019-08-20 09:01:50 +08:00
Wenkai Yin
a5d292c9c6 Fix bug of listing tags filtered by label 
Fixes #8249

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-19 16:10:16 +08:00
stonezdj
5fa8eb7854 Set default email to null if not provided 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-08-19 15:20:44 +08:00
Daniel Jiang
b3abd0316b
Merge pull request #8713 from reasonerjt/fix-8702 
Avoid overwriting system CVE whitelist by mistake
 2019-08-19 01:42:58 +08:00
He Weiwei
1bbfc023f1 fix(quota): fix computeResources method of qutoa interceptor 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-18 16:14:36 +00:00
wang yan
b9d6108624 add ping for adapter to wait for service ready 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-16 16:03:12 +08:00
wang yan
6e11ecc6fc Update codes per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-16 14:58:52 +08:00
Wang Yan
7a41d89ac8 Add quota sync api toi to sync quota data with backend storage 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-16 14:55:46 +08:00
Daniel Jiang
022d4e6ae8
Merge pull request #7462 from cd1989/enable-tags-detail-param 
List simple tags when detail set to false
 2019-08-16 14:25:29 +08:00
Wenkai Yin
5c286d799f Fix bug found in scheduler 
The scheduler hook handler doesn't parse the job status struct when handling the hook. This commit fixes it.

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-16 13:54:57 +08:00
Daniel Jiang
30bb2ddcdf Avoid overwriting system CVE whitelist by mistake 
Fixes #8702
Also enforce the code to mitigate the potential risk.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-08-16 13:28:16 +08:00
Ziming
c279b7f3e9
fix retention rule compute error (#8664) 
Change-Id: I16d7284b17508885e136f2d9ea5651978ba4a6d8
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-15 20:12:59 +08:00
Ziming Zhang
4dc6f12784 fix rule conflict, id error 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I379f3c29cee742d346fb57ade397be64fd76f59d
 2019-08-15 20:11:30 +08:00
Wenkai Yin
48b067f596 Handle the retention task status updating in concurrency 
Compare the status code when updating retention task status to avoid the concurrent issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-15 18:36:18 +08:00
Ziming Zhang
f854d4a25d add always rule again 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I34bac3b73e693b89e0b9debb28617352656ed1d9
 2019-08-15 15:26:22 +08:00
Ziming
4944799f70
Merge pull request #8651 from bitsf/remove_tagretention_always 
remove tag retention rule always
 2019-08-15 14:35:26 +08:00
Wang Yan
bf0b5a3fd0
Merge pull request #8663 from wy65701436/fix-quota-api 
Fix quota switch fail to get project size
 2019-08-15 10:49:49 +08:00
Qian Deng
4611630ab6
Merge pull request #8616 from heww/db-connection-pool 
feat(configuration,db): connection pool configs for db
 2019-08-15 09:48:20 +08:00
Daniel Jiang
b0c8561b54
Merge pull request #8189 from mmpei/8162-optimize-manifest-wait 
Optimize fetch manifest loop when handling notification
 2019-08-15 00:29:53 +08:00
Daniel Jiang
d9e1b90c5b
Merge pull request #8653 from vmlaguna/vmlaguna/template-fix 
Close <a> tag in error template
 2019-08-15 00:11:28 +08:00
wang yan
a947a4259d Fix quota switch fail to get project size 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-14 22:32:32 +08:00
Wang Yan
54cb39b7a7
Merge pull request #8655 from wy65701436/log-size 
Add log in quota size handler
 2019-08-14 16:09:45 +08:00
Pedro Laguna
3c34675e0f Close <a> tag in error template 
The error template is missing a closing <a> tag in the link to the harbor portal.

Signed-off-by: Pedro Laguna <44469313+vmlaguna@users.noreply.github.com>
 2019-08-14 08:37:44 +01:00
He Weiwei
98e1f68468 feat(configuration,db): connection pool configs for db 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-14 14:30:34 +08:00
wang yan
9601c9f206 Add log in quota size handler 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-14 14:10:09 +08:00
wang yan
838a923d23 fix code by review 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-14 12:47:12 +08:00
wang yan
9e0addee55 Enable usage sync when switch quota setting 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-08-14 12:47:12 +08:00
Ziming Zhang
d7e9924a01 remove tag retention rule always 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Ide98d58f64596e8110fbb9eb0d73df6ccbd99c80
 2019-08-13 21:10:57 +08:00
Steven Zou
1adc3a9469
Merge pull request #8606 from ywk253100/190807_stuck 
Fix replication tasks stuck in "InProgress" issue
 2019-08-13 15:59:20 +08:00
cd1989
da370bb331 List simple tags when detail set to false 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-12 15:16:19 +08:00
He Weiwei
c1cea42089 feat(quota,middleware): enable or disable quota per project by config 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-12 00:02:26 +00:00
peimingming
222c47142a Add chart and scanning event for webhook 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2019-08-11 18:01:07 +08:00
Wang Yan
54a39c7159
Merge pull request #8597 from heww/size-quota 
refactor(quota,middleware): implement size quota by quota interceptor
 2019-08-09 15:44:33 +08:00
Steven Zou
d2fbb98a8d
Merge pull request #8592 from bitsf/tag_retention_conflict_rule 
check rule conflict
 2019-08-09 13:59:46 +08:00
He Weiwei
e62c29123d refactor(quota,middleware): implement size quota by quota interceptor 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-08 23:55:54 +00:00
Wenkai Yin
8777c07d47 Fix replication tasks stuck in "InProgress" issue 
Fix replication tasks stuck in "InProgress" issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-08 15:42:42 +08:00
Ziming Zhang
fb5acdc64b check rule conflict 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Iea8c9ff4702873cb4ab3ebd943deec22bec418e6
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-08 14:23:53 +08:00
guanxiatao
e7fafd1941 webhook policy, job, event support 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2019-08-07 20:30:26 +08:00
Ziming Zhang
730d95edc8 check rule conflict 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I95f7e683b30fa6059b5bb49a2fc8f78dd7276079
 2019-08-07 17:52:47 +08:00
Steven Zou
f3ba25f656
Merge pull request #8536 from bitsf/tag_retention_task_num 
add task retain num
 2019-08-07 17:39:39 +08:00
Ziming Zhang
3b62addc76 check rule conflict 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I8af648f22aa3fe76024240d322a7fdae560e3fbd
 2019-08-07 15:38:21 +08:00
ethan
f463b7c449 project_test.go: many typos "respose" fixup 
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
 2019-08-07 13:39:05 +08:00
Wenkai Yin
216ef269b3 Populate pull/push time properties to the returning data when listing tags 
Populate pull/push time properties to the returning data when listing tags

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-07 11:47:05 +08:00
Wenkai Yin(尹文开)
556e07f0c4
Merge pull request #8551 from nlowe/feat/retention/GH-8548-Pulled-in-the-last-N-Days 
Retention: New Evaluator: Pulled within the last N Days
 2019-08-07 11:37:46 +08:00
Qian Deng
635066da92
Merge pull request #8466 from reasonerjt/remove-clair-notification 
Remove clair notifier
 2019-08-07 10:53:35 +08:00
Nathan Lowe
9a7df265ce
Retention: New Evaluator: Pulled within the last N Days 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-08-06 22:28:28 -04:00
Wenkai Yin(尹文开)
ec4fa753d7
Merge pull request #8552 from nlowe/feat/retention/GH-8549-Pushed-in-the-last-N-Days 
Retention: New Evaluator: Retain Images pushed within N days
 2019-08-07 09:55:05 +08:00
ethan
edd102c3b7
core/main.go: logging message correction 
Signed-off-by: ethan <guangming.wang@daocloud.io>
 2019-08-06 21:56:18 +08:00
Ziming Zhang
498a813299 retain nothing rule 
Change-Id: I4e7a4ecb40fe39b80e41a6d9bf8b5fb3968a41af
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-06 15:10:27 +08:00
Nathan Lowe
756352d271
Add metadata for daysps to the API for the UI 
Signed-off-by: Nathan Lowe <public@nlowe.me>
 2019-08-05 20:08:34 -04:00
Daniel Jiang
eec4fc2798 Remove clair notifier 
The way Harbor handles notification is problematic.
It currently triggers rescan, which will cause problem when there are
lot of images in the registry.
Such as #7316
This commit removes the notifier and we need to revisit the notification
to figure out how to map the notification to a particular image if need
the notification mechanism in future.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-08-06 01:58:15 +08:00
Ziming Zhang
3a80123a82 1. remove rule always 
2. update i18n

Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I0e2394f0a6b283d0efef7a44ed2d4afb9745eabd
 2019-08-05 17:59:00 +08:00
Ziming Zhang
026aee75d9 add task retain num 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I2f8b89454fe3bb9b56af237048c9e2b90783f434
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-08-05 17:57:09 +08:00
Ziming Zhang
decffdd6a4 add total page num for tag retention 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I58a250dbb643f6949c1e34aa3a84a01dc3e0b285
 2019-08-02 14:43:52 +08:00
He Weiwei
9778954852 feat(quota,middleware): image count quota support 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-08-01 14:48:59 +08:00
He Weiwei
8cc9314984
feat(helm-chart,quota): count quota support for helm chart (#8439) 
* feat(helm-chart,quota): count quota support for helm chart

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-31 16:48:40 +08:00
wang yan
4410cc93f9 add internal reg request handler chain 
this is for internal registry api call, the request should be intercpeted by quota middlerwares, like retag and delete.
Note: The api developer has to know that if the internal registry call in your api, please consider to use
NewRepositoryClientForLocal() to init the repository client, which can handle quota change.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-30 19:39:56 +08:00
Ziming
a5c31d7160
Merge pull request #8456 from ywk253100/190729_scheduler 
Register the scheduler job and fix bugs
 2019-07-30 08:57:10 +08:00
Steven Zou
4bf7f7b3e4
Merge pull request #8445 from steven-zou/fix/tag_retention 
refactor index registering processes
 2019-07-30 07:58:14 +08:00
Wenkai Yin
94c1cf8ac9 Register the scheduler job and fix bugs 
1. Register the scheduler job
2. Fix retention bugs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-30 07:37:47 +08:00
Wenkai Yin(尹文开)
c99f12fe33
Merge pull request #8444 from bitsf/tag_retention_exec_status 
get execution status on the fly
 2019-07-29 17:08:37 +08:00
Wang Yan
ca20281b35
Merge pull request #8438 from wy65701436/fix-quota-bug 
Fix logic error in quota middlewware on handling failure response
 2019-07-29 16:02:57 +08:00
Ziming Zhang
ba47b4c00f get execution status on the fly 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: Iefcf8946d7a2c7a27bc22bd326ee9723b4b79c66
 2019-07-29 14:48:39 +08:00
Steven Zou
6479a22a08 refacor index registering processes 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-29 14:01:47 +08:00
wang yan
1975f4a71d Fix logic error in quota middlewware on handling failure response 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-28 12:37:58 +08:00
wang yan
a23ff4e448 Update pull time in artifact table for docker image pull 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-28 12:30:20 +08:00
Ziming Zhang
83b045f5ec add ut for tag retention controller 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I1469ee13675537ec389a068e4bc29e457b402fa4
 2019-07-26 17:55:11 +08:00
Wenkai Yin(尹文开)
d45674960f
Merge pull request #8417 from goharbor/project-quota-dev 
Add feature project quota dev
 2019-07-26 15:41:09 +08:00
wang yan
54101180ae Update log level for init proxy 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-26 13:39:06 +08:00
He Weiwei
c566a48880 fix(api,project): fix repo, chart count missing in project summary 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-26 13:15:44 +08:00
wang yan
676b922c95 merge with latest master code with quota branch 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-26 01:32:45 +08:00
Wang Yan
b9ea3731f7
Merge pull request #8350 from wy65701436/blob-flow-dev 
Add size middleware to support quota
 2019-07-26 01:25:40 +08:00
Wang Yan
1dfc47d24e Add size middleware to support quota 
[Add]:
1, size middleware for quota size
2, count middleware for quota artifact count

[Support]:
1, put, patch, mount blob
2, put manifest

[Refactor]:
1, Add handle response for middlerware
2, Remove the modifyResponse for registry proxy
3, Use the custom response writer to recored status

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-26 00:28:36 +08:00
Wenkai Yin
2e9521ad45 Support to stop one execution of retention 
Support to stop one execution of retention

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-25 16:48:00 +08:00
He Weiwei
f3a2280033
Merge pull request #8384 from heww/quota-apis 
feat(quota,api): APIs for quotas
 2019-07-25 15:19:46 +08:00
He Weiwei
e625f2aa11 feat(quota,api): APIs for quotas 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-25 13:40:26 +08:00
Steven Zou
584dcd8571 support dry run of retention 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-25 12:53:19 +08:00
wang yan
4763864dae merge with latest master code with quota feature branch 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-24 08:47:05 -07:00
Steven Zou
c44747fd3c merge code from master and fix conflicts 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-24 17:27:37 +08:00
Wenkai Yin(尹文开)
cd411f6588
Merge pull request #8385 from ywk253100/190724_task_hook 
Implement the webhook handler for retention task
 2019-07-24 17:22:57 +08:00
Ziming
43c2af9857 map retention with policy (#8313) 
Signed-off-by: Ziming Zhang <zziming@vmware.com>

Implement the API and controller of tag retention
 - API handler
 - retention controller
 - dao
 2019-07-24 17:22:26 +08:00
Wenkai Yin
e2808f12cf Implement the webhook handler for retention task 
Implement the webhook handler for retention task

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-24 14:32:05 +08:00
He Weiwei
ce58c58c01 feat(quota,api): quota support for create project API 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-24 01:02:51 +08:00
Wenkai Yin
7362fae7cc Implement a common scheduler 
Implement a common scheduler that can be used globally

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-23 17:20:31 +08:00
wang yan
2292954a31 Merge branch 'master' of https://github.com/goharbor/harbor into project-quota-dev 2019-07-22 15:46:09 +08:00
Wang Yan
e8565a4539
Merge pull request #8335 from reasonerjt/add-oidc-ping-api 
Add API to ping OIDC endpoint
 2019-07-22 14:30:24 +08:00
Wang Yan
834e604ec0
Merge pull request #8246 from ninjadq/fix_chart_museum_500_error 
Fix: Internal server error with messy code when chartmuseum not work
 2019-07-22 11:07:55 +08:00
Steven Zou
deb021eb48 rebase and fix name conflicts among packages and vars 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-19 22:37:54 +08:00
Wenkai Yin
ddbde72d2e Move the pkg manager/controller declaration to the api package 
Move the pkg manager/controller declaration to the api package to avoid the dependency cycle

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-19 17:21:58 +08:00
Steven Zou
f0ea62caa9 Merge branch 'feature/tag_retention' into feature/tag_retention-performer 2019-07-19 16:00:13 +08:00
Steven Zou
c2b5d95e02 implement retain action performer 
Signed-off-by: Steven Zou <szou@vmware.com>
 2019-07-19 15:59:21 +08:00
Wenkai Yin
5f1d2bd644 Fix package import cycle issue 
Fix package import cycle issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-19 13:50:55 +08:00
He Weiwei
9c9b8d3a6d Merge branch 'master' into project-quota-dev 2019-07-19 10:02:51 +08:00
Daniel Jiang
96e2e0b145 Add API to ping OIDC endpoint 
This commit adds an API to help admin verify the OIDC endpoint is a
valid one.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-18 19:32:12 +08:00
stonezdj
13772b859e Fix OnBoardGroup issue 
Signed-off-by: stonezdj <stonezdj@gmail.com>

Fix issue when adding a HTTP user group to a project member, returns HTTP 500 error.
 2019-07-18 19:19:09 +08:00
Steven Zou
746d082e2e Merge branch 'master' into feature/tag_retention 2019-07-18 10:40:49 +08:00
Wenkai Yin(尹文开)
a64e089773
Merge pull request #8210 from stonezdj/http_group_dao2 
Add HTTP group support
 2019-07-17 15:22:36 +08:00
DQ
af58195a29 Fix: Internal server error with messy code when chartmuseum not work 
log err when doesn't get data from chart museum

Signed-off-by: DQ <dengq@vmware.com>
 2019-07-17 15:14:50 +08:00
Wenkai Yin
d6c6231e08 Implement the retention client 
Implement the retention client

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-17 14:04:29 +08:00
stonezdj
bb2ae7c093 Add HTTP group feature 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-07-16 15:38:46 +08:00
Ziming Zhang
c22c38994a retention api 
Change-Id: I70f2c34d6bb96ecf4cb5359e2b1ab2dbb99fdbf9
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-07-16 15:06:37 +08:00
Wang Yan
8ac6bdbbb0 Add quota workflow for quota 
1, apply count for manifest if it's a new image
2, insert data for artifact and artifact_blob

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-16 14:48:05 +08:00
wang yan
f066d986b9 merge with latest master code 2019-07-11 20:21:15 +08:00
Wenkai Yin
91b050a01b Implement the launcher 
The commit implements the launcher for tag retention

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-11 17:18:46 +08:00
Wenkai Yin(尹文开)
3bebf7bc64
Merge pull request #8238 from reasonerjt/project-cve-whitelist 
Enable project level CVE whitelist
 2019-07-10 14:41:01 +08:00
wang yan
6d0271ee5c Merge branch 'master' of https://github.com/goharbor/harbor into project-quota-dev 2019-07-10 10:57:10 +08:00
wang yan
7b38389898 update codes per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>

fix middlewares per review comments
1, add scheme1 and scheme2 check
2, change MustCompile to Compile

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-09 11:08:19 +08:00
wang yan
b3c5137a2f add copyright and fix codecy 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-09 11:08:19 +08:00
wang yan
57821b1b4c Refactor interceptors code with chain 
1, add a blob inteceptors for quota usage
2, add a manifest inteceptors for quota usage

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-07-09 11:08:11 +08:00
Daniel Jiang
8f5f0031c7 Enable project level CVE whitelist 
This commit update the project API to support "reuse_sys_cve_whitelist"
setting in project metadata and "cve_whitelist" in project request.
Also modify the interceptor to support project level CVE whitelist if
the reuse flag is false.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-08 18:55:54 +08:00
Daniel Jiang
c296f0ddfb
Merge pull request #8176 from stonezdj/http_group 
Refactor LDAP usergroup
 2019-07-08 09:54:31 +08:00
stonezdj
c0ed55445d Refactor LDAP group 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-07-05 14:44:18 +08:00
Daniel Jiang
8a9d352f54 Handle helm push in OIDC filter 
Fixes #8130
Enable OIDC filter to handle requests to /api/chartrepo/*

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-03 17:29:37 +08:00
Daniel Jiang
5d887ad0d8
Merge pull request #8179 from reasonerjt/interceptor-use-whitelist 
Apply CVE white list in interceptor
 2019-07-03 15:12:33 +08:00
Daniel Jiang
bba4b2a6a4 Apply CVE white list in interceptor 
Interceptor will filter the vulnerability in whitelist while calculating
the serverity of an image and determine whether or not to block client
form pulling it.

It will use the system level whitelist in this commit, another commit
will switch to project level whitelist based on setting in a project.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-07-03 14:13:00 +08:00
Daniel Jiang
6f166bc02c
Merge pull request #8154 from markpeek/markpeek-registry-health-check 
Switch registry health check to a 200 response url
 2019-07-03 10:29:35 +08:00
He Weiwei
720dcc72bd Fix read permission of project member read api 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2019-07-02 14:40:46 +08:00
mmpei
5dfc3f2402 Optimize fetch manifest loop when handling notification 
Signed-off-by: mmpei <peimingming@corp.netease.com>
 2019-07-01 17:54:52 +08:00
Steven Zou
5521b7b7ad
Merge pull request #7915 from bitsf/replication_ecr_1.9 
aws driver for replication
 2019-06-27 11:24:54 +08:00
Wenkai Yin(尹文开)
fce920bbee
Merge pull request #8075 from reasonerjt/sys-vuln-whitelist-api 
API for system level vulnerability whitelist
 2019-06-27 10:53:09 +08:00
Mark Peek
3cae31da54 Switch registry health check to a 200 response url 
The health check for the registry was using "/v2" which returned an
unauthorized response and put additional errors in the logs. Switch
to using "/" which returns an OK response with reduced logging.

Signed-off-by: Mark Peek <markpeek@vmware.com>
 2019-06-26 14:23:08 -07:00
Daniel Jiang
4aca812ff2 API for system level vulnerability whitelist 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-06-26 23:35:40 +08:00
Ziming Zhang
072bdd101b aws driver for replication 
Change-Id: I8792ffce2eaa5975359bb6159a1ba7b85926a925
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-06-25 19:11:27 +08:00
guanxiatao
b40ee6edc9 Set Event.Type value to event.EventTypeImageDelete instead of event.EventTypeImagePush when deleting 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2019-06-20 09:39:45 +08:00
Steven Zou
9bac5e602d
Merge pull request #8030 from ywk253100/190605_replication_bugfix 
Fix replication bug
 2019-06-13 19:12:29 +08:00
Wenkai Yin
5fef7585c7 Fix replication bug 
Fixes #7875, fixes #7968

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-06-13 17:17:29 +08:00
Daniel Jiang
c928924fa5
Merge pull request #8003 from reasonerjt/bump-up-go112 
Bump up to go 1.12.5 enable go.mod
 2019-06-11 13:21:32 +08:00
wang yan
a4b202d656 remove the id in the post body when to create a robot account 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-06-11 10:47:56 +08:00
Daniel Jiang
737eaa396c Bump up to go 1.12.5 enable go.mod 
This commit bumps up the version of Go to compile the code to v1.12.5,
and shifts to go.mod for managing depedency.
Some code from "harbor/tests" to "harbor/src/testing" to avoid depedency
loop of modules.

Note that in short term we will still vendor the dependency.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-06-11 00:42:20 +08:00
Qian Deng
592e40bacf
Merge pull request #7859 from wy65701436/return-robot-id 
Return account id when to issue a robot
 2019-05-29 10:55:20 +08:00
wang yan
15ca9bfb81 Add ensure settings when to search user in auth proxy 
This issue is regresssion that introduced by f92bc8076d (diff-42381e6df5f17ebd3d9165a325d5d8f4), the a.ensure() was removed from SearchUser(), which leads to the alwaysonboard cannot be updated.

Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-22 17:16:16 +08:00
wang yan
056cfc7e31 Return account id when to issue a robot 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-22 10:39:26 +08:00
wang yan
2068732eef add validation for robot account registration 
Signed-off-by: wang yan <wangyan@vmware.com>
 2019-05-15 15:03:35 +08:00
Wang Yan
3be14b7997
fix issue7793: ping ldap server is always success (#7795) 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-05-15 11:35:57 +08:00
Wenkai Yin
e399ffca54
Merge pull request #7799 from reasonerjt/oidc-onboard-user-name 
Set the real name of OIDC user when onboarding
 2019-05-15 11:35:42 +08:00
Daniel Jiang
5f11dbe675 Set the real name of OIDC user when onboarding 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-15 00:44:51 +08:00
stonezdj
99c0a5a498 fix issue7793: ping ldap server is always success 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2019-05-14 19:20:54 +08:00
Qian Deng
f607c5177d Fix frontend failure caused by absolute path 
Fix failures because front downlowd chart using relative path

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-14 13:22:06 +08:00
Qian Deng
cd6c5a9f10 Enable absolute url in helm chart 
assign public_url to chart-url
remove namespace merge in index.yaml

Signed-off-by: Qian Deng <dengq@vmware.com>
 2019-05-14 12:56:20 +08:00
Daniel Jiang
e963ee88c7 Update login controller to return 403 for redirection 
As the UI cannot handle 302, update the login controller to return 403
and put the redirection URL in a json response body.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2019-05-13 18:16:06 +08:00