Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2025-01-16 12:41:22 +01:00
Code Issues Projects Releases Wiki Activity
10,271 Commits 50 Branches 323 Tags 347 MiB
3ddc44e28b
Commit Graph

4763 Commits

Author SHA1 Message Date
Wenkai Yin(尹文开)
3408f0b577
Merge pull request #13909 from ywk253100/210106_task 
Provide a mechanism to sweep the execution/task records in task manager
 2021-01-07 13:44:21 +08:00
Will Sun
bd585481c7
Fix tag retention ui loading issue (#13905) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-07 10:20:12 +08:00
peimingming
28714f8b70 Support artifact hub replication using new API 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2021-01-06 20:46:04 +08:00
He Weiwei
ed31cf9417
feat: return scan report and summary by header (#13898) 
Add X-Accept-Vulnerabilities header to the list/get artifact and get
artifact vulnerability addition APIs, and these APIs will traverse the
mime types in this header and return the first report and summary found
from the mime type.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-06 17:54:36 +08:00
Wenkai Yin
fb871dbbe8 Provide a mechanism to sweep the execution/task records in task manager 
Provide a mechanism to sweep the execution/task records in task manager
Fixes #13888

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-06 16:41:46 +08:00
He Weiwei
e33ff932d1
test: fix the data race in the test of scan controller (#13899) 
Closes #13876

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2021-01-06 16:17:47 +08:00
stonezdj(Daojun Zhang)
b748852ee8
Merge pull request #13805 from stonezdj/201218_add_content_type_length 
Add content type and length in header
 2021-01-06 15:02:34 +08:00
Will Sun
a1a5ef5e20
support quay registry as proxy cache (#13902) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-06 14:15:15 +08:00
stonezdj
1d50be31aa Refactor Manifest cache process 
Separate manifest, manifest list and image index.

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2021-01-06 14:01:02 +08:00
Daniel Jiang
06e993ff76
Remove scanner-pull from system admin's permission (#13901) 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-01-06 12:35:58 +08:00
Wenkai Yin
d474750e9f Fix replication filter bug 
This commit fixes the bug of replication filter, see #13593 for more detail
Fixes #13593

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2021-01-06 09:36:31 +08:00
Will Sun
839c36c876
Fix bugs for robot account UI (#13894) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2021-01-05 22:28:56 +08:00
Ziming Zhang
8faa76a1b6 feat(retention) refactor task manager 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2021-01-05 12:08:03 +08:00
shaobo
434758b71e reform job status compare method 
Signed-off-by: shaobo <shaobo@caicloud.io>
 2021-01-04 21:34:53 +08:00
Daniel Jiang
d0152cb446
Merge pull request #13872 from reasonerjt/token-scope 
Refine the token scope generation
 2021-01-04 11:16:59 +08:00
Wang Yan
0271efd3f7
enable visible when to list/create robot (#13840) 
1, enable the visible attribute when to create/list robots
2, rename package name from robot2 to robot

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2021-01-04 10:24:31 +08:00
Daniel Jiang
eb75123638 Refine the token scope generation 
This commit directly maps the actoin permission in security context to
the scope generated by the token service in harbor-core.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2021-01-03 23:12:04 +08:00
prahaladdarkin
a890b28e1e
Store vulnerability data from scanner into a relational format (#13616) 
feat: Store vulnerability report from scanner into a relational format

Convert vulnerability report JSON obtained  from scanner into a relational format describe in:https://github.com/goharbor/community/pull/145

Signed-off-by: prahaladdarkin <prahaladd@vmware.com>
 2020-12-25 08:47:46 +08:00
Wang Yan
47841a04b9
fix quota cannot be updated after gc (#13844) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-24 20:10:57 +08:00
Will Sun
75da08303a
Improve scan all page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-24 17:36:03 +08:00
Wenkai Yin
923a538570 Provide the function to update extra attributes in the task manager 
Provide the function to update extra attributes in the task manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-12-24 15:52:37 +08:00
He Weiwei
7b4c4b76e7
feat: add trigger to the metrics of the scan all job (#13838) 
Add the trigger to the metrics of the scan all job so that the customer
can know who trigger the latest scan all job.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-24 14:30:43 +08:00
Wenkai Yin(尹文开)
d32a243508
Use the same transaction to remove the schedule to avoid dead lock in database (#13820) 
Use the same transaction to remove the schedule to avoid dead lock in database
Fixes #13808

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-12-23 15:37:51 +08:00
Will Sun
28734d7ac4
Fix a save button bug on system setting page (#13830) 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-23 13:00:45 +08:00
Daniel Jiang
c660727877
Merge pull request #13800 from reasonerjt/authproxy-redirect 
Add handler to handle redirect via authproxy
 2020-12-23 03:00:18 +08:00
stonezdj(Daojun Zhang)
bc0b6b43ed
Merge pull request #13791 from reasonerjt/oidc-redirect-extra-parm 
Add extra parms when forming redirect URI for OIDC
 2020-12-22 21:45:53 +08:00
Daniel Jiang
7321e3547d Add handler to handle redirect via authproxy 
This commit add a handler to handle the request to
"/c/authproxy/redirect".  Harbor is configured to authenticate against
an authproxy, if a request with query string `?token=xxxx`
is sent to this URI, the handler will do tokenreview according to the
setting of authproxy and simulate a `login` workflow based on the result
of token review.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-22 18:59:17 +08:00
He Weiwei
3831e82b20
refactor: remove code of admin job (#13819) 
Remove code of admin job as it's not needed by scan all/gc now.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-22 11:48:16 +08:00
He Weiwei
8fa03e3739
refactor: remove code of scan all job (#13821) 
Remove code of scan all job as it's implemented by execution now.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-22 11:39:18 +08:00
stonezdj
aa3002e7a5 Add content type and length in header 
Fixes #13740
Update ManifestExist to return Descriptor instead of digest
For docker 20.10 or containerd, it HEAD the manifest before pull, then
it GET the manifest with digest, add logic to handle this scenario and
correlate the tag between the digest in proxy cache

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-21 20:21:28 +08:00
Wenkai Yin(尹文开)
53c8ad8228
Merge pull request #13813 from heww/scan-all-apis 
refactor(api): move scan all apis to go-swagger
 2020-12-21 16:40:09 +08:00
fanjiankong
1b93a9f4b0 Remove some TCR UT Case 
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
 2020-12-21 16:24:46 +08:00
AllForNothing
b749ba4e54 Fix filter bug for replication tasks page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-21 15:11:35 +08:00
He Weiwei
ce6ed3eeb7 refactor(api): move scan all apis to go-swagger 
Move scan all APIs from beego to go-swagger.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-21 02:17:02 +00:00
Wang Yan
9bc6f3cee4
fix robot account update issue (#13741) 
* fix robot account update issue

enable the update method to support both v1 & v2 robot update

Signed-off-by: Wang Yan <wangyan@vmware.com>

* resolve review comments

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-18 20:01:26 +08:00
Wang Yan
6bc1047013
migration admin job data (#13766) 
1, migrate gc and scan all schedule to schedule/task/exectuion
2, migrate gc history to task/execution

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-18 16:35:24 +08:00
Will Sun
4392a626f3
Merge pull request #13804 from AllForNothing/scan-all 
Fix robot account UI issues
 2020-12-18 15:48:26 +08:00
Wang Yan
e9797d7c16
fix gc log not found issue (#13790) 
It needs to use the execution ID to get task firstly and then use the required task id to query GC log

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-18 14:50:50 +08:00
AllForNothing
b20cc474b3 Fix robot account UI issues 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-18 14:11:08 +08:00
stonezdj
3334defd92 Revert "fix issue" 
This reverts commit e6f80259
This reverts commit 918fe125

Signed-off-by: stonezdj <stonezdj@gmail.com>

Revert "Add content type and length in header"

This reverts commit ca379111

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-18 12:35:39 +08:00
stonezdj(Daojun Zhang)
05f327cfbd
Merge pull request #13751 from stonezdj/201211_add_content_type_length 
Add content type and length in header
 2020-12-17 18:19:38 +08:00
Wang Yan
dfe678457f
fixes robot account list permission issue (#13792) 
fixes #13786, do not add the removed project into the permission list of a system level robot account.

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-17 17:50:14 +08:00
He Weiwei
792dcc4ac3
fix(scan): returns 400 when artifact not support by scanner (#13785) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-17 16:58:49 +08:00
stonezdj
e6f80259f7 fix issue 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-17 16:33:17 +08:00
stonezdj
918fe125b7 fix issue 
Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-17 16:17:56 +08:00
stonezdj
ca37911113 Add content type and length in header 
Fixes #13740
Update ManifestExist to return Descriptor instead of digest

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-17 15:42:49 +08:00
He Weiwei
18b850782e fix: fix errors detected by codeql 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-17 06:15:34 +00:00
Daniel Jiang
c1c55d0cee Add extra parms when forming redirect URI for OIDC 
Fixes #13092

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-16 19:41:13 +08:00
Daniel Jiang
3b04d2f8f5
Escape the values to contains operator in dao packages (#13774) 
fixes #13018

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-16 14:19:20 +08:00
Will Sun
488d802a2b
Merge pull request #13768 from AllForNothing/permission-list 
Add new permissions to robot account
 2020-12-16 13:30:24 +08:00
Wang Yan
01eb60c36b
add permission checking for gc operation (#13756) 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-15 23:27:12 +08:00
AllForNothing
055572df3d Add new permissions to robot account 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-15 18:34:37 +08:00
stonezdj(Daojun Zhang)
1eb0287ecb
Merge pull request #13709 from stonezdj/201209_dockerhub_limit2 
Cache manifest list for proxy cache
 2020-12-15 14:03:39 +08:00
Will Sun
6849cd03a9
Merge pull request #13719 from sluetze/13568-typofix 
fixed small typos in variable names
 2020-12-15 13:47:30 +08:00
stonezdj
670a94835b Cache manifest list for proxy cache 
Fixes #13566: Quota of dockerhub is still used in v2.1.1 after the image is cached
Cache manifest list in redis cache.
Trade off between efficiency and data integrity, it might cause the proxy cache return the full content of a manifest list instead of the actual manifest list saved in the Harbor storage, which is a part of the manifest list. but this change doesn't break any /v2/ API, just caches full manifest list.

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-12-15 11:30:30 +08:00
He Weiwei
10e4350360
test(cache): increase sleep time to fix ut (#13761) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-14 19:22:42 +08:00
Wenkai Yin(尹文开)
fd900889c1
Merge pull request #13729 from ywk253100/201210_fix 
Tiny fixes for task manager
 2020-12-14 18:41:46 +08:00
Wenkai Yin(尹文开)
c6bfa04661
Merge pull request #13759 from bitsf/fix_tencenttcr_getAdapterInfo 
fix_tencenttcr_getAdapterInfo
 2020-12-14 17:30:22 +08:00
Wenkai Yin
69808f033e Tiny fixes for task manager 
1. Add update time for execution
2. Add unique constraint for schedule to avoid dup records when updating policies
3. Format replication log
4. Keep the webhook handler for legacy replication jobs to avoid jobservice resending the status change request

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-12-14 17:26:32 +08:00
Will Sun
fff1bbe8d9
Merge pull request #13757 from AllForNothing/new-auth-mode 
Add admin groups parameter to http auth
 2020-12-14 17:02:24 +08:00
AllForNothing
11b706adeb Add admin groups parameter to http auth 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-14 16:02:34 +08:00
Ziming Zhang
d8b5746df1 fix_tencenttcr_getAdapterInfo 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-12-14 16:02:19 +08:00
He Weiwei
af24a073dc
feat(api): support project name in the path of apis (#13744) 
Support project name in the path of projects and robotsV1 APIs.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-14 15:48:52 +08:00
Will Sun
949379f7bc
Merge pull request #13755 from AllForNothing/new-registry 
Improve regitries UI to cover new registries
 2020-12-14 14:45:23 +08:00
AllForNothing
8de8122119 Improve regitries UI to cover new registries 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-14 14:07:36 +08:00
He Weiwei
08580f9fec
refactor(scan): refactor scan/scan all job to task manager (#13684) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-14 13:34:35 +08:00
Will Sun
90f9bea965
Merge pull request #13752 from AllForNothing/improve-robot 
Improve refresh secret page for robot account UI
 2020-12-14 11:13:10 +08:00
Wenkai Yin(尹文开)
6569016d35
Merge pull request #13139 from wy65701436/migrate-gc 
Migrate gc to task manager
 2020-12-14 10:43:44 +08:00
Wang Yan
39e1a4f2b4 add extra attributes in the schedule table 
Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-14 02:28:52 +08:00
Wenkai Yin(尹文开)
43104ab0b9
Merge pull request #13724 from reasonerjt/http-auth-admin-grp 
Support admin group in http authproxy
 2020-12-11 13:06:26 +08:00
Daniel Jiang
4ff1c99ea0
Merge pull request #13539 from ninjadq/add_exporter_for_harbor 
Add exporter for harbor
 2020-12-11 12:52:03 +08:00
Daniel Jiang
fc94da54c1
Merge pull request #13728 from heww/fix-ut-for-get-reserved-resources 
test(quota): increase sleep time to fix fail of test
 2020-12-11 12:49:01 +08:00
Wenkai Yin(尹文开)
a4f21b1d85
Merge pull request #13694 from kofj/ghcr-provider 
GHCR Provider
 2020-12-11 09:41:10 +08:00
He Weiwei
2e09e54b38 test(quota): increase sleep time to fix fail of test 
Increase the sleep time to fix the fail in TestGetReservedResources of
quota

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-10 11:23:47 +00:00
wang yan
1bb79d402d update code per review comments 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-12-10 16:08:52 +08:00
Daniel Jiang
60e3668d43 Support admin group in http authproxy 
This commit adds admin_groups into the configuration of http_auth
settings, it's a string in the form of "group1, group2".  If the token
review result shows the user is in one of the groups in the setting he
will have the administrator role in Harbor.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-10 15:57:15 +08:00
Wang Yan
dba5522d0b Migrate to task manager (#129) 
1, remove the gc to new programming model
2, move api define to harbor v2 swagger
3, leverage task & execution manager to manage gc job schedule, trigger and log.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-12-10 14:00:33 +08:00
sluetze
5584e55070 fixed small typos in variable names 
Signed-off-by: sluetze <13255307+sluetze@users.noreply.github.com>
 2020-12-09 15:24:22 +01:00
DQ
85bf88c74e Add test case exporter 
- cache ut
- cli ut
- project collector ut
- system collector ut

Signed-off-by: DQ <dengq@vmware.com>
 2020-12-09 21:47:29 +08:00
DQ
d95f22448c Add cache for exporter 
Add timed cache for exporter
default cache time is 30s, cleanup job run every 4 hours

Signed-off-by: DQ <dengq@vmware.com>
 2020-12-09 21:22:40 +08:00
DQ
ade69e20ef Fix typo 
Signed-off-by: DQ <dengq@vmware.com>
 2020-12-09 20:42:21 +08:00
DQ
e6b53593ee Add exporter to harbor 
- add main file
- add pkg file
- update go.mod
- update vendor

Signed-off-by: DQ <dengq@vmware.com>
 2020-12-09 20:42:14 +08:00
Steven Zou
c12664122c fix(api):update stop job api 
- update StopJob() of basic worker
- update UnSchedule() of basic scheduler
- update the policy store to get more data

fix #13599 , fix #13597

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-12-09 20:42:01 +08:00
Steven Zou
4acb708938
Merge pull request #13705 from steven-zou/fix/js_api 
fix(api):update stop job api
 2020-12-09 13:41:58 +08:00
Will Sun
13900df810
Merge pull request #13710 from AllForNothing/ut-case 
Fix UI UT failure
 2020-12-09 12:13:34 +08:00
AllForNothing
2f2e85577d Fix UI UT failure 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-09 11:33:55 +08:00
fanjiankong
2898117411 GHCR Provider 
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
 2020-12-09 09:54:04 +08:00
Steven Zou
85783f3c0c fix(api):update stop job api 
- update StopJob() of basic worker
- update UnSchedule() of basic scheduler
- update the policy store to get more data

fix #13599 , fix #13597

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-12-08 23:15:05 +08:00
He Weiwei
e92674a42a
feat: add cache library and enable it in config manager (#13525) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-12-08 17:40:03 +08:00
疯魔慕薇
9f8a743da9 Tecent TCR Provider 
1. Docker image registry.
2. Helm chart registry.

Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
 2020-12-08 15:44:18 +08:00
AllForNothing
7cddb5845f Improve refresh secret page for robot account UI 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-08 13:42:32 +08:00
stonezdj(Daojun Zhang)
ec2f251d63
Merge pull request #13685 from wy65701436/robot-name-conflict 
fix robot name conflicate issue
 2020-12-07 11:18:49 +08:00
Wenkai Yin(尹文开)
65b6ae08bb
Merge pull request #13618 from mmpei/official-feature-artifact-hub 
Support replicate from artifact hub
 2020-12-07 10:47:05 +08:00
Daniel Jiang
fef5317aef
Merge pull request #13382 from flaviodsr/fix_core_init 
Fix deadlock on harbor-core initialization
 2020-12-04 19:51:56 +08:00
Will Sun
0840a10752
Merge pull request #13615 from AllForNothing/too-much-tag 
Add limitation to artifact tags
 2020-12-04 15:47:37 +08:00
AllForNothing
6e52fbdf71 Add system robot accout UI 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-12-04 14:19:36 +08:00
Wang Yan
e3a353d8ae fix robot name conflicate issue 
add project name into project level robot account name

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-04 12:40:59 +08:00
Wang Yan
d2fa2e6b84
update robot secret (#13654) 
* update robot secret

1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object

Signed-off-by: Wang Yan <wangyan@vmware.com>

* update robot secret

1, use SHA256 to generate and validate robot secret instread of symmetric encryption.
2, update the patch input object

Signed-off-by: Wang Yan <wangyan@vmware.com>
 2020-12-03 18:13:06 +08:00
Daniel Jiang
73c3f7c7ea Fix the pattern to match v2 catalog URI 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-12-03 12:53:04 +08:00
Wenkai Yin(尹文开)
ddb29f2243
Set timezone as UTC for database connection (#13661) 
Set timezone as UTC for database connection

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-12-03 08:55:48 +08:00