Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2025-01-04 23:17:45 +01:00
Code Issues Projects Releases Wiki Activity
10,388 Commits 56 Branches 321 Tags 341 MiB
634be34236
Commit Graph

376 Commits

Author SHA1 Message Date
Wenkai Yin
ea20690264 Implement task and execution manager 
Implement task and execution manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-06-23 17:10:58 +08:00
Wenkai Yin(尹文开)
b7b1ce457b
Merge pull request #10389 from lxShaDoWxl/fix/gitlab 
Fixed search images in registry gitlab
 2020-06-04 14:49:40 +08:00
Chlins Zhang
f862805244
Merge pull request #11998 from chlins/fix/replication-pull-blob-content-length 
fix(replication): ignore the problem of not getting Content-Length fr…
 2020-05-29 10:06:44 +08:00
mmpei
7c2bfb1378
Fix helmhub insecure issue when using with a proxy (#12014) 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2020-05-28 10:57:51 +08:00
chlins
54895def90 fix(replication): ignore the problem of not getting Content-Length from the pull blob api 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2020-05-27 13:00:54 +08:00
lxShaDoWxl
681e97482e fix(Registries/Gitlab): upgrade testing 
Signed-off-by: lxShaDoWxl <lxshadowxkingxl@gmail.com>
 2020-05-26 20:22:26 +06:00
lxShaDoWxl
47a57b80d7 Merge remote-tracking branch 'origin/master' into fix/gitlab 2020-05-06 13:26:18 +06:00
Ziming Zhang
97a7a6dc35 fix(retention) fix retention repository with slash 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-04-26 15:07:25 +08:00
Wang Yan
8a0e8627ff
replace pkg errors with lib errors (#11605) 
Fixes #9704

As we do want to unify error handling, so just decreprates pkg errors, use lib/errors instead for Harbor internal used errors model.

1, The lib/errors can cover all of funcs of pkg/errors, and also it has code attribute to define the http return value.
2, lib/errors can give a OCI standard error format, like {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}

If you'd like to use pkg/errors, use lib/errors instead. If it cannot meet your request, enhance it.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-15 22:41:45 +08:00
Wenkai Yin
df1f52dba4 Use insecure transport when creating the adapter for local instance in replication 
Use insecure transport when creating the adapter for local instance in replication

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-10 16:45:57 +08:00
Wenkai Yin
8ee3421176 Fix replication failure in kubernetes with hairpin mode disabled 
Fixes #11480
Fixes #11461
Fix replication failure in kubernetes with hairpin mode disabled

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-09 15:10:25 +08:00
Wenkai Yin
2171634000 Fix replication bug 
Remove the URL replacing logic temporarily to make replication work and will introduce a new solution for the hairpin issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-07 21:27:06 +08:00
Wenkai Yin
8f11cb7ff0 Support replication between Harbor 2.0 and 1.x 
Fixes #11374, fixes #11302, support replication between Harbor 2.0 and 1.x by providing versioning adapter

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-04 22:08:11 +08:00
wang yan
a11a70d941 move logger from common to lib 
The logger is the fundamental library, so move it into lib folder
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-02 14:09:03 +08:00
Qian Deng
b1284da96b
Merge pull request #11360 from ninjadq/rever_chart_api_change 
Rever chart api change
 2020-04-01 18:58:57 +08:00
DQ
9ff7d46e8f Rever chart version to original 
Because chart version should consistent with previous version

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 11:55:22 +08:00
Wenkai Yin
d9a5c71289 Some tiny improvement for replication 
1. Add timeout when transter artifacts
2. Check 404 error when unschedule the policy
3. Add line to mark the job failure in job log

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-31 14:50:37 +08:00
Wenkai Yin(尹文开)
6815e8dc4d
Merge pull request #11348 from ywk253100/200330_tag_filter 
Make sure the tag filter have the same behavior for empty value and *
 2020-03-30 14:02:37 +08:00
Wenkai Yin
7ec5595bd8 Make sure the tag filter have the same behavior for empty value and * 
Fixes #11233, make sure the tag filter have the same behavior for empty value and *

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-30 10:10:12 +08:00
He Weiwei
9c06c79ff4 refactor(errors): rename pkglib/error to lib/errors 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-29 05:19:05 +00:00
Steven Zou
1bbd3585e5
Merge pull request #11296 from ywk253100/200326_replication 
Support replicate images with media type application/vnd.docker.distribution.manifest.v1+json
 2020-03-27 18:12:12 +08:00
Steven Zou
36552ba18b
Merge pull request #11318 from ywk253100/200326_remove_pagination_default 
Iterate the link header when listing artifact
 2020-03-27 18:07:03 +08:00
Wenkai Yin
9a92b9e725 Fix bugs of replication 
1. Bump up the version of API used in replicatoin scheduler job
2. Check the error message to determine whether the job exists or not in jobservice when unschedule a job

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 23:12:02 +08:00
Wenkai Yin
afdfedcb49 Iterate the link header when listing artifact 
Fixes #11315
When specify no pagination in listing artifact request, the go-swagger will set the default value for them, so we need to iterate the link header to get all of artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 21:26:09 +08:00
Wenkai Yin
64e4651b3f Support replicate images with media type application/vnd.docker.distribution.manifest.v1+json 
Fixes #11272, support replicate images with media type application/vnd.docker.distribution.manifest.v1+json

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 19:09:48 +08:00
Wenkai Yin(尹文开)
8984979bd2
Relocate/rename some packages (#11183) 
Fixes #11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-24 20:45:45 +08:00
DQ
4c30995858 Refator tls config 
use default Httptransport instead of empty one
remove unused code

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
115185894f Merge internal Transport and Secure Transport 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
02dea3ad2c Add: mTLS configuration on CI 
Add internal_tls on ci
generate certs for ci

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
dcc6950af7 Feat: auto install ca in registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
b852605193 Feat: enable mtls in harbor replication 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
e6bb3b0977 Feat: enable tls related thing to jobservice 
Add tls related code in jobservice

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
da359f609f Feat: enable mtls in core 
add mtls related code in core

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
Wenkai Yin
5925e0862d Replicate tag deletion between Harbor instances 
This commit introduces the tag deletion as a new capability for registry adapters, and currently only Harbor supports it

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-17 09:27:02 +08:00
Steven Zou
b546d9167a
Merge pull request #11019 from bitsf/replication_adapter_sort 
feat(replication) sort the adapters shown on UI
 2020-03-12 14:53:34 +08:00
Ziming Zhang
d1d0601841 feat(replication) sort the adapters shown on UI 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-11 18:06:29 +08:00
Wenkai Yin(尹文开)
d644d23b25
Merge pull request #10370 from kofj/fix/aliacr 
FIX: AliACR Provider.
 2020-03-11 15:36:34 +08:00
Ziming Zhang
7d53a61a92 feat(replication) sort the adapters shown on UI 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-11 15:03:10 +08:00
Wenkai Yin(尹文开)
b02cab434f
Fire event when create/delete resources (#11010) 
1. Create/delete project
2. Create/delete repository
3. Push/pull/delete artifact
4. Create/delete tag

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-11 14:39:01 +08:00
Wenkai Yin
e237a686c4 Make replication work with new artifact(phase 2) 
Provide the resource type filter for users to choose when replicating from harbor to other registries

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-06 18:26:37 +08:00
Wenkai Yin
d4ba023457 Make sure replication work with OCI artifacts(phase 1) 
This commit updates the definition of replicated resource(artifacts replace the vtags) and refactor the filter part

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-03 17:50:51 +08:00
DQ
ff0c8b382c Refactor the version to variable 
Signed-off-by: DQ <dengq@vmware.com>
 2020-02-26 16:24:49 +08:00
Wenkai Yin
528f598268 Reimplement the registry client 
This commit reimplements the registry client under directory src/pkg/registry and removes the useless code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-24 14:36:26 +08:00
Wenkai Yin
a4ebbc6ecf Do enhancement for the registry authorizer 
This commit introduces a new wrapper authorizer which can authorize the request according to the auth scheme automatically

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-11 21:21:49 +08:00
lxShaDoWxl
d09ca5ae6d Merge remote-tracking branch 'origin/master' into fix/gitlab 2020-01-24 21:19:32 +06:00
lxShaDoWxl
cc1938a04c tests(Registries/Gitlab): Added tests for gitlab adapter 
Signed-off-by: lxShaDoWxl <lxshadowxkingxl@gmail.com>
 2020-01-24 21:17:45 +06:00
Wenkai Yin(尹文开)
8809b7aa5d
Merge pull request #9887 from chlins/fix/replication-huawei-adapter-authorization 
fix(replication): huawei adapter multi authorization
 2020-01-17 19:04:52 +08:00
Ziming Zhang
ed3bf041e2 fix(replication): aws ecr delete image 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
Change-Id: I5e38b813c2840e0270973c38680cb8f815e5ece9
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-01-13 10:58:13 +08:00
lxShaDoWxl
13892ba144 Merge remote-tracking branch 'origin/master' into fix/gitlab 2020-01-02 13:43:07 +06:00
lxShaDoWxl
73b69e0fa7 fix(Registries/Gitlab): Added support for regular expression search, so as not to request all repositories from gitlab remove * at the end and beginning and do a search, also fixed the tag name check 
Signed-off-by: lxShaDoWxl <lxshadowxkingxl@gmail.com>
 2020-01-02 13:42:13 +06:00
Wenkai Yin
0c776076bd Remove the workaround for blocking manifest list in replication 
Remove the workaround for blocking manifest list in replication

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-12-31 17:13:58 +08:00
慕薇疯魔
ad01d68396 FIX AliACR Provider. 
1. Pagination.
2. Resource filter.

Signed-off-by: 慕薇疯魔 <kfanjian@gmail.com>
 2019-12-29 17:52:23 +08:00
Martin Juhl
06594a1756 Squashed version of PR-9943 
Signed-off-by: Martin Juhl <m@rtinjuhl.dk>
 2019-12-02 11:59:33 +01:00
chlins
130e406d3c fix(replication): huawei adapter multi authorization 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2019-11-15 11:33:01 +08:00
Wenkai Yin
81e7b3b8ea Add adapter pattern for helm hub adapter 
Add adapter pattern for helm hub adapter

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-11-07 16:40:49 +08:00
hao.cheng
758e7aa06f code simplify 
Signed-off-by: hao.cheng <hao.cheng@daocloud.io>
 2019-10-24 17:42:42 +08:00
Ziming Zhang
1801bac03d refact replication adapter 
Change-Id: Ic28854089b8dcfcbc7e42065df5c19c64d5b85e7
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-10-24 12:24:55 +08:00
Ziming Zhang
5419e1a844 refact replication info to support special endpoints and credential 
Change-Id: I2f7a51d3aaf57bb6d1942526184f4e62ce3afeab
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-10-23 17:19:56 +08:00
Ziming
3725667448
Merge branch 'master' into feat/gitlab 2019-10-16 14:34:21 +08:00
Steven Zou
e0901410e1
Merge pull request #9381 from beautytiger/dev-191011-fixtime 
fix wrong time parse by wrong year representation
 2019-10-14 17:02:06 +08:00
Guangming Wang
bea8609a58 fix wrong time parse by wrong year representation 
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
 2019-10-11 22:59:40 +08:00
chlins
4ab3b864ae feat: add image replication adapter for quay.io 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2019-10-11 10:00:07 +08:00
Wenkai Yin(尹文开)
0a85acac9a
Merge pull request #9329 from chlins/feat/jfrog-docker-registry 
feat(replication): support for jfrog artifactory docker image replica…
 2019-10-10 09:41:41 +08:00
chlins
0e0c42d3c0 feat(replication): support for jfrog artifactory docker image replication 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2019-10-09 21:06:36 +08:00
lxShaDoWxl
a40b0f349b In order to integrate with gitlab it is necessary to get a list of available repositories using gitlab api. This is because gitlab registry does not give access to _catalog 
Signed-off-by: lxShaDoWxl <lxshadowxkingxl@gmail.com>
 2019-09-25 21:32:00 +06:00
Wenkai Yin(尹文开)
f3af35f07a
Merge pull request #9021 from ywk253100/190909_scan_cron 
Check the status behind error when trying to update the scan schedule
 2019-09-10 13:15:27 +08:00
lxShaDoWxl
6937731744 feat(Registries): added gitlab adapter 
Signed-off-by: lxShaDoWxl <lxshadowxkingxl@gmail.com>
 2019-09-09 21:21:21 +06:00
Wenkai Yin
3b07be5a72 Check the status behind error when trying to update the scan schedule 
Check the status behind error when trying to update the scan schedule

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-09-09 13:31:10 +08:00
cd1989
8990a621d6 Not use time pointer in model fields 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-09-06 17:48:08 +08:00
Wang Yan
87893abc5e
Merge pull request #8829 from ywk253100/190822_retry_status 
Add status revision to handle retrying in replication task
 2019-08-28 10:55:13 +08:00
Wenkai Yin
7924f37d86 Add status revision to handle retrying in replication task 
Add status revision to handle retrying in replication task

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-27 14:17:11 +08:00
cd1989
4b59346423 Exist core when signal received 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-23 09:34:41 +08:00
Wenkai Yin
2a57424f9d Fix bug in native replication adapter 
Use the stardard token authentication work flow even though the credentail isn't provided

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-14 18:17:38 +08:00
Steven Zou
1adc3a9469
Merge pull request #8606 from ywk253100/190807_stuck 
Fix replication tasks stuck in "InProgress" issue
 2019-08-13 15:59:20 +08:00
Wenkai Yin(尹文开)
34f845b804
Merge pull request #8594 from ywk253100/190731_replicate_chart 
Fix bug in Helm Hub and Harbor adapter for replicating chart
 2019-08-12 17:48:11 +08:00
Wenkai Yin
96a1ad89fa Only replicate "public" metadata when creating project in Harbor 
This commit filters the metadata when creating project in replication to avoid replicate unnecessary properties, such as retention policy

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-08 16:19:09 +08:00
Wenkai Yin
8777c07d47 Fix replication tasks stuck in "InProgress" issue 
Fix replication tasks stuck in "InProgress" issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-08 15:42:42 +08:00
Wenkai Yin
b7b8332626 replication helm hub adapter bug fix 
Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-08 14:14:00 +08:00
cd1989
870d7115c4 Refactor code to extract a common task runner 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-07 17:14:10 +08:00
cd1989
e2e540233b Use context for concurrency control 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-07 17:14:10 +08:00
cd1989
1f541c890c Improve performance for other registry adapters 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-07 17:14:10 +08:00
cd1989
254aa7e2b7 Fix DockerHub adapter registry URL problem 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-07 17:14:10 +08:00
cd1989
5f29d9a3b7 Improve DockerHub adapter performance 
Signed-off-by: cd1989 <chende@caicloud.io>
 2019-08-07 17:14:10 +08:00
Wenkai Yin
e0fac5b61a Fixes #8281, only show one tag in the task list to decrease the length of string 
Fixes #8281, only show one tag in the task list to decrease the length of string to avoid exceeding the limit in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-08-07 11:47:49 +08:00
Wenkai Yin
6464dcbd51 Register the replication adapter for Helm Hub 
Register the replication adapter for Helm Hub

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-30 16:28:06 +08:00
Wenkai Yin(尹文开)
9e6b022ce1
Merge pull request #8425 from ywk253100/190726_acr 
Fix #8319, got error when replicating image with Azure container registry
 2019-07-30 15:19:12 +08:00
Wenkai Yin
4dac036013 Fix #8319, got error when replicating image with Azure container registry 
Fix #8319, got error when replicating image with Azure container registry

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2019-07-30 12:58:22 +08:00
慕薇疯魔
0d71eb363a Aliyun ACR adapter 
1. aliyun ACR registry adapter.
2. Unit test case.
3. Update go modules and vendor.

Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
 2019-07-29 18:24:03 +08:00
Wenkai Yin(尹文开)
829d9a56b4
Merge pull request #8436 from kofj/improve_awsecr 
Improve awsecr parseRegion method.
 2019-07-29 11:23:09 +08:00
疯魔慕薇
55cead910f Improve awsecr parseRegion method. 
Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
 2019-07-28 11:16:44 +08:00
Steven Zou
b6c4ffee95
Merge pull request #8244 from mmpei/7980-chartReplication 
Support helm chart replication from helmhub
 2019-07-25 10:30:20 +08:00
Wenkai Yin(尹文开)
94f0fb506b
Merge pull request #8134 from chlins/refactor/small-refactor-for-huawei-adapter 
refactor: adjust some logic in huawei adapter
 2019-07-24 12:13:23 +08:00
peimingming
9f777ed43f Fix by comments 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2019-07-19 17:04:40 +08:00
chlins
fc713a1763 refactor: fix unit tests 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2019-07-12 11:33:31 +08:00
chlins
000fe3d084 refactor: adjust some logic in huawei adapter 
Signed-off-by: chlins <chlins.zhang@gmail.com>
 2019-07-12 11:33:30 +08:00
mmpei
f32670058a Fix by comments 
Signed-off-by: mmpei <peimingming@corp.netease.com>
 2019-07-10 17:31:59 +08:00
peimingming
4946c7bba7 Add helm chart replication from helmhub 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2019-07-10 11:01:02 +08:00
Ziming Zhang
0691a5ad9c fix replication dockerhub tls skip tls 
Change-Id: I9584d89cf9bda2b782b0fd556450a81be0fc7510
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2019-07-08 08:17:26 +00:00
Wenkai Yin(尹文开)
5f9420a5a7
Merge pull request #8190 from ywk253100/190701_replication 
Merge Default ImageRegistry into the native adapter to reduce the duplicate code
 2019-07-08 12:47:55 +08:00
Wenkai Yin(尹文开)
0f28fe42fd
Merge pull request #8167 from ywk253100/190622_replication 
Support v1 signed media type when do the replication
 2019-07-05 14:33:22 +08:00