Commit Graph

344 Commits

Author SHA1 Message Date
Daniel Jiang
c93858cdbc
Remove UT for helm hub adapter (#13255)
The helm hub adapter will not work due to the shift to artifact hub and
the API has changed. more details see #13244
Remove the UT for helm hub adapter to unblock CI.
Later we need to rework the adapter to replicate from artifact hub.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-10-14 10:22:03 +08:00
Ziming Zhang
699518a22e feature(replication) enable role based auth for AWS ECR
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-09-15 12:46:28 +08:00
Wenkai Yin
7816ff5e85 retry several times when failed to copy blob during the replication
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-09-02 16:07:20 +08:00
Ziming Zhang
bda66e9e1f Add aws China regions ECR support, both of cn-north-1 and cn-northwest-1 endpoints.
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-08-27 23:43:16 +08:00
Ziming Zhang
c3fde4e483 fix(replication) gcr deletion and tag deletion
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-08-26 15:18:05 +08:00
wang yan
648b80bc34 udpate resource list order
1, order label by creation time.
2, order webhook policy by creation time.
3, order replication policy by creation time.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-08-19 11:20:31 +08:00
chlins
b3012ec08d fix(replication): fix quay adapter authorization
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-08-12 22:12:08 +08:00
chlins
b765cfe0ce fix(replication): refactor quay adapter to fix authorization and support quay.io and enterprise quay (#10317)
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-08-08 13:17:01 +08:00
Wenkai Yin
219b9910eb Show the detail error message when failed to fetch the artifacts during replication
Show the detail error message when failed to fetch the artifacts during replication

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-08-07 18:31:02 +08:00
Wenkai Yin(尹文开)
bfb8f4aca0
Merge pull request #12622 from bitsf/fix_aliacr_pull
fix[replicatioin] aliyun acr pull with repo filter
2020-08-03 18:27:23 +08:00
Steven Zou
a6e85c1f8a
Merge pull request #12608 from bitsf/fix_gitlab_pull
fix[replication] gitlab pull-based multi namespace
2020-08-03 18:24:45 +08:00
Ziming Zhang
d16753ae96 fix[replicatioin] aliyun acr pull with repo filter
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-07-30 00:07:54 +08:00
stonezdj(Daojun Zhang)
5b1b94f25c
Merge pull request #12512 from ywk253100/200717_summary
Include the registry info in the project summary API for proxy cache project
2020-07-29 14:39:30 +08:00
Ziming Zhang
e7f8869ff7 fix[replication] gitlab pull-based multi namespace
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-07-28 15:51:49 +08:00
chlins
f70b674b3f fix(replication): fix jfrog replication when filter includes multi images
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-22 12:30:26 +08:00
Wenkai Yin
d7327d8d8e Include the registry info in the project summary API for proxy cache project
Include the registry info in the project summary API for proxy cache project

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-07-17 16:14:45 +08:00
Ziming Zhang
e7c89ce1d9 fix SWR replication adapter pull-based
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-07-16 19:19:18 +08:00
Wenkai Yin
02690d1d04 Suport filtering registries by type in listing registry API
Suport filtering registries by type in listing registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-07-07 10:30:46 +08:00
Wenkai Yin
ea20690264 Implement task and execution manager
Implement task and execution manager

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-06-23 17:10:58 +08:00
Wenkai Yin(尹文开)
b7b1ce457b
Merge pull request #10389 from lxShaDoWxl/fix/gitlab
Fixed search images in registry gitlab
2020-06-04 14:49:40 +08:00
Chlins Zhang
f862805244
Merge pull request #11998 from chlins/fix/replication-pull-blob-content-length
fix(replication): ignore the problem of not getting Content-Length fr…
2020-05-29 10:06:44 +08:00
mmpei
7c2bfb1378
Fix helmhub insecure issue when using with a proxy (#12014)
Signed-off-by: peimingming <peimingming@corp.netease.com>
2020-05-28 10:57:51 +08:00
chlins
54895def90 fix(replication): ignore the problem of not getting Content-Length from the pull blob api
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-05-27 13:00:54 +08:00
lxShaDoWxl
681e97482e fix(Registries/Gitlab): upgrade testing
Signed-off-by: lxShaDoWxl <lxshadowxkingxl@gmail.com>
2020-05-26 20:22:26 +06:00
lxShaDoWxl
47a57b80d7 Merge remote-tracking branch 'origin/master' into fix/gitlab 2020-05-06 13:26:18 +06:00
Ziming Zhang
97a7a6dc35 fix(retention) fix retention repository with slash
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-04-26 15:07:25 +08:00
Wang Yan
8a0e8627ff
replace pkg errors with lib errors (#11605)
Fixes #9704

As we do want to unify error handling, so just decreprates pkg errors, use lib/errors instead for Harbor internal used errors model.

1, The lib/errors can cover all of funcs of pkg/errors, and also it has code attribute to define the http return value.
2, lib/errors can give a OCI standard error format, like {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}

If you'd like to use pkg/errors, use lib/errors instead. If it cannot meet your request, enhance it.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-15 22:41:45 +08:00
Wenkai Yin
df1f52dba4 Use insecure transport when creating the adapter for local instance in replication
Use insecure transport when creating the adapter for local instance in replication

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-04-10 16:45:57 +08:00
Wenkai Yin
8ee3421176 Fix replication failure in kubernetes with hairpin mode disabled
Fixes #11480
Fixes #11461
Fix replication failure in kubernetes with hairpin mode disabled

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-04-09 15:10:25 +08:00
Wenkai Yin
2171634000 Fix replication bug
Remove the URL replacing logic temporarily to make replication work and will introduce a new solution for the hairpin issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-04-07 21:27:06 +08:00
Wenkai Yin
8f11cb7ff0 Support replication between Harbor 2.0 and 1.x
Fixes #11374, fixes #11302, support replication between Harbor 2.0 and 1.x by providing versioning adapter

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-04-04 22:08:11 +08:00
wang yan
a11a70d941 move logger from common to lib
The logger is the fundamental library, so move it into lib folder
Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-02 14:09:03 +08:00
Qian Deng
b1284da96b
Merge pull request #11360 from ninjadq/rever_chart_api_change
Rever chart api change
2020-04-01 18:58:57 +08:00
DQ
9ff7d46e8f Rever chart version to original
Because chart version should consistent with previous version

Signed-off-by: DQ <dengq@vmware.com>
2020-04-01 11:55:22 +08:00
Wenkai Yin
d9a5c71289 Some tiny improvement for replication
1. Add timeout when transter artifacts
2. Check 404 error when unschedule the policy
3. Add line to mark the job failure in job log

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-31 14:50:37 +08:00
Wenkai Yin(尹文开)
6815e8dc4d
Merge pull request #11348 from ywk253100/200330_tag_filter
Make sure the tag filter have the same behavior for empty value and *
2020-03-30 14:02:37 +08:00
Wenkai Yin
7ec5595bd8 Make sure the tag filter have the same behavior for empty value and *
Fixes #11233, make sure the tag filter have the same behavior for empty value and *

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-30 10:10:12 +08:00
He Weiwei
9c06c79ff4 refactor(errors): rename pkglib/error to lib/errors
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-29 05:19:05 +00:00
Steven Zou
1bbd3585e5
Merge pull request #11296 from ywk253100/200326_replication
Support replicate images with media type application/vnd.docker.distribution.manifest.v1+json
2020-03-27 18:12:12 +08:00
Steven Zou
36552ba18b
Merge pull request #11318 from ywk253100/200326_remove_pagination_default
Iterate the link header when listing artifact
2020-03-27 18:07:03 +08:00
Wenkai Yin
9a92b9e725 Fix bugs of replication
1. Bump up the version of API used in replicatoin scheduler job
2. Check the error message to determine whether the job exists or not in jobservice when unschedule a job

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-26 23:12:02 +08:00
Wenkai Yin
afdfedcb49 Iterate the link header when listing artifact
Fixes #11315
When specify no pagination in listing artifact request, the go-swagger will set the default value for them, so we need to iterate the link header to get all of artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-26 21:26:09 +08:00
Wenkai Yin
64e4651b3f Support replicate images with media type application/vnd.docker.distribution.manifest.v1+json
Fixes #11272, support replicate images with media type application/vnd.docker.distribution.manifest.v1+json

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-26 19:09:48 +08:00
Wenkai Yin(尹文开)
8984979bd2
Relocate/rename some packages (#11183)
Fixes #11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-24 20:45:45 +08:00
DQ
4c30995858 Refator tls config
use default Httptransport instead of empty one
remove unused code

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:10 +08:00
DQ
115185894f Merge internal Transport and Secure Transport
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
02dea3ad2c Add: mTLS configuration on CI
Add internal_tls on ci
generate certs for ci

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
dcc6950af7 Feat: auto install ca in registry
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
b852605193 Feat: enable mtls in harbor replication
Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00
DQ
e6bb3b0977 Feat: enable tls related thing to jobservice
Add tls related code in jobservice

Signed-off-by: DQ <dengq@vmware.com>
2020-03-18 19:22:09 +08:00