Commit Graph

49 Commits

Author SHA1 Message Date
Shengwen YU
315d8a8993
fix: update code in compliance with golangci-lint revive (#17087)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-07-01 14:52:47 +08:00
Shengwen YU
44bfb992f4
fix: update code for golangci-lint varcheck (#17045)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-21 16:56:36 +08:00
MinerYang
aab320591f
golangci-lint fix ineffassign (#17027)
Signed-off-by: yminer <yminer@vmware.com>

delete src/lib/redis/helper.go

Signed-off-by: yminer <yminer@vmware.com>
2022-06-21 12:01:59 +08:00
Shengwen YU
e9fca3de45
fix: refactor code for golangci-lint whitespace (#17005)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-16 17:42:46 +08:00
Shengwen YU
c2a9f5de64
fix: update code for golangci-lint gosimple (#16974)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-14 13:44:09 +08:00
Chenyu Zhang
6cfbf76781
fix(replication): azurecr replication with token (#16888) (#16947)
Fix azurecr use ACR token failed to list tags, the root cause is the
scope action of acr token is 'metadata_read' not 'pull' when list v2
tags API.

Signed-off-by: chlins <chenyuzh@vmware.com>
2022-06-13 10:05:05 +08:00
Shengwen YU
b43ba15f40
fix: golangci-lint errcheck (#16920)
Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2022-06-07 17:00:36 +08:00
Wang Yan
3eb668c0a4
fix replcation issue on accessory (#16912)
The tag/lable filter only works on the subject manifest, and if the subject manifest is mathed, all the accessories are marked as matched.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-05-27 01:36:38 +08:00
MinerYang
9d8e9158de
fix deadcode lint & update golangci-lint.yaml (#16896)
* fix deadcode lint & update golangci-lint.yaml

Signed-off-by: yminer <yminer@vmmware.com>

mock.go

Signed-off-by: yminer <yminer@vmmware.com>

commentfmt

Signed-off-by: yminer <yminer@vmmware.com>

mock.go update

Signed-off-by: yminer <yminer@vmmware.com>

update makefile

Signed-off-by: yminer <yminer@vmmware.com>

* update /pkg/allowlist/validator.go

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
2022-05-26 10:32:07 +08:00
Wang Yan
d26b5792c0
fix 16883 (#16911)
Fixes #16883, set the labels belonging to the subject manifest to the accessories on replication.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-05-26 01:30:53 +08:00
Wang Yan
d3d4ad6a34
fix staticcheck issues (#16828)
Fix the staticcheck problems that reported by golangci-line staticcheck

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-05-10 16:07:10 +08:00
Wang Yan
c333218ba2
fix close response missing (#16820)
It has to close the http response on deleting manifest

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-05-07 13:03:15 +08:00
MinerYang
083b44e719
bump up astaxie/beego@v1.12.1 to beego/beego/@v1.12.7 (#16770)
Signed-off-by: yminer <yminer@vmmware.com>

goo mod tidy

Signed-off-by: yminer <yminer@vmmware.com>

Co-authored-by: yminer <yminer@vmmware.com>
2022-04-28 11:58:44 +08:00
疯魔慕薇
1c4495361b
Fix & UT (#16279)
1. Fix TCR Adapter namespcae check.
2. Add Chart UT.

Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
2022-03-04 14:09:25 +08:00
He Weiwei
abd85284d2
feat: merge local and remote tags for repo of proxy cache project in list API (#16394)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2022-02-24 10:27:45 +08:00
Wang Yan
6ef4874011
fix replicate reference accessory (#16401)
fixes #16375

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-02-23 21:24:15 +08:00
chlins
b37a987073 fix: fix codeql alerts
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-02-15 17:25:02 +08:00
Chenyu Zhang
4ef2d65451
Merge pull request #16286 from chlins/fix/skip-replication-for-proxy-cache
fix: skip replication to proxy cache project
2022-02-07 17:06:03 +08:00
Wang Yan
0a183feab6
add cosign support in replication (#16282)
For the case Harbor-to-Harbor, the accessory can be replicated from source or to target.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2022-01-26 21:35:17 +08:00
chlins
b1afd2efb0 fix: skip replication to proxy cache project
Signed-off-by: chlins <chenyuzh@vmware.com>
2022-01-26 17:24:39 +08:00
rao yunkun
ce6bf73884 Merge remote-tracking branch 'upstream/main' 2021-12-05 16:34:04 +08:00
chlins
f880bc3361 fix(replication): enhance the replication rule validation
Signed-off-by: chlins <chenyuzh@vmware.com>
2021-11-12 09:53:11 +08:00
Chenyu Zhang
3aa698c7c9
fix(replication): list projects before replicate to reduce create duplicate project and requests to target registry (#15934)
Signed-off-by: chlins <chenyuzh@vmware.com>
2021-11-08 10:39:58 +08:00
chlins
9e8218f63b fix: handle codeql golang security issues
Signed-off-by: chlins <chenyuzh@vmware.com>
2021-10-22 10:51:26 +08:00
Wang Yan
f9d1294b45
roll back the runner count (#15792)
It takes about 1 hour to perform data for 40000 repositories per performance testing.

Roll back the runner count to 100 could speed the data preparation time.
It's safe since it only takes 100 DB connection counts at most per execution per core.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-10-14 19:05:30 +08:00
Wang Yan
729d2e6590 fix replication DB connection issue
fixes #15736

For the current imple, the GetWorker() may hang when there is no worker available, and will not release the DB connection.
In this case, the DB connection could reach the up limit that leads to harbor core for service unavailable.

1, move GetWorker() in the goroutine, release the DB connection for API.
2, reduce the worker count per harbor-core from 1024 to 10.
3, reduce the runner count per worker to 30.

After above, the max connection per harbor-core should be 300.

Worker: To control how many replicaiton exectuions can have at most at the same time.
Runner: To control the speed to generate an jobservice replicaiton job.

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-10-14 11:31:33 +08:00
Qian Deng
879eecc926 Refactor and Add trace to http Transport
* Refactor common http GetTransport function signature
* Remove redendent GetHTTPTransport and similar functions
* Update Authorized function signature to meet new HTTPTransport
* Add trace for default Transport

Signed-off-by: Qian Deng <dengq@vmware.com>
2021-09-18 10:58:52 +00:00
Ziming Zhang
d62aaaa625 fix(replication) gitlab filters may not work
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-09-13 17:23:12 +08:00
yunkunrao
246d863c59 Refactor ping method into util pkg
Signed-off-by: yunkunrao <yunkunrao@gmail.com>
2021-08-19 21:57:25 +08:00
Ziming Zhang
62808bf014 feat(replication) refactor replication filter Validate()
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-08-17 14:19:14 +08:00
Ziming Zhang
698c336421 feat(replication) add exclude decoration in policy
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-08-17 14:19:13 +08:00
Shengwen Yu
766e953325 fix: update adapter ut (mock http requests)
Closes: #15318

Signed-off-by: Shengwen Yu <yshengwen@vmware.com>
2021-07-22 19:45:36 +08:00
Wenkai Yin(尹文开)
f0e1ba4e1b
Merge pull request #15005 from ywk253100/210528_ns
Declare the capability of supported repository path component for registries
2021-05-31 15:53:40 +08:00
He Weiwei
c7a4af3170
test: mock the API requests for the huawei registry (#15009)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-28 17:09:48 +08:00
Wenkai Yin
741793a553 Declare the capability of supported repository path component for registries
Declare the capability of supported repository path component for registries

fixes #14981
fixes #14980

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-28 10:42:16 +08:00
Ziming Zhang
77b44a62e7 fix aws adapter url
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-05-27 23:29:12 +08:00
fanjiankong
306e519dbe Fix tencentcr adapter.
1. Limit API qps for the adapter.
2. Allow set qps via env.
3. Fix Tencnet SDK pagenation.
4. Fix resource filter.

Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2021-05-24 10:33:49 +08:00
Wenkai Yin(尹文开)
0fe551274d
Merge pull request #14921 from ywk253100/210519_db
Clean up tech debt codes
2021-05-20 13:44:10 +08:00
Wenkai Yin
998e392bb4 Clean up tech debt codes
Clean up tech debt codes

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-20 11:47:46 +08:00
Wenkai Yin
315f490d3c Update the field name of registry model
Update the field name of registry mode

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-18 17:26:20 +08:00
stonezdj(Daojun Zhang)
0dfc801a50
Error string should not be capitalized (#14840)
Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-05-12 16:53:37 +08:00
Steven Zou
cae82dc3d0
Merge pull request #14676 from bitsf/update_adapter_region
update aliyun and aws region
2021-04-19 11:17:07 +08:00
Ziming Zhang
e8450d99ce update aliyun and aws region
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-04-16 22:34:11 +08:00
Ziming Zhang
74bd82b3a1 fix(replication) aws adapter checkRepository
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-04-16 22:28:14 +08:00
stonezdj
60478f4990 Move common config api to lib/config
Register all config managers, and get it by getConfigManger()

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-04-13 19:43:33 +08:00
Wenkai Yin
09c3d042ea Improve the performance of replication
Improve the performance of replication by introducing a new API to check whether the blob can be mounted directly

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-04-12 09:54:25 +08:00
stonezdj
107e468b60 Refactor configure api to new programming model
Changes include:
1. Move core/config to controller/config
2. Change the job_service and gcreadonly to depends on lib/config instead of core/config
3. Move the config related dao, manager and driver to pkg/config
4. Adjust the invocation of the config API, most of then should provide a context parameter, when accessing system config, you can call it with background context, when accessing user config, the context should provide orm.Context

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-04-09 08:10:11 +08:00
Wenkai Yin(尹文开)
28596c3ffb
Refactor registry API (#14528)
* Refactor registry API

Refactor registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>

* Fix bugs of replications

1. Fix the scheduled replication doesn't work issue
2. Fix the destination name lost issue when updating replication policy

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-31 15:49:23 +08:00
Wenkai Yin
3d7fd070c7 Refeactor replication policy APIs
Refeactor replication policy APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-03-17 10:57:31 +08:00