Commit Graph

70 Commits

Author SHA1 Message Date
Wang Yan
588bf475ae
fix update pull time issue (#12601)
fixes #12574

let the tag ID as 0 when tags array is empty when to update pull time

Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-28 12:15:46 +08:00
Steven Zou
716da7f3ff fix(p2p-preheat):fix issues of triggering preheat
- fix invalid data type of vulnerability filter param
- add more debug logs
- add more logs in the preheat job
- fix issue of getting empty list when doing querying artifacts

Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-26 13:47:58 +08:00
He Weiwei
e3b1ec775f
refactor(quota): align Get and List methods of quota controller (#12434)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-22 11:18:05 +08:00
Steven Zou
cae087ba49
Merge pull request #12511 from steven-zou/fix/p2p_provider_health_check
fix(preheat):remove health property check when saving instance
2020-07-20 21:52:06 +08:00
Steven Zou
ee35e1ecc6
Merge pull request #12507 from chlins/fix/preheat-update-instance
fix(preheat): fix preheat handler PingInstance and UpdateInstance
2020-07-20 17:45:24 +08:00
Steven Zou
144edcc0c4 fix(preheat):fix ctl ut failure case
Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-20 17:14:17 +08:00
Steven Zou
6b9e0e66c4 Merge branch 'master' into fix/p2p_provider_health_check 2020-07-20 16:02:40 +08:00
chlins
78927af032 fix(preheat): fix preheat handler PingInstance and UpdateInstance
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-20 15:42:24 +08:00
Steven Zou
3c02b5da03 fix(preheat):remove health property check when saving instance
Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-20 13:53:06 +08:00
Wang Yan
24ed52112e
fix blob deleting status issue (#12481)
1, The update blob status method should udpate the blob version of the blob object as well, otherwise the GC job cannot handle the blob status transform(none - delete - deleting - deletefailed)
as the method is using version equals as the query condition.
2, For the deleting blob which marked for more than 2 hours, it should be set to delete failed in head blob & put manifest request

Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-20 11:44:29 +08:00
Ziming Zhang
8857e89e40 feature(redis) support redis sentinel
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-07-19 21:19:03 +08:00
Steven Zou
4d4a04fad4
Merge pull request #12478 from steven-zou/feat/read_pro_config
feat(p2p):enhance policy enforcer
2020-07-16 11:40:29 +08:00
Ted Guan
9e7edb7a6e
Fix for project metadata (#12410)
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2020-07-15 18:46:45 +08:00
Steven Zou
fcfde5a588 feat(p2p):enhance policy enforcer
Read security settings from the project configurations and override the preheat policy settings if necessary.

Check the project security settings and override the related settings in the policy if necessary.
NOTES: if the security settings (relevant with signature and vulnerability) are set at the project configuration,
they will have the highest priority and override the related settings of the preheat policy.
 e.g (use signature as an example, similar case to vulnerability severity part):
   if policy.signature = false and project.config.signature = true; then policy.signature = true
   if policy.signature = true and project.config.signature = true; then policy.signature = true
   if policy.signature = true and project.config.signature = false; then policy.signature = true
   if policy.signature = false and project.config.signature = false; then policy.signature = false

Signed-off-by: Steven Zou <szou@vmware.com>

Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-15 14:18:34 +08:00
He Weiwei
a22d803a95 refactor(quota): move pkg/types to pkg/quota/types
Closes #9664

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-14 14:28:53 +00:00
stonezdj
3abe77d6cb Add proxy cache feature
Update route to add proxy related middleware
Add proxy controller

Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-07-13 21:18:43 +08:00
chlins
08bd46e125 feat: add preheat execution api handler
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-13 13:14:08 +08:00
chlins
2863e68718 feat: add task controller
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-13 13:02:24 +08:00
疯魔慕薇
5d7f757b7b
Merge pull request #12428 from mmpei/official-master-p2p-200708
Add P2P trigger event and handler
2020-07-09 15:54:24 +08:00
peimingming
65c5561032 Add P2P trigger event and handler
Signed-off-by: peimingming <peimingming@corp.netease.com>
2020-07-09 11:20:22 +08:00
chlins
b6cab91bfa feat(preheat): add healthcheck methods for p2p preheat controller
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-08 09:58:01 +08:00
fanjiankong
3c1c799f0d Get instance by name.
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-07-07 22:06:38 +08:00
stonezdj(Daojun Zhang)
6f4e8150d5
Merge pull request #12383 from ywk253100/200702_registry_api
Suport filtering registries by type in listing registry API
2020-07-07 14:21:54 +08:00
Wenkai Yin
02690d1d04 Suport filtering registries by type in listing registry API
Suport filtering registries by type in listing registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-07-07 10:30:46 +08:00
chlins
37a00912b7 feat: add p2p preheat swagger yaml and implement preheat api policy handler
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-06 21:05:22 +08:00
chlins
254ea193fa feat: add p2p preheat controller policy releated logic
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-04 12:23:04 +08:00
Steven Zou
1ee3f00709 fix(p2p):remove the provider manager related to
- use real provider instance manager
- move mock insatnce manager to testing/pkg
- modify kraken deriver implementation to remove digest fetcher
- update related UT cases

Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-03 17:30:11 +08:00
Steven Zou
c7a050b629
Merge pull request #12389 from chlins/fix/p2p-preheat-policy-count-total
fix: add count method of policy manager to replace list method return…
2020-07-03 13:50:34 +08:00
chlins
ace21240a4 fix: add count method of policy manager to replace list method return wrong counts
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-03 11:59:32 +08:00
fanjiankong
a0c2d0ac9e feat(preheat):add preheat api, controller and manager
- define instance's api
- define extension models for api
- implement preheat controller
- implement preheat manager
- most code are picked up from the original P2P feat branch

Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-07-03 11:25:42 +08:00
Steven Zou
18137a5c55 feat(preheat):implement policy enforcer
- define policy enforcer interface
- implement the default enforcer
- registrer P2P preheat job to JS
- add the missing mock manager&controller in the src/testing pkg
- Add UT cases for enforcer
- fix #12285
- left one TODO: query provider instance by instance Manager

Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-02 11:33:11 +08:00
Steven Zou
d8e88ef5bc feat(preheat):add artifact filters for preheat policy
- add new selector based on vulnerability severity criteria
- add new selector based on signature(signed) criteria
- do change to the select factory method definition
- do changes to selector.Candidate model
- add preheat policy filter interface and default implementation
- add UT cases to cover new code

Signed-off-by: Steven Zou <szou@vmware.com>

misspelling
2020-06-30 10:48:21 +08:00
AllForNothing
fff6f7529a Replace all whitelist with allowlist
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-06-24 16:17:17 +08:00
wang yan
0e175017aa continue updating code per review comments
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 14:42:28 +08:00
wang yan
c10467eb36 continue refactor
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 13:10:57 +08:00
Wang Yan
de504993ad update blob controller & manager
1, add two more attributes, update_time and status
2, add delete and fresh update time method in blob mgr & ctr.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 13:10:57 +08:00
He Weiwei
69fe9e9bf7
Merge pull request #11106 from qinshaoxuan/fix_11051
Fix bug when scanner is unhealthy
2020-05-20 21:06:22 +08:00
guanxiatao
0aefd8f2e2 Add UT for webhook when replicating with docker registry
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2020-05-13 19:09:33 +08:00
guanxiatao
e8655c667c Core panic fix when triggering a webhook of docker-registry replication
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2020-05-11 10:33:23 +08:00
He Weiwei
b1c9d452ce feat(quota,webhook): send quota webhook for put and mount blob
Closes #11712

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-28 00:59:16 +00:00
stonezdj(Daojun Zhang)
fb90bc23f2
Merge pull request #11757 from heww/fix-issue-11692
feat(scan): support to scan artifact automatic after it pushed
2020-04-27 12:08:06 +08:00
He Weiwei
bc1f7b8079 feat(scan): support to scan artifact automatic after it pushed
Closes #11692

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-26 15:56:58 +00:00
Wang Yan
8d5fc4c752
Fix delete repository event issue (#11722)
In the delete repository scenario, the repository has already been removed from the DB, the repository cannot be retrived.
Just let the creation time as empty.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-26 16:51:13 +08:00
Ziming Zhang
97a7a6dc35 fix(retention) fix retention repository with slash
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-04-26 15:07:25 +08:00
Wang Yan
8a0e8627ff
replace pkg errors with lib errors (#11605)
Fixes #9704

As we do want to unify error handling, so just decreprates pkg errors, use lib/errors instead for Harbor internal used errors model.

1, The lib/errors can cover all of funcs of pkg/errors, and also it has code attribute to define the http return value.
2, lib/errors can give a OCI standard error format, like {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}

If you'd like to use pkg/errors, use lib/errors instead. If it cannot meet your request, enhance it.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-15 22:41:45 +08:00
He Weiwei
f5487479dd feat(scanner): make Clair and Trivy as reserved name for scanners
Closes #11524

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-15 02:26:02 +00:00
Wenkai Yin
7553845b4d Remove the duplicated const definition
Remove the duplicated const definition for artifact type

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-04-13 17:02:23 +08:00
He Weiwei
49c9e4f696
Merge pull request #11585 from heww/cleanup-quota
refactor(quota): cleanup code for quota
2020-04-13 15:11:17 +08:00
He Weiwei
c0349da812 refactor(quota): cleanup code for quota
1. Remove `common/quota` package.
2. Remove functions about quota in `common/dao` package.
3. Move `Quota` and `QuotaUsage` models from `common/models` to
`pkg/quota/dao`.
4. Add `Count` and `List` methods to `quota.Controller`.
5. Use `quota.Controller` to implement quota APIs.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-12 16:16:06 +00:00
He Weiwei
4623cec1e5 feat(scan): revert bearer token support for scanner
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-11 08:45:29 +00:00