Commit Graph

103 Commits

Author SHA1 Message Date
Wenkai Yin
d6288a43e8 Do some refine for the scheduler
1. Accept vendorType and vendorID when creating the schedule
2. Provide more methods in the scheduler interface to reduce the duplicated works of callers
3. Use a new ormer and transaction when creating the schedule

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-08-05 17:43:18 +08:00
He Weiwei
df1bdc1020 refactor(project): add more methods to project controller and manager
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-31 17:55:35 +00:00
Daniel Jiang
3445608e62
Merge pull request #12416 from jonasrosland/move-docs-to-website
Moving docs from the harbor to the website repo
2020-07-27 09:45:37 +08:00
Daniel Jiang
2041fd337b
Merge pull request #12448 from ninjadq/update_chart_client
Fix Chart V2 Can't parse
2020-07-22 13:31:05 +08:00
He Weiwei
06dc5bf10a
Merge pull request #12548 from heww/gen-and-check-mocks
chore(mocks): add make targets to generate and check mocks
2020-07-22 11:28:13 +08:00
He Weiwei
e3b1ec775f
refactor(quota): align Get and List methods of quota controller (#12434)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-22 11:18:05 +08:00
He Weiwei
aa55fcfce7 chore(mocks): add make targets to generate and check mocks
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-22 01:59:32 +00:00
jonasrosland
e7d1563c6b Moving docs from the harbor to the website repo
Signed-off-by: jonasrosland <jrosland@vmware.com>
2020-07-21 12:41:24 -04:00
DQ
f957acf763 Fix Chart V2 Can't parse
Use helm v3 libary to replace v2

Signed-off-by: DQ <dengq@vmware.com>
2020-07-21 10:07:08 +08:00
Steven Zou
46d7434d0b
Merge pull request #12473 from ywk253100/200706_scheduler
Refactor the scheduler with the task manager mechanism
2020-07-20 15:53:14 +08:00
Wenkai Yin
4dc4b6728c Refactor the scheduler with the task manager mechanism
Refactor the scheduler with the task manager mechanism, this will reduce the duplicate code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-07-20 14:03:15 +08:00
Wang Yan
24ed52112e
fix blob deleting status issue (#12481)
1, The update blob status method should udpate the blob version of the blob object as well, otherwise the GC job cannot handle the blob status transform(none - delete - deleting - deletefailed)
as the method is using version equals as the query condition.
2, For the deleting blob which marked for more than 2 hours, it should be set to delete failed in head blob & put manifest request

Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-20 11:44:29 +08:00
He Weiwei
a22d803a95 refactor(quota): move pkg/types to pkg/quota/types
Closes #9664

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-14 14:28:53 +00:00
Will Sun
96b2326e24
Merge pull request #12418 from goharbor/rename-master-role
Rename master role to maintainer
2020-07-08 11:02:24 +08:00
Steven Zou
b87aa3b9d5
chore(preheat):add mock policy enforcer (#12414)
Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-08 09:45:14 +08:00
Daniel Jiang
1637e6a588 Rename master role to maintainer
This commit rename the var name, text appearance, and swagger of "master" role
to "maintainer" role.
It only covers backend code.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-07-08 09:20:07 +08:00
Steven Zou
b56a49efe2
Merge pull request #12411 from kofj/instance_by_name
Get instance by name.
2020-07-07 23:20:01 +08:00
fanjiankong
3c1c799f0d Get instance by name.
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-07-07 22:06:38 +08:00
Wang Yan
c3baeac5ae
add time windows support in artifact trash (#12400)
support with time window to filter and delete item in artifact trash

Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-07 20:13:08 +08:00
chlins
37a00912b7 feat: add p2p preheat swagger yaml and implement preheat api policy handler
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-06 21:05:22 +08:00
Steven Zou
1ee3f00709 fix(p2p):remove the provider manager related to
- use real provider instance manager
- move mock insatnce manager to testing/pkg
- modify kraken deriver implementation to remove digest fetcher
- update related UT cases

Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-03 17:30:11 +08:00
chlins
ace21240a4 fix: add count method of policy manager to replace list method return wrong counts
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-03 11:59:32 +08:00
Steven Zou
264bd02892
Merge pull request #12378 from steven-zou/feat/preheat_policy_enforcement
feat(preheat):implement policy enforcer
2020-07-02 14:30:25 +08:00
Steven Zou
18137a5c55 feat(preheat):implement policy enforcer
- define policy enforcer interface
- implement the default enforcer
- registrer P2P preheat job to JS
- add the missing mock manager&controller in the src/testing pkg
- Add UT cases for enforcer
- fix #12285
- left one TODO: query provider instance by instance Manager

Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-02 11:33:11 +08:00
Wang Yan
57c72b7952
add get GC candidate (#12314)
* add get GC candidate

select non referenced blobs from table blob and exclude the ones in the time windows.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-30 20:41:17 +08:00
wang yan
0e175017aa continue updating code per review comments
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 14:42:28 +08:00
wang yan
c10467eb36 continue refactor
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 13:10:57 +08:00
Wang Yan
de504993ad update blob controller & manager
1, add two more attributes, update_time and status
2, add delete and fresh update time method in blob mgr & ctr.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 13:10:57 +08:00
Wenkai Yin(尹文开)
b08dfd57b6
Merge pull request #12127 from ywk253100/200603_label
Remove the label from artifacts when deleting a label
2020-06-11 14:49:14 +08:00
Wenkai Yin
06f7b7f763 Remove the label from artifacts when deleting a label
Fixes #12112, remove the label from artifacts when deleting a label

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-06-11 14:02:46 +08:00
Wang Yan
dec8397c21
Add api to delete blob and manifest (#12006)
* Add api to delete blob and manifest

Enable the capability of registry controller to delete blob and manifest

Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-06 01:34:23 +08:00
Wang Yan
c6860ac35f
handle stop signal in GC job (#11612)
Stop GC job in the init if receives the stop signal

Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-14 18:59:03 +08:00
He Weiwei
c0349da812 refactor(quota): cleanup code for quota
1. Remove `common/quota` package.
2. Remove functions about quota in `common/dao` package.
3. Move `Quota` and `QuotaUsage` models from `common/models` to
`pkg/quota/dao`.
4. Add `Count` and `List` methods to `quota.Controller`.
5. Use `quota.Controller` to implement quota APIs.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-04-12 16:16:06 +00:00
wang yan
a11a70d941 move logger from common to lib
The logger is the fundamental library, so move it into lib folder
Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-02 14:09:03 +08:00
Wang Yan
f6c0608e22
fix GC jobs upgrade issue (#11365)
Fixes #11313
Fixes #11275

1, Add more details log in GC job
2, Add type assertion for the upgrading case, the delete_untagged parameter is introduced from v2.0
3, Add UT

Signed-off-by: wang yan <wangyan@vmware.com>
2020-04-01 11:53:41 +08:00
Wang Yan
d6261d9456
Does not throw err in the notification job (#11363)
Fixes #11280, no error return but just log.

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-31 18:12:12 +08:00
He Weiwei
9c06c79ff4 refactor(errors): rename pkglib/error to lib/errors
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-29 05:19:05 +00:00
He Weiwei
73f3a305ce
refactor: rename testing/api to testing/controller (#11295)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-26 14:00:11 +08:00
Wenkai Yin(尹文开)
8984979bd2
Relocate/rename some packages (#11183)
Fixes #11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-24 20:45:45 +08:00
He Weiwei
5641ae49df feat(gc,quota): refersh quotas for projects after gc
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-20 09:13:50 +00:00
He Weiwei
21349e30af
feat(middleware,vulnerable): add image index checking for vulnerability prevention (#11084)
1. Skip vulnerability prevention checking when artifact is not
scannable.
2. Skip vulnerability prevention checking when artifact is image index
and its type is `IMAGE` or `CNAB`.
3. Skip vulnerability prevention checking when the artifact is pulling
by the scanner.
4. Change `hasCapability` from blacklist to whitelist.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-19 10:48:19 +08:00
Wang Yan
b4e941e961
drop table access log in migration (#11118)
Use the audit log instead, the access log table should be dropped after migration

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-18 19:04:38 +08:00
Wang Yan
0422721490
Enable pull time on getting manifest (#11110)
Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-18 18:38:37 +08:00
Wenkai Yin
3aca33acde Clean up some TODO items
1. Remove blob fetcher and cache

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-17 19:01:38 +08:00
Wang Yan
fbb3226e85
move notification handles and events metadata into api (#11085)
1, enable audit logs for notifications
2, move the handler and meatadata into API
3, use the notification middleware to send out notification

Signed-off-by: wang yan <wangyan@vmware.com>
2020-03-16 16:56:34 +08:00
He Weiwei
60f8595034
refactor(quota): implement internal quota APIs by quota controller (#11058)
1. Use quota controller to implement the internal quota APIs.
2. The internal quota APIs can exceed the quota limitations.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-16 10:20:17 +08:00
He Weiwei
ec31a87884
fix(blob,quota): process blobs already in registry no but associated with project (#11071)
1. Before put manifest request, ensure that the requested size resource
include the blobs which are referenced by the manifest but not
associated with project.
2. After put manifest request, associate the blobs which are referenced
by the manifest but not associated with project.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-14 21:39:44 +08:00
He Weiwei
2a243ef7a2
refactor(rbac): refactor rbac impl to improve performance (#9988)
1. Introduce `Evaluator` interface which do the permission checking.
2. `admin`, `lazy`, `rbac`, `namespace` and `evaluartor` set are implemented the
`Evaluator` interface.
3. Move project rbac implemention from `project` to `rbac` pkg to reduce
the name  conflict with project instance of model.
4. Do permission checking in security context by `Evaluator`.
5. Cache the regexp in rbac evaluator for casbin.
6. Cache evaluator in namespace evaluator to improve performance.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-12 23:42:53 +08:00
He Weiwei
12f16c8cec
feat(scan): support to scan image index (#11001)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-03-12 19:30:12 +08:00
Wenkai Yin
289f04d301 Restructure the packages of artifact
1. Introduce a new interface Processor to replace Abstractor and Descriptor
2. Provide the base processors for manifest and index to reduce the duplicate code
3. Move the child artifacts checking out of processor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-03-12 16:03:13 +08:00