He Weiwei
06dc5bf10a
Merge pull request #12548 from heww/gen-and-check-mocks
...
chore(mocks): add make targets to generate and check mocks
2020-07-22 11:28:13 +08:00
He Weiwei
e3b1ec775f
refactor(quota): align Get and List methods of quota controller ( #12434 )
...
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-22 11:18:05 +08:00
Wang Yan
eeb8fca255
add debugging env for GC time window ( #12528 )
...
* add debugging env for GC time window
For debugging, the tester/users wants to run GC to delete the removed artifact immediately instead of waitting for two hours, add the env(GC_BLOB_TIME_WINDOW) to meet this.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-22 11:09:01 +08:00
Will Sun
46fa64462a
Merge pull request #12532 from AllForNothing/proxy-sum
...
Get registrry from project summary api
2020-07-22 11:01:39 +08:00
Will Sun
f9a66ca666
Merge pull request #12531 from AllForNothing/gc-dry-run
...
Add dry run to gc page
2020-07-22 11:01:10 +08:00
He Weiwei
aa55fcfce7
chore(mocks): add make targets to generate and check mocks
...
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-22 01:59:32 +00:00
Daniel Jiang
4170b267a1
Merge pull request #12496 from reasonerjt/secret-mgr-proxycache
...
Provide secret manager for proxy cache project
2020-07-22 09:48:24 +08:00
fanjiankong
adbdaaffe6
Fix: preheat install update transaction.
...
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-07-22 08:15:18 +08:00
jonasrosland
e7d1563c6b
Moving docs from the harbor to the website repo
...
Signed-off-by: jonasrosland <jrosland@vmware.com>
2020-07-21 12:41:24 -04:00
Wenkai Yin(尹文开)
112e38a080
Merge pull request #12541 from stonezdj/20200720_tag_ret_proxy
...
Add default retention policy
2020-07-21 19:31:12 +08:00
He Weiwei
f18ff08008
Merge pull request #12538 from heww/db-tx-skippers
...
perf(db): skip tx for get blob, patch/put blob upload apis
2020-07-21 19:04:18 +08:00
stonezdj
07694db90d
Release connection after search ldap user
...
Fixes: 12162
Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-07-21 18:00:39 +08:00
stonezdj
2f0f3c45d2
Add default retention policy
...
Remove images that are not visited in last 7 days
Add a build method in policy models.go
Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-07-21 16:49:35 +08:00
DQ
b62bebc005
Update vendor file
...
Signed-off-by: DQ <dengq@vmware.com>
2020-07-21 10:16:22 +08:00
AllForNothing
2396410cce
get registrry from project summary api
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-07-21 10:10:09 +08:00
DQ
f957acf763
Fix Chart V2 Can't parse
...
Use helm v3 libary to replace v2
Signed-off-by: DQ <dengq@vmware.com>
2020-07-21 10:07:08 +08:00
AllForNothing
8d9b1e18d3
Add dry run to gc page
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-07-21 09:51:31 +08:00
He Weiwei
6db1a1cb91
perf(db): skip tx for get blob, patch/put blob upload apis
...
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-20 16:42:16 +00:00
Steven Zou
cae087ba49
Merge pull request #12511 from steven-zou/fix/p2p_provider_health_check
...
fix(preheat):remove health property check when saving instance
2020-07-20 21:52:06 +08:00
Steven Zou
ee35e1ecc6
Merge pull request #12507 from chlins/fix/preheat-update-instance
...
fix(preheat): fix preheat handler PingInstance and UpdateInstance
2020-07-20 17:45:24 +08:00
jwangyangls
25e0f161ea
Merge pull request #12508 from jwangyangls/unified-processing-session-expired
...
[fix][ui] Operation unified processing session expired
2020-07-20 17:21:54 +08:00
Steven Zou
144edcc0c4
fix(preheat):fix ctl ut failure case
...
Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-20 17:14:17 +08:00
Steven Zou
6b9e0e66c4
Merge branch 'master' into fix/p2p_provider_health_check
2020-07-20 16:02:40 +08:00
Daniel Jiang
14203169bf
Add GC mechanism to secret manager
...
When Generate is called and the size is larger than cap, GC will be
triggered.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-07-20 15:53:36 +08:00
Steven Zou
46d7434d0b
Merge pull request #12473 from ywk253100/200706_scheduler
...
Refactor the scheduler with the task manager mechanism
2020-07-20 15:53:14 +08:00
chlins
78927af032
fix(preheat): fix preheat handler PingInstance and UpdateInstance
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-20 15:42:24 +08:00
Wenkai Yin
4dc4b6728c
Refactor the scheduler with the task manager mechanism
...
Refactor the scheduler with the task manager mechanism, this will reduce the duplicate code
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-07-20 14:03:15 +08:00
Steven Zou
3c02b5da03
fix(preheat):remove health property check when saving instance
...
Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-20 13:53:06 +08:00
Steven Zou
a11c2f32da
Merge pull request #12502 from bitsf/fix_replication_swr_pull
...
fix SWR replication adapter pull-based
2020-07-20 13:25:47 +08:00
Wang Yan
24ed52112e
fix blob deleting status issue ( #12481 )
...
1, The update blob status method should udpate the blob version of the blob object as well, otherwise the GC job cannot handle the blob status transform(none - delete - deleting - deletefailed)
as the method is using version equals as the query condition.
2, For the deleting blob which marked for more than 2 hours, it should be set to delete failed in head blob & put manifest request
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-20 11:44:29 +08:00
Wenkai Yin(尹文开)
5a898c1661
Merge pull request #12510 from ywk253100/200717_copy_proxy
...
Prevent copying artifact to a proxy cache project
2020-07-20 11:10:46 +08:00
Wenkai Yin(尹文开)
c654b08346
Merge pull request #12506 from ywk253100/200717_save_registry
...
Don't return the error detail back to the client when adding registry
2020-07-20 11:09:36 +08:00
Ziming Zhang
8857e89e40
feature(redis) support redis sentinel
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-07-19 21:19:03 +08:00
Daniel Jiang
d891e023db
Merge pull request #9311 from airadier/autoonboard-and-custom-user-claim
...
Add options for automatic onboarding and username claim
2020-07-19 19:15:34 +08:00
Steven Zou
d9ca9bbc69
fix(jobservice):add job id in log getting error ( #12513 )
...
Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-17 22:33:41 +08:00
Wenkai Yin
d7327d8d8e
Include the registry info in the project summary API for proxy cache project
...
Include the registry info in the project summary API for proxy cache project
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-07-17 16:14:45 +08:00
wang yan
3345b8aae2
fix get manifest return code
...
When to call,
~~~ REQUEST ~~~
GET /v2/conformance/testrepo/manifests/.INVALID_MANIFEST_NAME
Per OCI distribution spec, it has to return 404, instead of 400 (project name required)
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-17 15:40:52 +08:00
Yogi_Wang
7b22b38c5c
[fix][ui] Operation unified processing session expired
...
fix #11268 includes the following functions:
delete group delete chart version delete project delete member add group delete repo delete artifact delete tag delete robot delete user delete registry delete label delete replication list replication
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-07-17 15:33:39 +08:00
Wenkai Yin
54a1155140
Prevent copying artifact to a proxy cache project
...
Prevent copying artifact to a proxy cache project
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-07-17 15:24:18 +08:00
Wenkai Yin
9493611666
Don't return the error detail back to the client when adding registry
...
Don't return the error detail back to the client when adding registry to avoid security issue
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-07-17 11:58:08 +08:00
Ziming Zhang
e7c89ce1d9
fix SWR replication adapter pull-based
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-07-16 19:19:18 +08:00
Daniel Jiang
840aa86dfa
Provide secret manager for proxy cache project
...
This commit provides the secret manager for proxy cache.
The secret is used for pushing blobs to local when it's proxied from
remote registry.
Each secret can be used only once and has a relatively short expiration
time.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-07-16 19:12:52 +08:00
Alvaro Iradier
81a7239c66
Better error handling
...
* Raise an internal error if username claim is not found, instead of just logging a warning
* Don't remove userInfoKey for session on error when it is not required
* Rename "OIDC Username Claim" to just "Username claim"
Signed-off-by: Alvaro Iradier <airadier@gmail.com>
2020-07-16 12:12:08 +02:00
Alvaro Iradier
6f88ff7429
Fix test suite and add test for userClaim
...
Signed-off-by: Alvaro Iradier <airadier@gmail.com>
2020-07-16 12:12:08 +02:00
Alvaro Iradier
714f989759
Add options for automatic onboarding and username claim
...
- Add an option in the UI to enable or disable the automatic user onboarding
- Add an option to specify the claim name where the username is retrieved from.
Signed-off-by: Alvaro Iradier <airadier@gmail.com>
2020-07-16 12:12:08 +02:00
Wang Yan
bad8f026fc
upgrade golang to v1.14.5 ( #12489 )
...
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-16 16:20:54 +08:00
Wang Yan
d73265d10d
revise gc job to align non blocking gc ( #12439 )
...
two phases:
1, mark, select the gc candidates bases on the DB and mark them as status delete.
2, sweep, select the candidate and mark it as status deleting and remove it from backend and database.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-16 14:35:54 +08:00
Steven Zou
4d4a04fad4
Merge pull request #12478 from steven-zou/feat/read_pro_config
...
feat(p2p):enhance policy enforcer
2020-07-16 11:40:29 +08:00
Ted Guan
9e7edb7a6e
Fix for project metadata ( #12410 )
...
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2020-07-15 18:46:45 +08:00
Steven Zou
fcfde5a588
feat(p2p):enhance policy enforcer
...
Read security settings from the project configurations and override the preheat policy settings if necessary.
Check the project security settings and override the related settings in the policy if necessary.
NOTES: if the security settings (relevant with signature and vulnerability) are set at the project configuration,
they will have the highest priority and override the related settings of the preheat policy.
e.g (use signature as an example, similar case to vulnerability severity part):
if policy.signature = false and project.config.signature = true; then policy.signature = true
if policy.signature = true and project.config.signature = true; then policy.signature = true
if policy.signature = true and project.config.signature = false; then policy.signature = true
if policy.signature = false and project.config.signature = false; then policy.signature = false
Signed-off-by: Steven Zou <szou@vmware.com>
Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-15 14:18:34 +08:00
He Weiwei
cadcd4b877
Merge pull request #12480 from heww/move-pkg-types
...
refactor(quota): move pkg/types to pkg/quota/types
2020-07-15 11:32:27 +08:00
Daniel Jiang
947eadaa72
Merge pull request #12440 from heww/remove-init-clair-db
...
refactor: remove initialization of clair db
2020-07-15 00:38:12 +08:00
He Weiwei
a22d803a95
refactor(quota): move pkg/types to pkg/quota/types
...
Closes #9664
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-14 14:28:53 +00:00
stonezdj
b9c861f3f1
Add disable push for proxy project
...
Add middleware for blob and manifest push operation
Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-07-14 10:13:38 +08:00
疯魔慕薇
f187509a90
Merge pull request #12454 from chlins/feat/list-providers-at-project-level
...
feat(preheat): add list providers under project level handler
2020-07-14 08:17:53 +08:00
chlins
7322d0ac7c
feat(preheat): add list providers under project level handler
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-13 21:53:50 +08:00
stonezdj
3abe77d6cb
Add proxy cache feature
...
Update route to add proxy related middleware
Add proxy controller
Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-07-13 21:18:43 +08:00
chlins
08bd46e125
feat: add preheat execution api handler
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-13 13:14:08 +08:00
chlins
2863e68718
feat: add task controller
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-13 13:02:24 +08:00
fanjiankong
a99aa21c8a
Enable RBAC control in the preheat API
...
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-07-13 11:06:25 +08:00
AllForNothing
4ec919dfe6
Query preheat instance by name
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-07-11 17:41:56 +08:00
Julien Vey
84c34afa22
Fix typo in sql log
...
Signed-off-by: Julien Vey <vey.julien@gmail.com>
2020-07-10 14:29:04 +02:00
Will Sun
ec1ac6dbc8
Merge pull request #12344 from AllForNothing/p2p
...
Add P2p preheat distribution instance UI
2020-07-10 16:01:04 +08:00
Steven Zou
1dfc93c3f6
Merge pull request #12430 from kofj/preheat_and_healthcheck
...
Preheat and healthcheck
2020-07-10 15:17:34 +08:00
Steven Zou
fdff077ff0
Merge pull request #12445 from chlins/fix/preheat-instance-and-policy-name-validation
...
fix(preheat): validate instance/policy name and set unique name
2020-07-10 15:14:44 +08:00
chlins
38d14dff30
fix(preheat): validate instance/policy name, set unique filed and policy
...
manager adds parsePolicy
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-10 10:29:47 +08:00
fanjiankong
080afbfe1b
Add preheat APIs, handlers.
...
1. Manual preheat.
2. Instance health check.
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-07-10 09:48:35 +08:00
He Weiwei
039aef5356
refactor: remove initialization of clair db
...
To fetch vulnerability database updated time of the Clair had moved to
the Clair adapter so removes the initialization of clair db in the core.
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-09 15:26:14 +00:00
He Weiwei
9483559d18
Merge pull request #12433 from heww/fix-db-max-open-conns
...
fix(db): set max open conns of sql.DB manually
2020-07-09 17:58:14 +08:00
疯魔慕薇
5d7f757b7b
Merge pull request #12428 from mmpei/official-master-p2p-200708
...
Add P2P trigger event and handler
2020-07-09 15:54:24 +08:00
He Weiwei
e095958a27
fix(db): set max open conns of sql.DB manually
...
Due to the issues of beego v1.12.1 and v1.12.2, we set the max open conns
ourselves.
Closes #12403
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-07-09 03:35:41 +00:00
peimingming
65c5561032
Add P2P trigger event and handler
...
Signed-off-by: peimingming <peimingming@corp.netease.com>
2020-07-09 11:20:22 +08:00
fanjiankong
8a44ee400d
Instance handler.
...
- Add logic of preheat instance methods without RBAC.
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-07-09 00:01:14 +08:00
Steven Zou
3b2934bf48
Merge pull request #12419 from chlins/feat/p2p-preheat-healthcheck-controller
...
feat(preheat): add healthcheck methods for p2p preheat controller
2020-07-08 23:49:26 +08:00
Will Sun
71e50bd364
Merge pull request #12420 from AllForNothing/proxy-from-api
...
Add query string for getting registry
2020-07-08 14:20:38 +08:00
Will Sun
96b2326e24
Merge pull request #12418 from goharbor/rename-master-role
...
Rename master role to maintainer
2020-07-08 11:02:24 +08:00
AllForNothing
2c422435d9
Add query string for getting registry
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-07-08 10:42:55 +08:00
chlins
b6cab91bfa
feat(preheat): add healthcheck methods for p2p preheat controller
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-08 09:58:01 +08:00
Steven Zou
b87aa3b9d5
chore(preheat):add mock policy enforcer ( #12414 )
...
Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-08 09:45:14 +08:00
Will Sun
cd66899abb
Merge pull request #12417 from AllForNothing/rename-role
...
Rename project role for UI
2020-07-08 09:21:57 +08:00
Daniel Jiang
1637e6a588
Rename master role to maintainer
...
This commit rename the var name, text appearance, and swagger of "master" role
to "maintainer" role.
It only covers backend code.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-07-08 09:20:07 +08:00
AllForNothing
706c6bdb2b
Rename project role
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-07-08 09:15:01 +08:00
Steven Zou
b56a49efe2
Merge pull request #12411 from kofj/instance_by_name
...
Get instance by name.
2020-07-07 23:20:01 +08:00
fanjiankong
3c1c799f0d
Get instance by name.
...
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-07-07 22:06:38 +08:00
Wang Yan
c3baeac5ae
add time windows support in artifact trash ( #12400 )
...
support with time window to filter and delete item in artifact trash
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-07 20:13:08 +08:00
AllForNothing
d01ff31dc8
Add P2p preheat distribution instance UI
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-07-07 19:55:56 +08:00
stonezdj(Daojun Zhang)
6f4e8150d5
Merge pull request #12383 from ywk253100/200702_registry_api
...
Suport filtering registries by type in listing registry API
2020-07-07 14:21:54 +08:00
Wenkai Yin
02690d1d04
Suport filtering registries by type in listing registry API
...
Suport filtering registries by type in listing registry API
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-07-07 10:30:46 +08:00
Steven Zou
642953b9ef
Merge pull request #12395 from chlins/feat/preheat-controller-policy-part
...
Feat/preheat controller policy part
2020-07-06 22:21:34 +08:00
chlins
37a00912b7
feat: add p2p preheat swagger yaml and implement preheat api policy handler
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-06 21:05:22 +08:00
jwangyangls
e498893777
Merge pull request #12391 from jwangyangls/fix-replication-repo
...
[fix] [replication] Success rate should be displayed as 0 when status is inprogress
2020-07-06 10:09:20 +08:00
chlins
254ea193fa
feat: add p2p preheat controller policy releated logic
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-04 12:23:04 +08:00
Steven Zou
2f2c545a0f
Merge pull request #12392 from steven-zou/fix/remove_inst_provider_todo
...
fix(p2p):remove the provider manager related to
2020-07-03 23:59:55 +08:00
Will Sun
7dfab5858c
Merge pull request #12374 from AllForNothing/proxy-cache
...
Add proxy cache ui
2020-07-03 18:14:25 +08:00
Steven Zou
1ee3f00709
fix(p2p):remove the provider manager related to
...
- use real provider instance manager
- move mock insatnce manager to testing/pkg
- modify kraken deriver implementation to remove digest fetcher
- update related UT cases
Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-03 17:30:11 +08:00
Yogi_Wang
de4793c52d
[fix] [replication] Success rate should be displayed as 0 when status is inprogress
...
1.[replication] sccess rate should be displayed as 0 when status is inprogress
2.[repository] fix get repo name error when the repo contains the project name field
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-07-03 16:19:18 +08:00
Steven Zou
c7a050b629
Merge pull request #12389 from chlins/fix/p2p-preheat-policy-count-total
...
fix: add count method of policy manager to replace list method return…
2020-07-03 13:50:34 +08:00
Steven Zou
f3fcb96570
Merge pull request #12335 from kofj/p2p_preheat_api
...
feat(preheat):add preheat api, controller and manager
2020-07-03 13:47:04 +08:00
chlins
ace21240a4
fix: add count method of policy manager to replace list method return wrong counts
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-07-03 11:59:32 +08:00
Wang Yan
e8784de5fe
support list blobs by update time ( #12385 )
...
Add support list blob with update time.
As introduces the time window in GC, it wants to list the blobs less than specific time.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-03 11:50:53 +08:00
Daniel Jiang
cf07ff0052
Merge pull request #12386 from wy65701436/delete-blob-not-found
...
fix return value for blob & manifest not found
2020-07-03 11:37:31 +08:00
Daniel Jiang
8252930083
Merge pull request #12356 from wy65701436/put-mf-mw
...
add middlware for put manifest
2020-07-03 11:30:25 +08:00
Daniel Jiang
650142e536
Merge pull request #12381 from wy65701436/fix-delete-manifest
...
fix delete manifest route issue
2020-07-03 11:29:06 +08:00
fanjiankong
a0c2d0ac9e
feat(preheat):add preheat api, controller and manager
...
- define instance's api
- define extension models for api
- implement preheat controller
- implement preheat manager
- most code are picked up from the original P2P feat branch
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-07-03 11:25:42 +08:00
AllForNothing
a13642c2af
Add proxy cache ui
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-07-03 10:20:55 +08:00
wang yan
95bee9a0cc
fix return value for blob & manifest not found
...
When to delete an non exist blob/manifest, the API has to return a 404 instead of 500
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-02 19:18:57 +08:00
wang yan
7d1507feaa
update code arrording to the review comments
...
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-02 14:57:04 +08:00
wang yan
67be511a85
add middlware for put manifest
...
The middleware is to handle manifest(blob) status in different push manifest situation, similar with blob
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-02 14:57:04 +08:00
Steven Zou
264bd02892
Merge pull request #12378 from steven-zou/feat/preheat_policy_enforcement
...
feat(preheat):implement policy enforcer
2020-07-02 14:30:25 +08:00
wang yan
12c92dbfce
fix delete manifest route issue
...
The repository name contains blackslash, the mux router has to use the * to match the blackslash. Otherwise the caller(gc job) will get a 404.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-07-02 13:42:48 +08:00
Steven Zou
18137a5c55
feat(preheat):implement policy enforcer
...
- define policy enforcer interface
- implement the default enforcer
- registrer P2P preheat job to JS
- add the missing mock manager&controller in the src/testing pkg
- Add UT cases for enforcer
- fix #12285
- left one TODO: query provider instance by instance Manager
Signed-off-by: Steven Zou <szou@vmware.com>
2020-07-02 11:33:11 +08:00
Wang Yan
57c72b7952
add get GC candidate ( #12314 )
...
* add get GC candidate
select non referenced blobs from table blob and exclude the ones in the time windows.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-30 20:41:17 +08:00
Chlins Zhang
5bfe82612a
Merge pull request #12342 from chlins/feat/p2p-preheat-policy-dao-and-manager
...
feat: add p2p preheat policy dao and manager(#12286 )
2020-06-30 16:35:42 +08:00
chlins
15e4361d6e
feat: add p2p preheat policy dao and manager( #12286 )
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-06-30 15:56:50 +08:00
Steven Zou
a06c03625d
Merge pull request #12349 from steven-zou/feat/p2p_policy_evaluator
...
feat(preheat):add artifact filters for preheat policy
2020-06-30 14:56:58 +08:00
Steven Zou
d8e88ef5bc
feat(preheat):add artifact filters for preheat policy
...
- add new selector based on vulnerability severity criteria
- add new selector based on signature(signed) criteria
- do change to the select factory method definition
- do changes to selector.Candidate model
- add preheat policy filter interface and default implementation
- add UT cases to cover new code
Signed-off-by: Steven Zou <szou@vmware.com>
misspelling
2020-06-30 10:48:21 +08:00
Wang Yan
468ba50a7e
handle blob status chanage in put blob middlware ( #12315 )
...
* handle blob status chanage in put blob middlware
After blob is uploaded success, the middleware will update the blob status accordingly.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-29 16:38:47 +08:00
Daniel Jiang
7d50a6aab6
Merge pull request #12281 from pcfens/upgrade_aws_module
...
Upgrade aws-sdk-go to 1.32.5 for OIDC support
2020-06-29 16:35:11 +08:00
Steven Zou
8e2c334b43
Merge pull request #12338 from steven-zou/feat/p2p_preheat_job
...
feat(preheat):add preheat job
2020-06-29 12:27:05 +08:00
Steven Zou
6424480f37
feat(preheat):add preheat job
...
- implement the preheat job
- add relevant UT case
- fix issue #12285
Signed-off-by: Steven Zou <szou@vmware.com>
2020-06-28 23:08:02 +08:00
chlins
dd9e97f755
feat: add p2p preheat policy model
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-06-28 10:46:50 +08:00
Steven Zou
df86ae1ad0
feat(preheat):add preheat drivers
...
- define preheat driver interface
- implement dragonfly driver
- implememt kraken driver
- add related UT cases with testify framework
- fix #10870 #10871
- some code are picked up from the original P2P feat branch
Signed-off-by: Steven Zou <szou@vmware.com>
2020-06-25 23:39:34 +08:00
AllForNothing
fff6f7529a
Replace all whitelist with allowlist
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-06-24 16:17:17 +08:00
Phil Fenstermacher
33069e0f98
Upgrade aws-sdk-go
...
Versions of the Go AWS SDK newer than 1.23.13 support OIDC in EKS.
Running Harbor on EKS doesn't require keys in a configmap for the
registry to authenticate to S3 when using the newer library.
Signed-off-by: Phil Fenstermacher <pcfens@wm.edu>
2020-06-23 09:04:12 -04:00
Wenkai Yin(尹文开)
202916e396
Merge pull request #12280 from ywk253100/200616_task_manager
...
Implement task and execution manager
2020-06-23 18:44:44 +08:00
Wenkai Yin
ea20690264
Implement task and execution manager
...
Implement task and execution manager
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-06-23 17:10:58 +08:00
wang yan
0e175017aa
continue updating code per review comments
...
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 14:42:28 +08:00
wang yan
446739f967
rebase with latest source code
...
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 13:10:57 +08:00
wang yan
c10467eb36
continue refactor
...
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 13:10:57 +08:00
Wang Yan
de504993ad
update blob controller & manager
...
1, add two more attributes, update_time and status
2, add delete and fresh update time method in blob mgr & ctr.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-23 13:10:57 +08:00
Wang Yan
58b7242a25
move send error to source lib ( #12175 )
...
* move send error to source lib
Move the sendError into library in case the cycle dependency as regsitry and core are now the consumers.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-19 01:04:50 +08:00
Steven Zou
ffd889f82a
Merge pull request #12239 from ywk253100/200615_task_manager_dao
...
Implement execution/task DAO for task manager
2020-06-17 17:56:00 +08:00
stonezdj(Daojun Zhang)
91bff55b66
Merge pull request #12214 from stonezdj/20200611_add_proxyservice_secret
...
Add temporary secret for harbor proxy service
2020-06-17 10:46:13 +08:00
Wenkai Yin
5a1827768a
Implement execution/task DAO for task manager
...
Implement execution/task DAO for task manager
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-06-16 10:24:58 +08:00
Wenkai Yin
127988b70c
Define the task manager interface and data model
...
Define the task manager interface and data model
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-06-15 18:42:09 +08:00
stonezdj(Daojun Zhang)
56ca3dddda
Merge pull request #12203 from ywk253100/200610_proxy_cache_api
...
Update creating project API to support proxy cache project
2020-06-15 15:24:08 +08:00
stonezdj
82f59cb760
Add temporary secret for harbor proxy service
...
Use GenerateRandomStringWithLength function to generate secret for harbor_proxyservice
Add harbor-proxyservice secret used by proxy service
Signed-off-by: stonezdj <stonezdj@gmail.com>
2020-06-15 14:43:43 +08:00
Daniel Jiang
373ac25b9b
Merge pull request #12195 from reasonerjt/fix-post-user-403
...
Makes api/users return 401 for request not authenticated
2020-06-11 15:51:52 +08:00
Steven Zou
64f03cc7fe
Merge pull request #12079 from ninjadq/chartrepo_enumeration_master
...
Fix: chart repo response code inconsistent
2020-06-11 15:08:14 +08:00
Wenkai Yin(尹文开)
b08dfd57b6
Merge pull request #12127 from ywk253100/200603_label
...
Remove the label from artifacts when deleting a label
2020-06-11 14:49:14 +08:00
Wenkai Yin
06f7b7f763
Remove the label from artifacts when deleting a label
...
Fixes #12112 , remove the label from artifacts when deleting a label
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-06-11 14:02:46 +08:00
DQ
e1313da2f0
Fix: chart repo response code inconsistent
...
Response code should consistent whether namespace is existed or not
Signed-off-by: DQ <dengq@vmware.com>
2020-06-11 13:55:12 +08:00
Will Sun
3f4c8cc983
Merge pull request #12191 from AllForNothing/new-2.0.1
...
Fix UI issues with label target 2.0.1
2020-06-11 12:09:56 +08:00
AllForNothing
a773115cc3
Fix UI issues with label target 2.0.1
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-06-11 11:29:18 +08:00
jwangyangls
4b496438b1
Merge pull request #12204 from jwangyangls/fix-harbor-icon-safari-break
...
[fix][ui] about dialog Icon cutoff in safari
2020-06-11 10:15:09 +08:00
Daniel Jiang
9b4f2cb0bc
Merge pull request #12194 from reasonerjt/unknown-serverity-lower
...
Lower the severity of a "Unknown" vulnerability
2020-06-10 20:31:26 +08:00
Yogi_Wang
622cb4da2b
[fix][ui] about dialog Icon cutoff in safari
...
fix #12144
env 10.78.96.167
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-06-10 17:28:34 +08:00
Wenkai Yin
a79bb127b3
Update creating project API to support proxy cache project
...
Update creating project API to support proxy cache project
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-06-10 17:14:12 +08:00
Daniel Jiang
3b776d1a47
Makes api/users return 401 for request not authenticated
...
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-06-10 16:37:49 +08:00
Daniel Jiang
091dbc3454
Lower the severity of a "Unknown" vulnerability
...
This commit lower the actual severity of "Unknown" vulnerability to the
same level of "None"
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-06-10 15:16:25 +08:00
Daniel Jiang
0ac5568619
Merge pull request #12183 from reasonerjt/project-id-401
...
Change status code for projects API
2020-06-09 23:09:14 +08:00
jwangyangls
9e1778b32a
Merge pull request #12165 from jwangyangls/popup-unauthorized
...
[fix][unauthorized] Support show relogin when unauthorized in popup modal
2020-06-09 13:05:45 +08:00
Daniel Jiang
f73aa3ce80
Change status code for projects API
...
Update to projects API so it will not differentiate if a project
does not exist or the user doesn't have permission to access it.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-06-09 08:24:06 +08:00
Wang Yan
c993103e01
enable storage driver of gcs and oss ( #12180 )
...
Add go build tags for gcs and oss, otherwise these drivers cannot be registered and the error "StorageDriver is not regsited: GCS" will raise on registryctl launch under the setting of GCS storage.
These build tags are designed in the distribution, just refer to https://github.com/docker/distribution/blob/release/2.7/registry/storage/driver/gcs/gcs.go#L13
Pin the google cloud API to a old version is because distribution depends on it, otherwise go mode will use v0.17.0 that go-migrate is using as the dependency version, but this version will break the compile process with following error:
harbor/pkg/mod/google.golang.org/cloud@v0.0.0-20151119220103-975617b05ea8/storage/acl.go:65:16: invalid type assertion: v.(map[string]<inter>) (non-interface type *storage.ObjectAccessControl on left)
that's bacause another dependency google.golang.org/cloud requires the pinned version of google.golang.org/api.
The pinned package should be removed once https://github.com/docker/distribution/pull/3019 is merged, and distribution ships their v2.8.0
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-09 00:57:27 +08:00
Yogi_Wang
6f7aa68c89
[fix][unauthorized] Support show relogin when unauthorized in popup modal
...
1.support show relogin when unauthorized in popup modali
2.change change member role error handle
3.change helm chart service error handle
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-06-08 16:40:34 +08:00
Wang Yan
dec8397c21
Add api to delete blob and manifest ( #12006 )
...
* Add api to delete blob and manifest
Enable the capability of registry controller to delete blob and manifest
Signed-off-by: wang yan <wangyan@vmware.com>
2020-06-06 01:34:23 +08:00
Daniel Jiang
9f159393df
Merge pull request #12163 from reasonerjt/disable-webhook-test
...
Remove the API to test a webhook
2020-06-05 18:14:39 +08:00
Daniel Jiang
39a5efd54c
Merge pull request #12160 from reasonerjt/bump-up-go-migrator
...
Bump up golang-migrate
2020-06-05 18:13:40 +08:00
Daniel Jiang
9a9e7d61fc
Remove the API to test a webhook
...
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-06-05 13:16:18 +08:00
Daniel Jiang
bc7ede21c7
Bump up golang-migrate
...
This commit bumps it up to 4.11.0
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-06-05 10:29:02 +08:00
Wenkai Yin(尹文开)
b7b1ce457b
Merge pull request #10389 from lxShaDoWxl/fix/gitlab
...
Fixed search images in registry gitlab
2020-06-04 14:49:40 +08:00
Steven Zou
6db856c3e2
Merge pull request #12121 from steven-zou/fix/job_stop_status_issue
...
fix(jobservice):mismatch status issue when stopping job
2020-06-03 18:13:23 +08:00
Steven Zou
3cd47af9a5
fix(jobservice):mismatch status issue when stopping job
...
- returnning nil instead of error when trying to stop a job that has been in the final status(Error/Success/Stopped)
- do enhancements to the periodic job unschedule func
- fix a UT nil ptr issue
Signed-off-by: Steven Zou <szou@vmware.com>
2020-06-03 16:18:42 +08:00
AllForNothing
8bf77d01f4
Fix bugs with label 'target 2.0.1'
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-06-03 11:25:22 +08:00
jwangyangls
6939446c3c
Merge pull request #12115 from jwangyangls/fix-issue-add-label
...
[fix] Fix issue in front ui 2.0.1
2020-06-02 15:52:37 +08:00
Yogi_Wang
459314308b
[fix] issue in front ui 2.0.1
...
1.fix add label issue
2.fix unauthorized user can contrl page
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2020-06-02 13:54:22 +08:00
Steven Zou
c7c1742b88
Merge pull request #12106 from heww/clean-clair-url
...
refactor(configuration): cleanup unneeded CLAIR_URL configuration in core
2020-06-01 19:24:19 +08:00
Steven Zou
a032c546f8
Merge pull request #12096 from steven-zou/fix/remove_checkin_data
...
fix(jobservice):fix issues of jobservice
2020-05-29 16:49:33 +08:00
He Weiwei
d97be71234
refactor(configuration): cleanup unneeded CLAIR_URL configuration in core
...
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-05-29 07:27:50 +00:00
Steven Zou
51f1b2e590
fix(jobservice):fix issues of jobservice
...
- never expire the jobs that are not entering the final status (Error,Success or Stopped)
- set different expireation time to the jobs with different status
- never store the `check_in` data in the redis db to save space
Signed-off-by: Steven Zou <szou@vmware.com>
2020-05-29 11:36:08 +08:00
Chlins Zhang
f862805244
Merge pull request #11998 from chlins/fix/replication-pull-blob-content-length
...
fix(replication): ignore the problem of not getting Content-Length fr…
2020-05-29 10:06:44 +08:00
mmpei
7c2bfb1378
Fix helmhub insecure issue when using with a proxy ( #12014 )
...
Signed-off-by: peimingming <peimingming@corp.netease.com>
2020-05-28 10:57:51 +08:00
He Weiwei
9f5f8e4681
Merge pull request #12077 from heww/fix-issue-11951
...
fix(cve-whitelist): fix panic in `Get` method of whitelist manager
2020-05-27 19:16:11 +08:00
He Weiwei
0d1b9c96ff
fix(cve-whitelist): fix panic in Get
method of whitelist manager
...
Closes #11951
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-05-27 09:55:54 +00:00
chlins
54895def90
fix(replication): ignore the problem of not getting Content-Length from the pull blob api
...
Signed-off-by: chlins <chlins.zhang@gmail.com>
2020-05-27 13:00:54 +08:00
Will Sun
31a943bb42
Merge pull request #12043 from AllForNothing/disable-test
...
Remove test button for webhook
2020-05-27 11:46:58 +08:00
AllForNothing
ea753bec97
Remove test button for webhook
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-05-27 10:26:41 +08:00
Will Sun
da0e80b712
Merge pull request #11452 from jd615645/master
...
Chinese(Traditional) translation #402
2020-05-27 10:01:25 +08:00
lxShaDoWxl
681e97482e
fix(Registries/Gitlab): upgrade testing
...
Signed-off-by: lxShaDoWxl <lxshadowxkingxl@gmail.com>
2020-05-26 20:22:26 +06:00
lxShaDoWxl
05f78a561a
Merge remote-tracking branch 'origin/master' into fix/gitlab
2020-05-24 16:05:32 +06:00
Will Sun
2ff0638f8b
Merge pull request #10576 from hobti01/repo-slash
...
fix: Allow forward slash in destination namespace
2020-05-21 10:50:55 +08:00
yuzhiquan
6332daef31
struct instead of int within signal chan, and trans Printf to Println
...
Signed-off-by: yuzhiquan <yuzhiquanlong@gmail.com>
2020-05-21 00:47:55 +08:00
Wenkai Yin
a31315aa36
Fix replication bugs #11974 , #11939
...
Fix replication bugs: fix #11974 , fix #11939
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-05-21 00:25:49 +08:00
He Weiwei
69fe9e9bf7
Merge pull request #11106 from qinshaoxuan/fix_11051
...
Fix bug when scanner is unhealthy
2020-05-20 21:06:22 +08:00
ctu
7ea8ed2588
🐈 Add zh-tw translate
...
Signed-off-by: ctu <ctu@cs.nctu.edu.tw>
2020-05-20 10:56:29 +00:00
Tim Hobbs
36747087ce
fix: Allow forward slash in destination namespace
...
Signed-off-by: Tim Hobbs <timothy.hobbs@ic-consult.com>
2020-05-20 11:27:50 +02:00
Daniel Jiang
b803975b36
Merge pull request #11971 from wy65701436/fixes-11949
...
fix conformance issue on deleting manifest
2020-05-20 13:19:47 +08:00
AllForNothing
2d141a919d
Add more UT for create project component
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-05-19 17:39:42 +08:00
wang yan
f0fabb9ef8
fix conformance issue on deleting manifest
...
fixes #11949
When to call delete manifest API with a tag as the reference, Harbor should give a unsupported error code.
Reference: Note that a manifest can only be deleted by digest. https://github.com/opencontainers/distribution-spec/blob/master/spec.md#delete-manifest
Signed-off-by: wang yan <wangyan@vmware.com>
2020-05-19 15:58:33 +08:00
Wang Yan
b1793e795c
fix conformance test ErrorsCodes failure ( #11961 )
...
fixes #11945
When client calls PUT with an invalid digtest, Harbor has to give a 400 instead of 500.
Signed-off-by: wang yan <wangyan@vmware.com>
2020-05-18 17:20:23 +08:00
guanxiatao
0aefd8f2e2
Add UT for webhook when replicating with docker registry
...
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2020-05-13 19:09:33 +08:00
AllForNothing
1c670329dc
Fix duration and log bugs for replication task UI
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-05-13 12:38:23 +08:00
Will Sun
5289b3519e
Merge pull request #11860 from AllForNothing/docker-logo
...
Modify doc and fix some ui bugs
2020-05-11 11:00:36 +08:00
guanxiatao
e8655c667c
Core panic fix when triggering a webhook of docker-registry replication
...
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
2020-05-11 10:33:23 +08:00
AllForNothing
c50cbbf3cc
Modify doc and fix some ui bugs
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-05-11 09:39:33 +08:00
Will Sun
119751e0fd
Merge pull request #11835 from AllForNothing/ui-bugs
...
Fix some UI bugs
2020-05-07 11:45:57 +08:00
AllForNothing
bdf6c5b8d9
Fix some UI bugs
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-05-06 16:41:35 +08:00
lxShaDoWxl
47a57b80d7
Merge remote-tracking branch 'origin/master' into fix/gitlab
2020-05-06 13:26:18 +06:00
AllForNothing
90e34e0104
Improve i18n service
...
Signed-off-by: AllForNothing <sshijun@vmware.com>
2020-05-06 14:45:56 +08:00
Ziming
56609a8026
fix(retention) set pushtime and pulltime for untagged ( #11786 )
...
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-04-28 19:20:46 +08:00