Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-12-19 15:17:43 +01:00
Code Issues Projects Releases Wiki Activity
9,386 Commits 55 Branches 319 Tags 338 MiB
a8f159728b
Commit Graph

4172 Commits

Author SHA1 Message Date
Wenkai Yin(尹文开)
e8cc84738a
Merge pull request #11309 from ywk253100/200326_error 
Fix bugs of replication
 2020-03-27 10:31:03 +08:00
Wang Yan
a5c1eae81a
give the username to anonymous when to pull public resource without authN (#11306) 
For pull a public resource, there is no need to login, give the access name to anonymous in the audit logs

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-27 10:24:59 +08:00
Wenkai Yin
9a92b9e725 Fix bugs of replication 
1. Bump up the version of API used in replicatoin scheduler job
2. Check the error message to determine whether the job exists or not in jobservice when unschedule a job

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 23:12:02 +08:00
Wenkai Yin
afdfedcb49 Iterate the link header when listing artifact 
Fixes #11315
When specify no pagination in listing artifact request, the go-swagger will set the default value for them, so we need to iterate the link header to get all of artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 21:26:09 +08:00
Daniel Pacak
48df949c30
feat(trivy): Return Trivy DB update timestamp in /api/v1/metadata response (#11285) 
Resolves: #11284

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-26 19:22:28 +08:00
Wenkai Yin
64e4651b3f Support replicate images with media type application/vnd.docker.distribution.manifest.v1+json 
Fixes #11272, support replicate images with media type application/vnd.docker.distribution.manifest.v1+json

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 19:09:48 +08:00
Wenkai Yin
213c534e8a Return 404 rather than 500 error when getting registry info 
In Harbor 2.0, the replication isn't supported between instances with different versions, this commit returns the 404 error when trying to get the registry info whose version is different with the current one

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 18:07:11 +08:00
He Weiwei
73f3a305ce
refactor: rename testing/api to testing/controller (#11295) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-26 14:00:11 +08:00
jwangyangls
96572c3c86
Merge pull request #11254 from jwangyangls/nightly-case-3 
[Fix]   Fix issue 2.0
 2020-03-26 12:14:37 +08:00
Yogi_Wang
cba4490a5a [Fixed] Fix bug for 2.0 and add case for trivy 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
1.add case for trivy
2.vunerbility refresh bug
3.scan mutiple artifact
4.fix global search bug
5.disable delete tag btn when remove immutable tag
6.cancel selectRow when add label or remove label;fix #11195
7.fix cron tootip
 2020-03-26 11:39:57 +08:00
qinshaoxuan
df9c2bdc46 Fix bug when scanner is unhealthy 
The function GetRegistrationByProject should not return err when Ping
return err.  The return value 'registration' has 'Health' field which
shows the scanner health status.

Resolves: #11051
See also: #9788, #9807

Signed-off-by: qinshaoxuan <qinshaoxuan@baidu.com>
 2020-03-26 11:25:47 +08:00
Wang Yan
da8902da53
Handle empty orlist in orm query (#11270) 
Fixes #11267
When caller parse an empty orlist to orm lib, it will parse the empty vaule to beego orm.
But beego will panic if the query string is empty.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-26 11:23:20 +08:00
Wenkai Yin(尹文开)
d05817c8a2
Update the URL checking logic of auth proxy security generator (#11180) 
As we don't support bearer token in Harbor 2.0, the URL checking logic in auth proxy security generator should be updated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-25 01:19:11 +08:00
Wenkai Yin(尹文开)
8984979bd2
Relocate/rename some packages (#11183) 
Fixes #11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-24 20:45:45 +08:00
Ted Guan
e49a247d3d
Replication webhook support (#11179) 
* replication webhook support

Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>

* replication webhook support with ut fixed

Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2020-03-23 18:45:58 +08:00
Wang Yan
168637a743
Add permission check for audit logs API (#11154) 
add a base method to require system admin permission

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-23 16:59:10 +08:00
Wang Yan
15d2a93aa2
Fix orm query setter issue (#11177) 
For the Andlist, the query setter should ignore it

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-23 16:05:20 +08:00
Wenkai Yin(尹文开)
1762bfae69
Merge pull request #11158 from ywk253100/200320_repository_api 
Add "_self" suffix for repository API to avoid conflict
 2020-03-23 14:45:52 +08:00
Wenkai Yin(尹文开)
c4af6ff824
Fix bug when deleting the repository (#11121) 
Fixes #10997 by looping the artifact candidates until all of them are deleted

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-23 10:38:47 +08:00
Wenkai Yin(尹文开)
8688f78cd2
Merge pull request #11144 from ywk253100/200319_security_middleware 
Rewrite the filters with middleware mechinism
 2020-03-23 10:12:48 +08:00
Will Sun
b740903314
Merge pull request #11143 from AllForNothing/resolver 
Add routing-resolvers
 2020-03-23 10:12:17 +08:00
Wenkai Yin
0453709b74 Rewrite the filters with middleware mechinism 
Fixes 10532,rewrite the filters with middleware mechinism

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-23 09:26:20 +08:00
jwangyangls
61d7eaa405
Merge pull request #11160 from jwangyangls/repo-pagination 
[fix] fix repo pagination
 2020-03-20 19:46:02 +08:00
Wenkai Yin(尹文开)
5f982bfee1
Merge pull request #11163 from heww/gc-refresh-quotas 
feat(gc,quota): refersh quotas for projects after gc
 2020-03-20 19:23:36 +08:00
Wenkai Yin(尹文开)
bf3b185357
Merge pull request #11162 from reasonerjt/rm-reset-pwd 
Remove route entry to reset password
 2020-03-20 19:17:53 +08:00
Yogi_Wang
2786a3347c [fix] fix repo pagination 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-20 17:22:28 +08:00
He Weiwei
5641ae49df feat(gc,quota): refersh quotas for projects after gc 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-20 09:13:50 +00:00
Wenkai Yin
dca06b6ede Add "_self" suffix for repository API to avoid conflict 
Add "_self" suffix for repository API to avoid conflict

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-20 17:08:08 +08:00
danfengliu
b51076ffa8
Merge pull request #11151 from jwangyangls/oci-nightly-change-2 
[OCI] Change nightly case and change delete artifact words
 2020-03-20 16:32:03 +08:00
Yogi_Wang
13ae4482ab [OCI] Change nightly case and change delete artifact words 
1.nightly: fix tag retention and immutable tag case xpath
2.nightly: fix the part of delete repo button xpath
3.nightly: fix the api version when GC
4.nightly: fix add label of artifact xpath
5.text:   change delete artifact show words
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-20 15:59:16 +08:00
Daniel Jiang
9b750f60df Remove route entry to reset password 
fixes #10712
The functions in CommonController are kept as a reference.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-20 14:05:21 +08:00
Wang Yan
63cf1fce7f
Unescape tags query when to list artifact (#11148) 
The query string is encoded by UI, and we have to unescape the "=" in "q=tag=nil",
otherwise, the query doesn't work, and returns 400

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-20 10:07:34 +08:00
Steven Zou
e8d5ba9491
Merge pull request #11004 from steven-zou/fix/update_js_ut_case 
fix[js_ut]:update stop job case of js
 2020-03-19 15:49:08 +08:00
Wang Yan
dc6eec8a73
Enable API logs test case (#11142) 
1, enable user view log api test case
2, update project logs api permission check
3, use project ctl instead in permission check base method

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-19 14:56:37 +08:00
AllForNothing
887d693fa4 Add routing-resolvers 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-19 14:45:04 +08:00
He Weiwei
21349e30af
feat(middleware,vulnerable): add image index checking for vulnerability prevention (#11084) 
1. Skip vulnerability prevention checking when artifact is not
scannable.
2. Skip vulnerability prevention checking when artifact is image index
and its type is `IMAGE` or `CNAB`.
3. Skip vulnerability prevention checking when the artifact is pulling
by the scanner.
4. Change `hasCapability` from blacklist to whitelist.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-19 10:48:19 +08:00
Wang Yan
9e4fdc571a
update internal common error (#10994) 
1, New support construct with string or err
2, Add Wrap/Errorf method

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-19 10:46:07 +08:00
Qian Deng
cf90ec27f2
Merge pull request #10706 from ninjadq/enable_tls_on_all_components 
Enable tls on all components
 2020-03-18 21:25:40 +08:00
Wenkai Yin(尹文开)
dbedcf960c
Merge pull request #11128 from wy65701436/perf-list-artifact 
simplify query string when to list artifact
 2020-03-18 21:00:19 +08:00
Wenkai Yin(尹文开)
c505c82d57
Merge pull request #11126 from ywk253100/200318_label_resource 
Remove the API to listing the resources that added with the specific label
 2020-03-18 20:58:28 +08:00
He Weiwei
fe39bb6a2a
feat(quota,notification): notification for quota exceeded and warning (#11123) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-18 20:24:23 +08:00
DQ
4c30995858 Refator tls config 
use default Httptransport instead of empty one
remove unused code

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
b93092e012 Add tls for trivy 
Add trivy tls cert files
Add tivey tls env and config
enhance gencert

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
c954969bcd Add mTLS configs 
mTLS only enabled in jobservice and registryctl

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
454382149f TLS update for chart, clairadapter, registry 
Remove trustca in chartmuseum
Remove trustca in registry
Add tls in clair-adapter

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
03e11c63c7 Fix docker file with secure tls change 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
115185894f Merge internal Transport and Secure Transport 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
02dea3ad2c Add: mTLS configuration on CI 
Add internal_tls on ci
generate certs for ci

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
dcc6950af7 Feat: auto install ca in registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
b852605193 Feat: enable mtls in harbor replication 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
40e67f3b14 Feat: Enable mtls for registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
07a1d51693 Feat: enable tls in registryctlAdd tls related code in registryctl 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
10753caf90 Feat: enable tls in chart 
add tls related code in chart server

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
e6bb3b0977 Feat: enable tls related thing to jobservice 
Add tls related code in jobservice

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
da359f609f Feat: enable mtls in core 
add mtls related code in core

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
Wang Yan
b4e941e961
drop table access log in migration (#11118) 
Use the audit log instead, the access log table should be dropped after migration

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-18 19:04:38 +08:00
wang yan
3deef8a7d4 simplify query string when to list artifac 
To improve the performance of loading repository page, make the query set thinner.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-18 18:53:33 +08:00
Wenkai Yin
c92d9e4034 Remove the API to listing the resources that added with the specific label 
As we introduce a new table to record the relationship between the artifacts and labels, the current way to list label's resources doesn't work anymore, and the API isn't needed by any features, remove it in 2.0

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-18 18:47:44 +08:00
Wang Yan
0422721490
Enable pull time on getting manifest (#11110) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-18 18:38:37 +08:00
Wenkai Yin
256796ea9b Remove the project manager from context 
Remove the project manager introduced when integrated with Admiral from the context

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-18 15:28:02 +08:00
Wenkai Yin(尹文开)
eb2af6095e
Merge pull request #11107 from ywk253100/filter_replication_pull 
Filter the pulling manifest request from replication service
 2020-03-18 14:36:29 +08:00
Wenkai Yin(尹文开)
798dda8604
Escapse the repository name in the link header returned in response (#11037) 
Escapse the repository name in the link header returned in response

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-18 14:29:15 +08:00
He Weiwei
7d20154db5
fix: remove old artifact model (#11112) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-18 14:20:06 +08:00
jwangyangls
2f7ce0da1c
Merge pull request #11096 from jwangyangls/fix-some-issue-oci 
[OCI] changes some show words
 2020-03-18 14:00:13 +08:00
Wang Yan
050967f95f
Use new query model to get audit logs (#11113) 
* Use new query model to get audit logs

leverage the query builder to build query, remove the old style query string

Signed-off-by: wang yan <wangyan@vmware.com>

* Switch to new API  for  project log page

Signed-off-by: AllForNothing <sshijun@vmware.com>

Co-authored-by: AllForNothing <sshijun@vmware.com>
 2020-03-18 13:46:49 +08:00
Yogi_Wang
891ef80e46 [OCI] changes some show words 
1. search result show artifact count
2. replication shows changes both to all
3. fix delete bug when delete some artifact
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-18 12:17:30 +08:00
Wang Yan
7af0bd5ed7
Fix delete scan report on deleting artifact (#11102) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-17 21:25:06 +08:00
Wenkai Yin(尹文开)
f02c5570a7
Merge pull request #11111 from ywk253100/200317_clean_todo 
Clean up some TODO items
 2020-03-17 20:17:22 +08:00
He Weiwei
e3c5c37668
fix(scan): assign repository pull access policy to robot account when scan artifact (#11109) 
The v2auth middleware will check whether the requestor  has the pull or
push permissions for the repository, and forbid the request when the
requestor does not have the permission.  We need to assign repository
pulling permission to the robot account for the scanner, otherwise
scanner will be failed to pull the artifact.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-17 19:30:21 +08:00
He Weiwei
f8983fe198
feat(log): track request id in the log message (#11095) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-17 19:29:59 +08:00
Wenkai Yin
3aca33acde Clean up some TODO items 
1. Remove blob fetcher and cache

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-17 19:01:38 +08:00
Wenkai Yin
e8935dd804 Filter the pulling manifest request from replication service 
Filter the pulling manifest request from replication service so that the audit log will not record the pulling action

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-17 16:32:11 +08:00
Wang Yan
ce2257dc22
update project logs api to v2.0 (#11097) 
use audit log api to get project logs

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-17 11:58:43 +08:00
Will Sun
2b6fb4abcf
Merge pull request #11073 from AllForNothing/permission 
Swith to new API for recent log page
 2020-03-17 11:25:29 +08:00
Wenkai Yin(尹文开)
411c73bd79
Merge pull request #11046 from ywk253100/200312_replication 
Replicate tag deletion between Harbor instances
 2020-03-17 10:58:06 +08:00
jwangyangls
89cdd7a9f9
Merge pull request #11089 from jwangyangls/clear-dead-code 
[OCI] Remove dead code
 2020-03-17 10:03:18 +08:00
jwangyangls
580b74035d
Merge pull request #11007 from jwangyangls/artifact-filter-changes 
[OCI] Artifact filter params changes in ui
 2020-03-17 10:02:50 +08:00
Wenkai Yin
5925e0862d Replicate tag deletion between Harbor instances 
This commit introduces the tag deletion as a new capability for registry adapters, and currently only Harbor supports it

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-17 09:27:02 +08:00
Yogi_Wang
61fa461e91 [OCI] Remove dead code 
1. remove tag service / tag model
2. remove retag service
3. remove artifact service some function
4. remove repository service / repository model  /repositoryItem model
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-16 17:24:21 +08:00
Wang Yan
fbb3226e85
move notification handles and events metadata into api (#11085) 
1, enable audit logs for notifications
2, move the handler and meatadata into API
3, use the notification middleware to send out notification

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-16 16:56:34 +08:00
Wenkai Yin(尹文开)
a83c78c1a5
Merge pull request #11064 from ywk253100/200313_cached_token 
Check expired or not when getting token from cache
 2020-03-16 16:27:18 +08:00
Wenkai Yin(尹文开)
89eeeb29ca
Change tag count to artifact count in search result (#11068) 
Change tag count to artifact count in search result

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-16 14:28:59 +08:00
Wenkai Yin(尹文开)
d250e6998e
Fix bug when reading the readme.md of helm chart (#11059) 
Fix bug when reading the readme.md of helm chart

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-16 14:26:50 +08:00
jwangyangls
46fe1182b7
Merge pull request #11043 from jwangyangls/add-OPENPOLICYAGENT 
[OCI] Add the icon of openpolicyagent artifact
 2020-03-16 13:39:10 +08:00
Daniel Jiang
2615339f4c
Merge pull request #11076 from reasonerjt/csrf-secure-flag 
make Secure flag of CSRF cookie adapt to config
 2020-03-16 11:47:51 +08:00
He Weiwei
60f8595034
refactor(quota): implement internal quota APIs by quota controller (#11058) 
1. Use quota controller to implement the internal quota APIs.
2. The internal quota APIs can exceed the quota limitations.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-16 10:20:17 +08:00
Daniel Jiang
cbd2619035 make Secure flag of CSRF cookie adapt to config 
fixes #11074

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-16 04:03:49 +08:00
Wang Yan
9cc6e88a65
add notification middleware (#11072) 
the notification is for send out the event after DB transaction complete.
It's safe to send hook as this middleware is after transaction in the response path.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-14 22:34:36 +08:00
He Weiwei
ec31a87884
fix(blob,quota): process blobs already in registry no but associated with project (#11071) 
1. Before put manifest request, ensure that the requested size resource
include the blobs which are referenced by the manifest but not
associated with project.
2. After put manifest request, associate the blobs which are referenced
by the manifest but not associated with project.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-14 21:39:44 +08:00
Wenkai Yin
c6940e8184 United error response format for management APIs (legacy and v2.0 APIs) 
United error response format for management APIs (legacy and v2.0 APIs)

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-13 22:00:08 +08:00
AllForNothing
05431a149d Swith to new API for recent log page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-13 19:56:44 +08:00
Wenkai Yin(尹文开)
4a97cd270d
Merge pull request #11038 from ywk253100/200312_upgrade 
Repair the count usage during the upgrading
 2020-03-13 16:19:12 +08:00
He Weiwei
37e6fa5c92
fix(transaction): change to use value in the ctx to decide whether commit tx (#11062) 
Type assertion not work when the ctx in the request changed in the next
handler, so change to use value in the ctx to decide whether to commit
tx.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-13 15:19:13 +08:00
Wenkai Yin
05255a7ea7 Check expired or not when getting token from cache 
Check expired or not when getting token from cache

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-13 14:57:30 +08:00
Wenkai Yin
a4a1913598 Repair the count usage during the upgrading 
As the count quota is against artifact rather than tag in 2.0, the count usage should be recalculated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-13 13:59:48 +08:00
Yogi_Wang
b32a8db114 [OCI] Add the icon of openpolicyagent artifact 
1. add image
2. fix show ui clearly when dark
3. fix chinese words of replication name filter tooltip
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-13 12:01:00 +08:00
Daniel Jiang
2e7eb8872e
Move ArtifactInfo to internal package (#11055) 
To avoid depedency loop, this commit moves the model of ArtifactInfo to
internal pacakge, so that a controller can it from context when needed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-13 11:16:13 +08:00
He Weiwei
2a243ef7a2
refactor(rbac): refactor rbac impl to improve performance (#9988) 
1. Introduce `Evaluator` interface which do the permission checking.
2. `admin`, `lazy`, `rbac`, `namespace` and `evaluartor` set are implemented the
`Evaluator` interface.
3. Move project rbac implemention from `project` to `rbac` pkg to reduce
the name  conflict with project instance of model.
4. Do permission checking in security context by `Evaluator`.
5. Cache the regexp in rbac evaluator for casbin.
6. Cache evaluator in namespace evaluator to improve performance.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-12 23:42:53 +08:00
Ziming Zhang
8ffa79801b feature(tag_retention) add checkbox for user to control whether remove untagged image 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-12 22:50:21 +08:00
He Weiwei
12f16c8cec
feat(scan): support to scan image index (#11001) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-12 19:30:12 +08:00
Wenkai Yin
289f04d301 Restructure the packages of artifact 
1. Introduce a new interface Processor to replace Abstractor and Descriptor
2. Provide the base processors for manifest and index to reduce the duplicate code
3. Move the child artifacts checking out of processor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-12 16:03:13 +08:00
Ziming Zhang
25b5c3796b enhance(replication) update healthy status immediately 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-12 14:55:50 +08:00
Steven Zou
b546d9167a
Merge pull request #11019 from bitsf/replication_adapter_sort 
feat(replication) sort the adapters shown on UI
 2020-03-12 14:53:34 +08:00
Ziming
b597d9d59a
feat(ci) enhance govet check performance (#11008) 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-12 11:03:24 +08:00
Wenkai Yin(尹文开)
dcaccbc757
Merge pull request #10945 from ywk253100/200305_swagger_model 
Remove the "x-go-type" for artifact definition in swagger
 2020-03-12 10:47:00 +08:00
Wenkai Yin
4ccc3da99b Remove the "x-go-type" for artifact definition in swagger 
Using "x-go-type" may cause the inconsistence between the swagger definition and the real data model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-12 10:06:22 +08:00
Will Sun
1831aeb45c
Merge pull request #11023 from AllForNothing/robot 
Improve UI for add robot page
 2020-03-12 09:46:20 +08:00
Will Sun
878c004d9e
Merge pull request #11025 from jwangyangls/upgrade-clr 
[feat] Upgrade clarity to 2.3.8
 2020-03-12 09:45:42 +08:00
He Weiwei
89dfe24f19
feat(quota): add Request and Refresh middlewares for APIs (#10907) 
1. Introduce ReqquestMiddleware and RefereshMiddleware.
2. Add request middlware to copy artifact, mount blob, put blob upload,
put manifest, upload chart verson APIs.
3. Add refresh project middleware to delete manifest, delete artifact,
delete chart version, delete repository APIs.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-11 22:25:40 +08:00
Will Sun
aa73f16a20
Merge pull request #11027 from AllForNothing/webhook 
Improve Webhook UI
 2020-03-11 18:35:29 +08:00
Wenkai Yin(尹文开)
c2826d0368
Merge pull request #11030 from mmpei/webhook-dev-slack 
add support slack in webhook
 2020-03-11 18:20:58 +08:00
Ted Guan
4ac31c6d46
Add API for query supported event types and notify types; Return policy name in last trigger info; Remove project_id unique constraint in table notification_policy (#11029) 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2020-03-11 18:06:58 +08:00
Ziming Zhang
d1d0601841 feat(replication) sort the adapters shown on UI 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-11 18:06:29 +08:00
peimingming
3a6d1d75d0 add support slack in webhook 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2020-03-11 17:19:38 +08:00
AllForNothing
a19900e96e Improve webhook UI 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-11 16:51:21 +08:00
Wenkai Yin(尹文开)
8452100148
Merge pull request #10942 from ywk253100/200305_reference 
Persistent the URLs and annotations of artifact references in database
 2020-03-11 16:20:18 +08:00
Will Sun
fd3997678b
Merge pull request #10993 from AllForNothing/gc-ui 
Add new parameter for GC page
 2020-03-11 15:49:15 +08:00
AllForNothing
2fdb01ef1a Improve UI for add robot page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-11 15:42:33 +08:00
Yogi_Wang
dccf125016 [feat] Upgrade clarity to 2.3.8 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-11 15:41:33 +08:00
Wenkai Yin(尹文开)
d644d23b25
Merge pull request #10370 from kofj/fix/aliacr 
FIX: AliACR Provider.
 2020-03-11 15:36:34 +08:00
Ziming Zhang
7d53a61a92 feat(replication) sort the adapters shown on UI 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-11 15:03:10 +08:00
Wenkai Yin(尹文开)
b02cab434f
Fire event when create/delete resources (#11010) 
1. Create/delete project
2. Create/delete repository
3. Push/pull/delete artifact
4. Create/delete tag

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-11 14:39:01 +08:00
He Weiwei
41edfaf3a6
fix(api): escape path paramters before APIs and unescape them in the Prepare of operations (#11013) 
1. Escape the path paramters before the APIs.
2. Unescape the path paramters in the Prepare stage of the swagger
operations.

Closes #10860

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-11 12:18:40 +08:00
stonezdj(Daojun Zhang)
c7fd3bdfc5
Refactor event model (#10876) 
Move src/pkg/notification/model/const.go to src/pkg/notifier/model/const.go
Add auditlog handler to log project event, repo event, artifact event and tag event.

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-03-11 11:51:28 +08:00
Yogi_Wang
fe047a2ad3 [OCI] Artifact filter params changes in ui 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-11 11:21:08 +08:00
Ziming Zhang
5622a20058 feat(pkg) move artifactselector to src/internal 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-10 17:01:25 +08:00
Steven Zou
b2434945ff fix[js_ut]:update stop job case of js 
remove the status checking loop

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-03-10 15:54:10 +08:00
Ziming
890200ea19
feature(tag_retention) add webhook for deleted artifacts (#10982) 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-10 10:46:58 +08:00
Will Sun
cb370f8dd9
Merge pull request #10980 from AllForNothing/scan-bug 
Fix  a bug for scanning
 2020-03-10 09:51:10 +08:00
Wenkai Yin(尹文开)
307dbc6fba
Accept the pagination information in the separated query string (#10991) 
Accept the pagination information in the separated query string

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-09 22:19:42 +08:00
AllForNothing
de009f49fb Add new parameter for GC page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-09 17:51:00 +08:00
Yogi_Wang
a8a7975522 Csrf change to v2.0 in ui 
1.delete personal xsrf service
2.change to direactive get token
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-09 15:08:50 +08:00
Wenkai Yin(尹文开)
30896f3a10
Merge pull request #10968 from ywk253100/200306_artifact_query 
Add support for querying artifact by labels and tags
 2020-03-09 14:10:19 +08:00
Daniel Jiang
6d89553c4d
Merge pull request #10937 from reasonerjt/csrf-2.0 
Update CSRF mechanism
 2020-03-09 12:31:08 +08:00
Wang Yan
073d95b89f
add scanner pull check in policy checker middleware (#10971) 
Scanner uses the robot account to pull image and scan, the policy checker should bypass the
pull action even the policy enabled, otherwise the scan job will fail.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-09 12:11:54 +08:00
Daniel Jiang
7897fd752b
Merge pull request #10969 from reasonerjt/rm-travis 
Remove dependency on travis-ci
 2020-03-09 12:06:57 +08:00
Wenkai Yin(尹文开)
c0542776e0
Merge pull request #10961 from ywk253100/200306_replication 
Make replication work with new artifact(phase 2)
 2020-03-09 11:58:26 +08:00
Wenkai Yin(尹文开)
52c6d354d1
Merge pull request #10967 from ywk253100/200307_auth_header 
Only set "Www-Authenticate" header for registry API
 2020-03-09 11:58:09 +08:00
Wang Yan
7b8aca6cd2
Merge pull request #10973 from wy65701436/inlimited-robot 
add no expriation limited robot account
 2020-03-09 11:30:21 +08:00
Wang Yan
4c167b7a33
Merge pull request #10972 from wy65701436/conformance-fix 
fix Conformance testing failure
 2020-03-09 11:28:45 +08:00
Wenkai Yin
8bd632316c Only set "Www-Authenticate" header for registry API 
If "Www-Authenticate" header is set for Harbor management API, the browser will show a basic auth dialog when get 401 error, this commit moves the header to the registry APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-09 10:57:42 +08:00
AllForNothing
c5e7e51b60 Fix a bug for scanning 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-09 10:52:25 +08:00
wang yan
18bd2f162c fix Conformance testing failure 
1, Return DIGEST_INVALID error in delete manifest instead of NOT_FOUND
2, Disable return 500 in immutable middleware
3, Return empty array in catalog and tags API instead of null

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-09 10:29:10 +08:00
Wenkai Yin(尹文开)
e4bee937ff
Merge pull request #10970 from wy65701436/remove-regtoken 
remove middleware regtoken
 2020-03-09 09:41:46 +08:00
Daniel Jiang
0f0e27179b Remove dependency on travis-ci 
Github actions work fine, we no longer needs travi-ci to trigger the
tests.
This commit removes it.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-09 01:30:38 +08:00
Daniel Jiang
ae5ffce83a Update CSRF mechanism 
This commit replaces beego's CSRF mechanism with gorilla's csrf library.
The criteria for requests to skip the csrf check remain the same.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-09 01:15:54 +08:00
wang yan
b23111063d add no expriation limited robot account 
"-1" means the robot account is a permanent account, no expiration time set.
The ExpiresAt claim is optional, so if it's not set, it will still be considered a valid claim

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-08 16:47:40 +08:00
Wang Yan
e86d3a728c
Merge pull request #10899 from steven-zou/fix/failure_js_ut_cases 
fix[js]:fix ut case faulure
 2020-03-07 19:10:02 +08:00
wang yan
ddc0f83ccd remove middleware regtoken 
Remove it since we don's use bearer token as the registry token and the skipper of scanner pull will
be covered in the robot account access scope.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-07 19:06:30 +08:00
Wenkai Yin
b14762ee17 Add support for querying artifact by labels and tags 
Add support for querying artifact by labels and tags

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-07 11:32:15 +08:00
Wenkai Yin
e237a686c4 Make replication work with new artifact(phase 2) 
Provide the resource type filter for users to choose when replicating from harbor to other registries

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-06 18:26:37 +08:00
Wang Yan
50e9d1a56e
Merge pull request #10951 from wy65701436/readonly-filter 
skip configuration update in readonly mode
 2020-03-06 16:22:14 +08:00
Wang Yan
ea45fee3fe
Merge pull request #10954 from wy65701436/fix-gc-job 
Fix gc issue on clean the artifact trash
 2020-03-06 14:47:05 +08:00
Wenkai Yin(尹文开)
63cf1041f7
Merge pull request #10941 from ywk253100/200304_query_label 
Implement query string builder
 2020-03-06 13:18:10 +08:00
wang yan
2b0b7576b2 Fix gc issue on clean the artifact trash 
1, enable dao test for artifact trash
2, set default flush trash table to false
3, hanlder empty parameter in API call
4, add registry auth info into jobservice container

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-06 03:11:31 +08:00
wang yan
8b4211717e skip configuration update in readonly mode 
Admin must have a way to switch off the readonly by call configuration api,
either internal or external.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-06 00:34:26 +08:00
He Weiwei
c8ca6a5ccf
Remove the readonly filter (#10944) 
Remove the readonly filter as we have introduced readonly middleware

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-05 18:59:35 +08:00
He Weiwei
d21318dfcf
Use project controller rather than the manager in API handlers (#10946) 
Use project controller rather than the manager in API handlers

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-05 18:58:47 +08:00
Wenkai Yin(尹文开)
84bd30c570
Merge pull request #10923 from ywk253100/200301_replication 
Make sure replication work with OCI artifacts(phase 1)
 2020-03-05 17:48:56 +08:00
Wenkai Yin
8abb630b4c Implement query string builder 
This commit defines the API query string format and provides the builders to build query string to query model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-05 17:46:11 +08:00
stonezdj(Daojun Zhang)
49619e1907
Merge pull request #10939 from wy65701436/access-log-mgr 
add audit logs API
 2020-03-05 16:24:21 +08:00
AllForNothing
5b440082dc Add new status "scan unsupported" for artifact list 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-05 15:33:28 +08:00
Wang Yan
e79f4fd270
Merge pull request #10908 from wy65701436/middleware-blocker 
add delete manifest middleware
 2020-03-05 12:00:43 +08:00
wang yan
df237a5b17 add audit logs API 
1, add API entry for get audit logs
2. add audit log manager to hanlder CRUD

Use the new format of audit log to cover differernt resource, artifact/tag/repostory/project

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-05 11:40:51 +08:00
Wenkai Yin
76c04b0219 Persistent the URLs and annotations of artifact references in database 
Persistent the URLs and annotations of artifact references in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-05 10:54:45 +08:00
Will Sun
a5d9a3b65d
Merge pull request #10863 from AllForNothing/api-center 
Fix Api cennter
 2020-03-05 10:00:15 +08:00
wang yan
3bb574db35 use delete manifest to handle immutable and signature 
1, Use signature manager to get signature
2, Check the immutable and signature status when deleting.
3, Remove the immutable middleware for delelte manifest

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-04 18:03:34 +08:00
AllForNothing
f46a61e522 Improve scan funciton 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-04 16:09:27 +08:00
jwangyangls
3a6b675dbd
Merge pull request #10887 from jwangyangls/filter-artifact-mutiple 
Filter artifact by type/tag  and change error setting
 2020-03-04 11:50:30 +08:00
Wenkai Yin
e45eaeec74 Fix transaction issue 
More detail: // https://www.postgresql.org/message-id/002e01c04da9%24a8f95c20%2425efe6c1%40lasting.ro

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-04 08:01:11 +08:00
Wenkai Yin
d4ba023457 Make sure replication work with OCI artifacts(phase 1) 
This commit updates the definition of replicated resource(artifacts replace the vtags) and refactor the filter part

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-03 17:50:51 +08:00
He Weiwei
69119b6410
feat(addition-link): only set vuls addition link when artifact scanable (#10892) 
1. Add Checker to check the scannable status of the artifact.
2. Only set vulnerabilities addition link when the artifact scanable in the
project.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-03 16:41:54 +08:00
Yogi_Wang
4d3aa26853 Filter artifact by type/tag and change error setting 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-03 16:16:17 +08:00
jwangyangls
0d45308fbc
Merge pull request #10902 from jwangyangls/fix-xsrf-422 
Fix xsrf error status bug when after beego update
 2020-03-03 15:38:07 +08:00
Ziming
70dcca6579
Merge pull request #10857 from bitsf/remove_pkg_art 
feat(oci) remove dead code pkg/art
 2020-03-03 11:50:15 +08:00
Steven Zou
bd0e401cae fix[js]:fix ut case faulure 
- refactor default context creation to avoid data race
- refactor the timer interval in c_worker UT cases to avoid receieving signals at the same time

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-03-03 11:47:41 +08:00
Wang Yan
54227f1ba2 update chart sdk to support helm v3 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-02 11:39:09 +08:00
Ziming Zhang
aee2c672e7 feat(oci) remove pkg/art dead code for OCI 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-02 10:34:34 +08:00
Yogi_Wang
2bbb37e6b5 Fix xsrf error status bug when after beego update 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-02 10:10:01 +08:00
Wenkai Yin(尹文开)
8de3fab3c5
Merge pull request #10841 from ywk253100/200223_upgrade 
Migrate artifact data in 2.0
 2020-02-28 18:36:40 +08:00
Wenkai Yin(尹文开)
bd0a8e9f8f
Merge pull request #10864 from ywk253100/200227_response 
Set the "Link" and "location" header
 2020-02-28 18:25:02 +08:00
Wenkai Yin
4c9b59c904 Migrate artifact data in 2.0 
Abstract extra attributes and annotations for artifacts stored in database

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-28 18:09:02 +08:00
He Weiwei
0f5a115a65
feat(artifact): add Walk method to artifact controller (#10881) 
1. Add Walk method to artifact controller.
2. Only query references when artifact is image index.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-02-28 17:19:36 +08:00
Wenkai Yin
e3bbcb66d1 Set the "Link" and "location" header 
Set the "Link" and "location" header

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-28 17:09:18 +08:00
Wenkai Yin(尹文开)
3d336bfac3
Merge pull request #10882 from wy65701436/tag-controller 
add tag controller
 2020-02-28 16:39:58 +08:00
wang yan
2d4fc0c4da move out the tags methods of artifact ctl 
1, move the tag methods out of artifact ctl, let api to call tag ctr
2, update the ensure sequence for existing tag

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-28 15:49:39 +08:00
Wang Yan
48163f2666
Merge pull request #10879 from reasonerjt/oidc-filter-update-v2 
CLI Secret should handle /v2/* API
 2020-02-28 11:58:58 +08:00
wang yan
79cf21f82f add tag controller 
use the tag controller to handle CRUD of tags, especially the delete scenario, it could validate
the immutable and signature. And move the code of tag handling from artifact controller to tag controller

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-28 11:42:10 +08:00
Daniel Jiang
de9cd1f964 CLI Secret should handle /v2/* API 
As we swtich to basic auth for /v2/* API
The CLI secret should handle /v2/* API so that OIDC user can use the
secret to do push/pull
This commit makes such change.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-02-28 02:07:10 +08:00
Wenkai Yin
40890d2635 Add "Docker-Distribution-Api-Version" header for the 401 response of registry API 
This is needed for "docker manifest" commands: https://github.com/docker/cli/issues/989

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-27 17:32:14 +08:00
AllForNothing
d41c5496a2 Fix Api cennter 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-02-27 15:55:20 +08:00
Wenkai Yin(尹文开)
e3f73a3efd
Merge pull request #10792 from ninjadq/fix_chart_api_for_v2_0 
Fix URL issue introduced by api version
 2020-02-27 08:01:50 +08:00
AllForNothing
a8f9de7a7f Fix scanning function 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-02-26 18:14:35 +08:00
DQ
bc4c25181f Fix chart api for oci registry introduece api version 
currently api version part is added in url. This pr is to solve the break of chart related api

Signed-off-by: DQ <dengq@vmware.com>
 2020-02-26 17:05:02 +08:00
DQ
ff0c8b382c Refactor the version to variable 
Signed-off-by: DQ <dengq@vmware.com>
 2020-02-26 16:24:49 +08:00
Yogi_Wang
f1ed010d9c Improve artifact 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-02-26 15:23:57 +08:00
Wang Yan
e9d09c705e
Merge pull request #10844 from ywk253100/200225_artifact 
Use the repository name of artifact model
 2020-02-26 14:29:45 +08:00
Wenkai Yin
02c2647e1e Use the repository name of artifact model 
As we store the repository name in the artifact table, we can use it direclty in the code to reduce the database query

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-26 13:37:09 +08:00
Yogi_Wang
7dc27ab4eb Fix some detail function 
1.change common property to Extra Attributes;
2.fix delete repo bug
3.disable index artifact action(nothing changed when refactoring the route completely)
4.annotations show in artifact list
5.add validation in  add tag
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-02-26 12:51:42 +08:00
AllForNothing
41dd5656e6 Upgrade api level to v2 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-02-26 10:13:34 +08:00
Wenkai Yin(尹文开)
b156c74f6a
Merge pull request #10831 from wy65701436/readonly-global 
add readonly to beego middleware
 2020-02-25 16:10:55 +08:00
wang yan
b336875ebf add readonly to beego middleware 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-25 15:00:39 +08:00
Wenkai Yin(尹文开)
5f9c976e95
Merge pull request #10837 from heww/scan-api 
feat(scan): add scan API in v2.0
 2020-02-25 14:45:19 +08:00
Wenkai Yin(尹文开)
90d1c9f287
Merge pull request #10834 from ywk253100/200225_repo 
Implement the API to get the specified repository
 2020-02-25 14:39:52 +08:00
He Weiwei
55a21cd444 feat(scan): add scan API in v2.0 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-02-25 06:01:37 +00:00
jwangyangls
3174734473
Merge pull request #10819 from jwangyangls/add-ut-oci 
Add copy artifact and update repo info and add ut
 2020-02-25 12:43:04 +08:00
Wenkai Yin
bb3ff0d752 Implement the API to get the specified repository 
Implement the API to get the specified repository

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-25 12:02:50 +08:00
Yogi_Wang
92f84f2aee Add copy artifact and update repo info and add ut 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-02-25 10:39:03 +08:00
Wenkai Yin(尹文开)
bf4d141a77
Merge pull request #10827 from reasonerjt/refresh-notary-test-data 
Refresh notary test data
 2020-02-25 08:08:27 +08:00
Daniel Jiang
340726f7d3 Refresh notary test data 
Refresh the valid signature data before it's expired

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-02-24 19:46:17 +08:00
Wang Yan
948d45604c Revise the GC job flow, 
1, set harbor to readonly
2, select the candidate artifacts from Harbor DB.
3, call registry API(--delete-untagged=false) to delete manifest bases on the results of #2
4, clean keys of redis DB of registry, clean artifact trash and untagged from DB.
5, roll back readonly.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-24 18:29:55 +08:00
Wenkai Yin
528f598268 Reimplement the registry client 
This commit reimplements the registry client under directory src/pkg/registry and removes the useless code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-24 14:36:26 +08:00
stonezdj(Daojun Zhang)
c2a77c2825
Merge pull request #10751 from ywk253100/200213_delete_useless 
Remove dead code
 2020-02-24 10:22:58 +08:00
Will Sun
ca871d0eb5
Merge pull request #10790 from AllForNothing/routing-modify 
Improve routing and UI for artifact pages
 2020-02-24 09:39:36 +08:00
Wenkai Yin(尹文开)
c446774d23
Merge pull request #10816 from reasonerjt/merge-art-mani-middleware 
Merge artifactInfo and ManifestInfo
 2020-02-24 08:10:32 +08:00
Daniel Jiang
46c72ae372 Merge artifactInfo and ManifestInfo 
This commit gets rid of middleware info middleware, and make artifact
info the single source of truth in terms of the artifact a request
handles.  Fixes #10574

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-02-23 18:07:19 +08:00
Wenkai Yin
bd204464f3 Remove dead code 
Remove dead code

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-23 17:11:46 +08:00
Wenkai Yin
ab3aad4d50 Implement the resolver for CNAB 
This commit introduces a new resolver to resolver metadata for CNAB

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-23 07:23:22 +08:00
He Weiwei
f36152a560
feat(vulnerability): assemble vulnerabilities info for artifact (#10800) 
1. Assemble scan overview to artifact when scanner enabled in the
project of the artifact.
2. Set addition link for vulnerabilities to artifact when scanner
enabled in the project of the artifact.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-02-22 13:29:58 +08:00
stonezdj
29aa59ff18 Move core/notifier to pkg/notifier 
Update package reference in related components

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-02-22 11:05:27 +08:00
Wenkai Yin(尹文开)
f7d248b968
Merge pull request #10789 from ywk253100/200203_upgrade 
Upgrade the artifact table
 2020-02-21 21:24:34 +08:00
Wenkai Yin
9312b788dc Upgrade the artifact table 
Split the table artifact into artifact and tags, and populate related data

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-21 20:37:31 +08:00
He Weiwei
d1cef70cd1
refactor(scan,auth): remove bearer token auth support for scan job (#10781) 
Harbor v2.0 has changed to use basic auth to pull image so remove bearer
token support for scan job

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-02-21 17:58:24 +08:00
AllForNothing
8bff170c89 Improve routing and UI for artifact pages 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-02-21 15:00:49 +08:00
He Weiwei
88fcacd4b7
feat(middleware): add blob middlewares (#10710) 
1. Add middleware to record the accepted blob size for stream blob
upload.
2. Add middleware to create blob and associate it with project after blob upload
complete.
3. Add middleware to sync blobs, create blob for manifest and associate blobs
with the manifest after put manifest.
4. Add middleware to associate blob with project after mount blob.
5. Cleanup associations for the project when artifact deleted.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-02-20 23:20:34 +08:00
Ziming
0bc32410f3
Merge pull request #10742 from bitsf/oci_tag_retention 
requirement(oci) implement tag retention for oci
not include ChartClient yet
 2020-02-20 20:31:49 +08:00
Wenkai Yin(尹文开)
86b3e47f81
Merge pull request #10733 from ywk253100/200213_copy_artifact 
Implement copy artifact API
 2020-02-20 17:31:26 +08:00
Wenkai Yin
c4d4850845 Implement copy artifact API 
Copy artifact into the repository from the specified artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-20 16:59:35 +08:00
Yogi_Wang
99d02a14f5 Fix issue from louis and improve artifact list 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-02-20 10:15:33 +08:00
Wenkai Yin
1db0077096 Implement delete/update repository API 
Implement delete/update repository API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-20 08:39:26 +08:00
Ziming Zhang
94e23dc954 requirement(oci) implement tag retention for oci 
Change-Id: Ib36660835d2666b35124e66254c33b5fc19aaf77
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-02-20 00:43:20 +08:00
Wang Yan
22021a988e
Merge pull request #10753 from wy65701436/artifact-trash 
add artifact trash manager
 2020-02-19 19:14:48 +08:00
Steven Zou
f1374737f6
Merge pull request #10694 from danielpacak/feature/install_with_trivy 
chore(install): Add --with-trivy arg to the installation script
 2020-02-19 16:27:57 +08:00
Wang Yan
f160505686 add artifact trash manager 
1, move the deleted artifact into trash
2, disable GC to delete the untagged manifest

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-19 14:52:58 +08:00
He Weiwei
4ad02de348
Scan reorganize (#10735) 
* refactor(scan,scanner): move scan and scanner controllers to api pkg

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* feat(scan-all-job): move artifacts query from job to notification

Move artifact query from scan all job to its notification handler to
ensure that the components in pkg will not call controllers in api.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-02-19 12:21:54 +08:00
Daniel Pacak
1fbc3dcb39 refactor: Allow EnsureScanners to accept multiple scanner registrations 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-18 17:31:26 +01:00
Daniel Pacak
93f05b3643 refactor(scanners): Allow RemoveImmutableScanners() to accept multiple endpoint URLs 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-18 12:47:27 +01:00
Wenkai Yin
9d2f1d4d66 Refactor the logic of deleting artifact 
Delete the child artifacts along with the parent when deleting an artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-18 16:20:17 +08:00
Wenkai Yin
c8c944c6e8 Move the scan overview populating logic to API handler 
Move the scan overview populating logic to API handler to avoid importing cycle

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-18 13:34:08 +08:00
Wenkai Yin(尹文开)
560dd8ce7b
Merge pull request #10745 from reasonerjt/artifact-signature-populate-v2 
Artifact signature populate
 2020-02-18 13:24:55 +08:00
jwangyangls
360e342d56
Merge pull request #10719 from jwangyangls/oci-master-1 
Oci ui include artifact list and artifact summary
 2020-02-18 13:04:59 +08:00
Yogi_Wang
2553ee3831 Oci ui include artifact list and artifact summary 
Signed-off-by: Yogi_Wang <yawang@vmware.com>

Signed-off-by: AllForNothing <sshijun@vmware.com>
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-02-18 12:37:03 +08:00
Wenkai Yin(尹文开)
341cb88cba
Merge pull request #10727 from ywk253100/200214_bump_up_legacy_api_version 
Bump up legacy api version to v2.0
 2020-02-18 10:37:25 +08:00
Daniel Jiang
5a6e9331fd
Artifact signature populate (#7) 
* Populate signature status in artifact API

This Commit add signature status into response of list artifact API.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-02-18 01:42:56 +08:00
Wang Yan
e5eb711827
Merge pull request #10720 from wy65701436/chart-details 
Get addition properties for chart
 2020-02-17 13:36:24 +08:00
wang yan
8029f70ae5 Get addition properties for chart 
1, Get readme.md content into addition
2, Get dependency of chart
3, Get values of chart

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-17 12:19:10 +08:00
Wenkai Yin(尹文开)
d7903fcf1b
Merge pull request #10682 from ywk253100/200126_label 
Support add/remove label to/from artifact
 2020-02-17 11:33:45 +08:00
Wenkai Yin
eceb9b2345 Fix bugs when pushing image(with index) and CNAB 
1. As "List" method of artifact DAO doesn't return the artifacts that referenced by other and without tag, so we introduce a new method "GetByDigest" to check the existence of artifact
2. The "Www-Authenticate" header is needed to be returned when the request is unauthorized. This is required in the OCI distribution spec and is needed when pushing CNAB

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-14 15:24:14 +08:00
AllForNothing
121314358a Switch APIs to v2.0 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-02-14 13:16:31 +08:00
Wenkai Yin
94787ea60d Bump up the version of legacy APIs to v2.0 
Bump up the version of legacy APIs to v2.0

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-14 13:16:30 +08:00
Daniel Pacak
fdf70f5dc9 refactor: Use if/else statements 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-13 16:26:06 +01:00
Daniel Pacak
5f110c800a feat: Initialize scanner registrations properly 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-13 14:02:04 +01:00
Wenkai Yin
93731eeb2e Support add/remove label to/from artifact 
This commit add supporting for adding/removing label to/from artifacts and populates labels when listing artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-13 10:46:23 +08:00
Daniel Pacak
a642667ffc chore(install): Add --with-trivy arg to the installation script 
Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-02-12 23:47:56 +01:00
He Weiwei
75bbf9d0bf
fix(middleware): escape and clean path for the skipper (#10674) 
1. Escape and clean request path for `legacyAPISkipper`.
2. Escape and clean request path for `MethodAndPathSkipper`.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-02-11 22:29:55 +08:00
Wenkai Yin(尹文开)
d66c1a4a21
Merge pull request #10612 from ywk253100/200202_replication_basic_auth 
Do enhancement for the registry authorizer
 2020-02-11 22:09:40 +08:00
Wenkai Yin
a4ebbc6ecf Do enhancement for the registry authorizer 
This commit introduces a new wrapper authorizer which can authorize the request according to the auth scheme automatically

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-11 21:21:49 +08:00
Will Sun
647f11f26b
Merge pull request #10688 from AllForNothing/relative 
UI should use relative path for back-end APIs
 2020-02-11 18:06:42 +08:00
Daniel Jiang
9fb676c219 Fix nilpointer issue in v2 auth middleware 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-02-11 15:49:39 +08:00
Wang Yan
582fac5dae
Merge pull request #10687 from wy65701436/fix-catalog 
fix catalog pagination issue
 2020-02-11 10:40:35 +08:00
He Weiwei
c4f77069c8
Hide projects in global logs where user has limited guest role (#10639) 
Signed-off-by: Mark Huang <mhuang@pivotal.io>

Co-authored-by: Mark Huang <mhuang@pivotal.io>
 2020-02-11 10:14:30 +08:00
Wang Yan
e68b474dbc Merge pull request #10581 from ywk253100/100123_list_artifact 
Add query string to listing artifact API to support specify only show the tagged artifacts
 2020-02-10 18:12:06 +08:00
wang yan
beb7664b33 fix catalog pagenation issue 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-10 18:03:33 +08:00
Wang Yan
65dc54b059 Merge pull request #10626 from ywk253100/200125_handle_error 
Unify the method/style to handle error in handler/middleware
 2020-02-10 17:47:12 +08:00
AllForNothing
c932ca9d2b UI should use relative path for back-end APIs 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-02-10 15:37:09 +08:00
Wenkai Yin
0373e08e58 Add query string to listing artifact API to support specify only show the tagged artifacts 
Specify the query string "tags=NOT_NULL" to the listing artifact API to only return the tagged artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-10 14:53:51 +08:00
Wang Yan
71414a9bc3 Merge pull request #10670 from wy65701436/upgrade-beego-12.1 
update beego version to v1.12.1
 2020-02-10 13:57:07 +08:00
wy65701436
b2e3761f62 update beego version to v1.12.1 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-09 16:39:48 +08:00
Wenkai Yin
af4dd142bc Unify the method/style to handle error in handler/middleware 
This commit provides a "SendError" method to unify the way to handle error in handlers/middlewares

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-08 07:37:00 +08:00
Wenkai Yin
0f6057a22c Implement get addition API for image 
This commit implements the API to get build history of image with manifest version 2 and populates the addition links when listing/getting the artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-07 20:21:38 +08:00
Wenkai Yin(尹文开)
8a74fcb074
Merge pull request #10652 from wy65701436/fix-10552 
Use controller rather than manager in the API handler and middleware
 2020-02-07 19:30:51 +08:00
Wang Yan
6bad9f62ba
Merge pull request #10662 from ywk253100/200206_delete_repo 
Implement repository deletion API
 2020-02-07 17:51:03 +08:00
Wang Yan
5679c174c6
Merge pull request #10643 from ywk253100/200204_auth 
Add permission check for artifact related APIs
 2020-02-07 11:25:37 +08:00
wang yan
0fbbd674c2 Use controller rather than manager in the API handler and middleware 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-07 11:23:17 +08:00
Wenkai Yin
c267aaa474 Implement repository deletion API 
Implement repository deletion API based on the new design

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-07 09:36:35 +08:00
Wang Yan
72f9e02dd9
Merge pull request #10628 from wy65701436/af-immu 
set the immutable status on getting/listting tag
 2020-02-05 21:03:29 +08:00
wang yan
596a6261ca set the immutable status on getting/listting tag 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-05 16:24:13 +08:00
Wenkai Yin
6087647895 Add permission check for artifact related APIs 
Add permission check for artifact related APIs

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-05 15:58:39 +08:00
Wenkai Yin(尹文开)
260d43db4f
Merge pull request #10627 from ywk253100/200202_reference 
Add child artifact digest in reference model
 2020-02-05 13:31:12 +08:00
wang yan
a53df4863d fix issue on listing robot accounts 
Update the query FuzzyMatch when to list robot accounts per project

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-05 11:36:18 +08:00
Wenkai Yin
7930e4f38a Add child artifact digest in reference model 
As we only provide the API to get artifact information via project name, repository name and digest, the digest of child artifact must be returned

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-04 15:59:06 +08:00
Wenkai Yin
c1746cc675 Update the method called by registry handler 
Use ListTags instead in the registry tag listing handler

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-04 15:16:28 +08:00
Wang Yan
fa41168e8e
Merge pull request #10587 from ywk253100/200122_route 
Register registry handler with the new methods of Route
 2020-02-04 14:28:15 +08:00
Wenkai Yin(尹文开)
26ade207d3
Merge pull request #10588 from ywk253100/200126_delete_get_artifact 
Implement artifact/tag related API
 2020-02-04 14:14:28 +08:00
Wenkai Yin
793b23a444 Implement artifact/tag related API 
Implement APIs:
1. Get artifact
2. Delete artifact
3. Create tag
4. Delete tag

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-04 13:46:55 +08:00
He Weiwei
791439086d
feat(api,permission): add HasPermission, HasProjectPermission in BaseAPI (#10618) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-02-04 08:42:00 +08:00
He Weiwei
b1437c1341
refactor(security): add NewContext and FromContext to security pkg (#10617) 
1. Add `NewContext` and `FromContext` funcs in security pkg.
2. Add `Name` func in `security.Context` interface to make the checking
for the `/api/internal/configurations` API clear.
3. Get the security from the context to prepare change the security
filter to middleware.
4. Remove `GetSecurityContext` in filter pkg.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-02-03 17:43:36 +08:00
Wenkai Yin
ef3af85a5b Register registry handler with the new methods of Route 
Register registry handler with the new methods of Route

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-02-03 16:55:08 +08:00
Wang Yan
9c0d400817
Merge pull request #10549 from wy65701436/middleware-pc 
add Middleware policycheck into v2 handler
 2020-02-03 15:58:45 +08:00
Will Sun
acfcd2d175
Merge pull request #10489 from AllForNothing/postinstall 
Fix postinstall script in Docker
 2020-02-03 14:13:12 +08:00
wang yan
661f7a7902 resolve conflict with latest code 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-02-03 10:34:09 +08:00
Daniel Jiang
2064a1cd6d Switch to basic authentication for registry 
1. Add basic authorizer for registry which modify the request
to add basic authorization header to request based on configuration.
2. Set basic auth header for proxy when accessing registry
3. Switche the registry to use basic auth by default and use the basic
authorizer to access Harbor.
4. Make necessary change to test cases, particularly
"test_robot_account.py" and "docker_api.py", because the error is
changed after siwtched to basic auth from token auth.  #10604 is opened
to track the follow up work.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-01-31 21:46:47 +09:00
Wenkai Yin(尹文开)
a1b25e1fec
Switch to new registry API handlers (#10596) 
Switch to new registry API handlers

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-01-30 21:21:07 +08:00
Wenkai Yin
d0ac70d26c Implement the get/delete handler for registry API 
Implement the get/delete handler for registry API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-01-28 11:01:17 +08:00
Wang Yan
9db84f2880 add Middleware policycheck into v2 handler 
1, add content trust middleware in new v2 handler
2, add vulnable middlware in new v2 hanlder

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-01-28 00:24:58 +09:00
Daniel Jiang
5f8acc3896 Add middlewares for permission checking for v2 API 
When the registry shifts from token auth to basic auth, we'll use the middleware to check permission.
This commit add middlewares for populate the artifact info and check
permission based on request to /v2/* api via security context

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-01-27 12:53:15 +08:00
lxShaDoWxl
d09ca5ae6d Merge remote-tracking branch 'origin/master' into fix/gitlab 2020-01-24 21:19:32 +06:00
lxShaDoWxl
cc1938a04c tests(Registries/Gitlab): Added tests for gitlab adapter 
Signed-off-by: lxShaDoWxl <lxshadowxkingxl@gmail.com>
 2020-01-24 21:17:45 +06:00
He Weiwei
b594861658 feat(middleware): add transaction middleware for v2 and v2.0 APIs 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-01-22 05:00:39 +00:00
Wenkai Yin(尹文开)
f9d8951aed
Merge pull request #10519 from ywk253100/200116_router 
Wrap the beego router and provide a unified view for users to register routes
 2020-01-22 10:20:36 +08:00