Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-12-02 15:14:09 +01:00
Code Issues Projects Releases Wiki Activity
9,310 Commits 53 Branches 317 Tags 335 MiB
7c2bfb1378
Commit Graph

4128 Commits

Author SHA1 Message Date
Daniel Jiang
08f9ffa000 Reenable token auth for cli 
Docker CLI fails if it's not logged in upon seeing "basic" realm challenging while pinging the "/v2" endpoint. (#11266)
Some CLI will send HEAD to artifact endpoint before pushing (#11188)(#11271)

To fix such problems, this commit re-introduce the token auth flow to the CLIs.

For a HEAD request to "/v2/xxx" with no "Authoirzation" header, the v2_auth middleware populates the
"Www-Authenticate" header to redirect it to token endpoint with proper
requested scope.

It also adds security context to based on the content of the JWT which has the claims of the registry.
So a request from CLI carrying a token signed by the "/service/token" will have proper permissions.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-04 00:05:58 +08:00
danfengliu
616c2d9e0f
Merge pull request #11408 from jwangyangls/nightly-case-trivy-3 
[Test Case] Add nightly case for CVE
 2020-04-03 19:23:26 +08:00
Yogi_Wang
2610fe530f [Test Case] Add nightly case for CVE 
1. add nightly case for cve
2. change translate words
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-03 16:59:45 +08:00
He Weiwei
e9543a1e3c
Fix merge scan summary (#11392) 
* fix(scan): fix ScanStatus when merge NativeReportSummary

1. Running and success status is high priority when merge ScanStatus of
NativeReportSummary, otherwise chose the bigger status.
2. Merge scan logs of referenced artifacts when get the scan logs of
image index.

Closes #11265

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* fix(portal): fix the annotation for the scan completed percent in scan overview

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-03 16:21:36 +08:00
Wenkai Yin(尹文开)
2783fd0950
Merge pull request #11276 from mmpei/offical-webhook-slack-fix 
fix webhook slack test error
 2020-04-03 10:37:21 +08:00
He Weiwei
c0246e2130
Merge pull request #11400 from heww/fix-issue-11391 
fix(log): correct file and line when use logger
 2020-04-02 22:23:15 +08:00
peimingming
5924658092 fix webhook slack test error 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2020-04-02 20:02:27 +08:00
He Weiwei
207463e91e fix(log): correct file and line when use logger 
1. When use the helper functions of log pkg, the depth is 4 to get the
correct file and line.
2. Whe use the default logger of log pkg, the depth is 3 to get the
correct file and line.

Closes #11391

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-02 10:49:38 +00:00
Will Sun
2c3552904d
Merge pull request #11393 from AllForNothing/improve-webhook 
Improve webhook UI according to the UX
 2020-04-02 17:51:13 +08:00
wang yan
7104461716 fix artifact dao UT issue 
The update column should be PullTime instead of PushTime

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-02 14:45:43 +08:00
AllForNothing
ba5fd67b08 Improve webhook UI according to the UX 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-02 14:30:21 +08:00
wang yan
a11a70d941 move logger from common to lib 
The logger is the fundamental library, so move it into lib folder
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-02 14:09:03 +08:00
jwangyangls
4ea7b13215
Merge pull request #11385 from jwangyangls/nightly-case-trivy-2 
[Nightly] Project Level Image Serverity Policy
 2020-04-02 11:14:21 +08:00
Yogi_Wang
01f8291bb7 [Nightly] Project Level Image Serverity Policy 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-02 10:16:57 +08:00
danfengliu
f11e713ada
Merge pull request #11380 from jwangyangls/nightly-case-trivy-1 
[Nightly] add case about trivy
 2020-04-01 22:45:40 +08:00
Steven Zou
1f6301267c
Merge pull request #11369 from steven-zou/fix/issue_#11361 
fix[lua_scripts]:add default values for tonumber
 2020-04-01 19:01:02 +08:00
Qian Deng
b1284da96b
Merge pull request #11360 from ninjadq/rever_chart_api_change 
Rever chart api change
 2020-04-01 18:58:57 +08:00
Wenkai Yin(尹文开)
d187a8e69e
Merge pull request #11333 from ywk253100/200325_copy 
Update the existence checking logic when copying artifact
 2020-04-01 18:09:20 +08:00
Wang Yan
4594d58ba8
add clean untagged blobs in gc job (#11248) 
Fixes #11190, delete all of non-referenced blobs of each project in GC job, thun the quota
can be released.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-01 18:01:46 +08:00
Will Sun
b0d0b292cd
Merge pull request #11288 from AllForNothing/nightly-webhook 
Fix nightly cases for webhook
 2020-04-01 17:37:11 +08:00
Wenkai Yin(尹文开)
9f4f3be00d
Merge pull request #11364 from ywk253100/200331_replication 
Some tiny improvement for replication
 2020-04-01 17:29:08 +08:00
Wenkai Yin
e4d42deb75 Make sure the tag filter have the same behavior for empty value and * 
Fixes #11233, make sure the tag filter have the same behavior for empty value and *

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-01 17:15:23 +08:00
Yogi_Wang
24b57715ab [Night] add case about trivy 
`
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-01 17:06:34 +08:00
AllForNothing
e6e3f0a6af Fix nightly cases for webhook 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-01 16:27:40 +08:00
DQ
6216073d2a Add ui change 
using api/chartreport for ui

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 11:55:22 +08:00
DQ
9ff7d46e8f Rever chart version to original 
Because chart version should consistent with previous version

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 11:55:22 +08:00
Wang Yan
f6c0608e22
fix GC jobs upgrade issue (#11365) 
Fixes #11313
Fixes #11275

1, Add more details log in GC job
2, Add type assertion for the upgrading case, the delete_untagged parameter is introduced from v2.0
3, Add UT

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-01 11:53:41 +08:00
Wenkai Yin(尹文开)
c2c9fa28eb
Merge pull request #11368 from heww/fix-ongoing-of-metrics 
fix(scan): ongoing is true for schedule scan all only when job is running
 2020-03-31 20:10:40 +08:00
He Weiwei
1a7cad3a14
Merge pull request #11370 from heww/fix-issue-11198 
fix(scan): add scanner name as prefix for name of the robot when submit scan job
 2020-03-31 19:23:28 +08:00
Steven Zou
b5fceae734 fix[lua_scripts]:add default values for tonumber 
- add default values for the integer vars converted by tonumber()

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-03-31 18:20:52 +08:00
Wang Yan
d6261d9456
Does not throw err in the notification job (#11363) 
Fixes #11280, no error return but just log.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-31 18:12:12 +08:00
He Weiwei
120be36fec fix(scan): ongoing is true for schedule scan all only when job is running 
Closes #11289

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-31 10:08:22 +00:00
AllForNothing
0275108cb2 Fix bugs for round 1 testing 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-31 16:21:37 +08:00
jwangyangls
ceded08507
Merge pull request #11362 from jwangyangls/refact-artifact-tag 
[OCI] Refact artifact tag
 2020-03-31 15:46:45 +08:00
Yogi_Wang
a6e986df62 [OCI] Refact artifact tag 
1.get artifact tag from another api
2.add refresh button  in artifact tag
3.fix permission change
4.some ui style
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-31 14:57:39 +08:00
Wenkai Yin
d9a5c71289 Some tiny improvement for replication 
1. Add timeout when transter artifacts
2. Check 404 error when unschedule the policy
3. Add line to mark the job failure in job log

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-31 14:50:37 +08:00
Wang Yan
34d05dae58
fix content trust middleware bypass scanner pull (#11321) 
Fixes #11206
1, fix middleware doesn't work for docker pull without auth
2, fix middleware doesn't bypass scanner pull

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-31 11:12:21 +08:00
jwangyangls
3c6f5cce54
Merge pull request #11352 from jwangyangls/fix-bug-2.0-2 
[Fix] some harbor 2.0 UI bug
 2020-03-31 11:02:06 +08:00
He Weiwei
86d446ce81
fix(log): change log level from warning to debug when unescape path params (#11359) 
Closes #11186

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-31 10:33:18 +08:00
Daniel Jiang
fdb82ae4fa
Merge pull request #11349 from reasonerjt/fix-10602 
Not checking for registry credentials in v2auth
 2020-03-31 10:26:33 +08:00
Daniel Jiang
37f9d650bd Not checking for registry credentials in v2auth 
That was added to support core process sending request to `/v2/xxx`.
It's no longer needed after reworking the flow.
This commit removes this.

Fixes #10602, as it's not a case we need to support for now.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-31 01:08:37 +08:00
He Weiwei
f4d96d85f8 fix(scan): add scanner name as prefix for name of the robot when submit scan job 
Closes #11198

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-30 16:52:04 +00:00
Steven Zou
3d6c65f53b
fix[UT]:improve the UT cases of enqueuer (#11358) 
- change the test cron spec
- use cretaed timer for timeout

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-03-31 00:09:20 +08:00
Steven Zou
f2beed577f
fix[logger]:update log ID validation logic (#11351) 
Signed-off-by: Steven Zou <szou@vmware.com>
 2020-03-31 00:08:55 +08:00
Yogi_Wang
661867240d [Fix] 2.0 UI bug 
1.fix #11312
2.fix #11235
3.fix #11230
4.fix #11209
5.fix #11199
6.fix #11034
7.fix #9926
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-30 17:49:42 +08:00
Wenkai Yin(尹文开)
4faff18b2d
Merge pull request #11339 from ywk253100/200328_limit_offset 
Add "order by" clause to avoid the duplicat rows
 2020-03-30 17:14:44 +08:00
Wenkai Yin
fb975d902c Add "order by" clause to avoid the duplicat rows 
Add "order by" clause to avoid the duplicat rows: https://www.postgresql.org/docs/9.6/queries-limit.html

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-30 16:42:43 +08:00
Wenkai Yin(尹文开)
6815e8dc4d
Merge pull request #11348 from ywk253100/200330_tag_filter 
Make sure the tag filter have the same behavior for empty value and *
 2020-03-30 14:02:37 +08:00
He Weiwei
fbae9f0c25
Merge pull request #11347 from heww/refactor-errors 
Refactor errors
 2020-03-30 13:06:07 +08:00
Wenkai Yin
7ec5595bd8 Make sure the tag filter have the same behavior for empty value and * 
Fixes #11233, make sure the tag filter have the same behavior for empty value and *

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-30 10:10:12 +08:00
Will Sun
9a205ddbc3
Merge pull request #11332 from AllForNothing/scan-result-modify 
Modify UI for scanning result
 2020-03-30 09:47:45 +08:00
He Weiwei
1bf142c33b refactor: use lib/errors to instead of scan/errs 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-29 05:19:06 +00:00
He Weiwei
9c06c79ff4 refactor(errors): rename pkglib/error to lib/errors 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-29 05:19:05 +00:00
Wenkai Yin(尹文开)
98759642b7
Add API to list tags under the specific repository (#11336) 
Add API to list tags under the specific repository

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-29 12:19:54 +08:00
Steven Zou
1bbd3585e5
Merge pull request #11296 from ywk253100/200326_replication 
Support replicate images with media type application/vnd.docker.distribution.manifest.v1+json
 2020-03-27 18:12:12 +08:00
Steven Zou
36552ba18b
Merge pull request #11318 from ywk253100/200326_remove_pagination_default 
Iterate the link header when listing artifact
 2020-03-27 18:07:03 +08:00
Wang Yan
eccb8aa708
append pull permission for push policy (#11303) 
Fixes #11225
As registry changes to basic auth, the push action lost the pull permission.
Add it in the robot security context.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-27 17:10:04 +08:00
AllForNothing
3c51e37702 Modify UI for scanning result 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-27 16:41:38 +08:00
He Weiwei
033d6dac6b
fix(quota): allowed to put blob which size is zero (#11314) 
Closes #11239

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-27 14:39:25 +08:00
Wenkai Yin(尹文开)
f4ad0fbf00
Use the same logic to parse the registry URL (#11320) 
Use the same logic to parse the registry URL to fix #11274

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-27 10:33:20 +08:00
Wenkai Yin(尹文开)
e8cc84738a
Merge pull request #11309 from ywk253100/200326_error 
Fix bugs of replication
 2020-03-27 10:31:03 +08:00
Wang Yan
a5c1eae81a
give the username to anonymous when to pull public resource without authN (#11306) 
For pull a public resource, there is no need to login, give the access name to anonymous in the audit logs

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-27 10:24:59 +08:00
Wenkai Yin
9a92b9e725 Fix bugs of replication 
1. Bump up the version of API used in replicatoin scheduler job
2. Check the error message to determine whether the job exists or not in jobservice when unschedule a job

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 23:12:02 +08:00
Wenkai Yin
afdfedcb49 Iterate the link header when listing artifact 
Fixes #11315
When specify no pagination in listing artifact request, the go-swagger will set the default value for them, so we need to iterate the link header to get all of artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 21:26:09 +08:00
Daniel Pacak
48df949c30
feat(trivy): Return Trivy DB update timestamp in /api/v1/metadata response (#11285) 
Resolves: #11284

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-26 19:22:28 +08:00
Wenkai Yin
64e4651b3f Support replicate images with media type application/vnd.docker.distribution.manifest.v1+json 
Fixes #11272, support replicate images with media type application/vnd.docker.distribution.manifest.v1+json

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 19:09:48 +08:00
Wenkai Yin
213c534e8a Return 404 rather than 500 error when getting registry info 
In Harbor 2.0, the replication isn't supported between instances with different versions, this commit returns the 404 error when trying to get the registry info whose version is different with the current one

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 18:07:11 +08:00
He Weiwei
73f3a305ce
refactor: rename testing/api to testing/controller (#11295) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-26 14:00:11 +08:00
jwangyangls
96572c3c86
Merge pull request #11254 from jwangyangls/nightly-case-3 
[Fix]   Fix issue 2.0
 2020-03-26 12:14:37 +08:00
Yogi_Wang
cba4490a5a [Fixed] Fix bug for 2.0 and add case for trivy 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
1.add case for trivy
2.vunerbility refresh bug
3.scan mutiple artifact
4.fix global search bug
5.disable delete tag btn when remove immutable tag
6.cancel selectRow when add label or remove label;fix #11195
7.fix cron tootip
 2020-03-26 11:39:57 +08:00
qinshaoxuan
df9c2bdc46 Fix bug when scanner is unhealthy 
The function GetRegistrationByProject should not return err when Ping
return err.  The return value 'registration' has 'Health' field which
shows the scanner health status.

Resolves: #11051
See also: #9788, #9807

Signed-off-by: qinshaoxuan <qinshaoxuan@baidu.com>
 2020-03-26 11:25:47 +08:00
Wang Yan
da8902da53
Handle empty orlist in orm query (#11270) 
Fixes #11267
When caller parse an empty orlist to orm lib, it will parse the empty vaule to beego orm.
But beego will panic if the query string is empty.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-26 11:23:20 +08:00
Wenkai Yin(尹文开)
d05817c8a2
Update the URL checking logic of auth proxy security generator (#11180) 
As we don't support bearer token in Harbor 2.0, the URL checking logic in auth proxy security generator should be updated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-25 01:19:11 +08:00
Wenkai Yin(尹文开)
8984979bd2
Relocate/rename some packages (#11183) 
Fixes #11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-24 20:45:45 +08:00
Ted Guan
e49a247d3d
Replication webhook support (#11179) 
* replication webhook support

Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>

* replication webhook support with ut fixed

Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2020-03-23 18:45:58 +08:00
Wang Yan
168637a743
Add permission check for audit logs API (#11154) 
add a base method to require system admin permission

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-23 16:59:10 +08:00
Wang Yan
15d2a93aa2
Fix orm query setter issue (#11177) 
For the Andlist, the query setter should ignore it

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-23 16:05:20 +08:00
Wenkai Yin(尹文开)
1762bfae69
Merge pull request #11158 from ywk253100/200320_repository_api 
Add "_self" suffix for repository API to avoid conflict
 2020-03-23 14:45:52 +08:00
Wenkai Yin(尹文开)
c4af6ff824
Fix bug when deleting the repository (#11121) 
Fixes #10997 by looping the artifact candidates until all of them are deleted

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-23 10:38:47 +08:00
Wenkai Yin(尹文开)
8688f78cd2
Merge pull request #11144 from ywk253100/200319_security_middleware 
Rewrite the filters with middleware mechinism
 2020-03-23 10:12:48 +08:00
Will Sun
b740903314
Merge pull request #11143 from AllForNothing/resolver 
Add routing-resolvers
 2020-03-23 10:12:17 +08:00
Wenkai Yin
0453709b74 Rewrite the filters with middleware mechinism 
Fixes 10532,rewrite the filters with middleware mechinism

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-23 09:26:20 +08:00
jwangyangls
61d7eaa405
Merge pull request #11160 from jwangyangls/repo-pagination 
[fix] fix repo pagination
 2020-03-20 19:46:02 +08:00
Wenkai Yin(尹文开)
5f982bfee1
Merge pull request #11163 from heww/gc-refresh-quotas 
feat(gc,quota): refersh quotas for projects after gc
 2020-03-20 19:23:36 +08:00
Wenkai Yin(尹文开)
bf3b185357
Merge pull request #11162 from reasonerjt/rm-reset-pwd 
Remove route entry to reset password
 2020-03-20 19:17:53 +08:00
Yogi_Wang
2786a3347c [fix] fix repo pagination 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-20 17:22:28 +08:00
He Weiwei
5641ae49df feat(gc,quota): refersh quotas for projects after gc 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-20 09:13:50 +00:00
Wenkai Yin
dca06b6ede Add "_self" suffix for repository API to avoid conflict 
Add "_self" suffix for repository API to avoid conflict

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-20 17:08:08 +08:00
danfengliu
b51076ffa8
Merge pull request #11151 from jwangyangls/oci-nightly-change-2 
[OCI] Change nightly case and change delete artifact words
 2020-03-20 16:32:03 +08:00
Yogi_Wang
13ae4482ab [OCI] Change nightly case and change delete artifact words 
1.nightly: fix tag retention and immutable tag case xpath
2.nightly: fix the part of delete repo button xpath
3.nightly: fix the api version when GC
4.nightly: fix add label of artifact xpath
5.text:   change delete artifact show words
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-20 15:59:16 +08:00
Daniel Jiang
9b750f60df Remove route entry to reset password 
fixes #10712
The functions in CommonController are kept as a reference.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-20 14:05:21 +08:00
Wang Yan
63cf1fce7f
Unescape tags query when to list artifact (#11148) 
The query string is encoded by UI, and we have to unescape the "=" in "q=tag=nil",
otherwise, the query doesn't work, and returns 400

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-20 10:07:34 +08:00
Steven Zou
e8d5ba9491
Merge pull request #11004 from steven-zou/fix/update_js_ut_case 
fix[js_ut]:update stop job case of js
 2020-03-19 15:49:08 +08:00
Wang Yan
dc6eec8a73
Enable API logs test case (#11142) 
1, enable user view log api test case
2, update project logs api permission check
3, use project ctl instead in permission check base method

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-19 14:56:37 +08:00
AllForNothing
887d693fa4 Add routing-resolvers 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-19 14:45:04 +08:00
He Weiwei
21349e30af
feat(middleware,vulnerable): add image index checking for vulnerability prevention (#11084) 
1. Skip vulnerability prevention checking when artifact is not
scannable.
2. Skip vulnerability prevention checking when artifact is image index
and its type is `IMAGE` or `CNAB`.
3. Skip vulnerability prevention checking when the artifact is pulling
by the scanner.
4. Change `hasCapability` from blacklist to whitelist.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-19 10:48:19 +08:00
Wang Yan
9e4fdc571a
update internal common error (#10994) 
1, New support construct with string or err
2, Add Wrap/Errorf method

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-19 10:46:07 +08:00
Qian Deng
cf90ec27f2
Merge pull request #10706 from ninjadq/enable_tls_on_all_components 
Enable tls on all components
 2020-03-18 21:25:40 +08:00
Wenkai Yin(尹文开)
dbedcf960c
Merge pull request #11128 from wy65701436/perf-list-artifact 
simplify query string when to list artifact
 2020-03-18 21:00:19 +08:00
Wenkai Yin(尹文开)
c505c82d57
Merge pull request #11126 from ywk253100/200318_label_resource 
Remove the API to listing the resources that added with the specific label
 2020-03-18 20:58:28 +08:00
He Weiwei
fe39bb6a2a
feat(quota,notification): notification for quota exceeded and warning (#11123) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-18 20:24:23 +08:00
DQ
4c30995858 Refator tls config 
use default Httptransport instead of empty one
remove unused code

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
b93092e012 Add tls for trivy 
Add trivy tls cert files
Add tivey tls env and config
enhance gencert

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
c954969bcd Add mTLS configs 
mTLS only enabled in jobservice and registryctl

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
454382149f TLS update for chart, clairadapter, registry 
Remove trustca in chartmuseum
Remove trustca in registry
Add tls in clair-adapter

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
03e11c63c7 Fix docker file with secure tls change 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:10 +08:00
DQ
115185894f Merge internal Transport and Secure Transport 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
02dea3ad2c Add: mTLS configuration on CI 
Add internal_tls on ci
generate certs for ci

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
dcc6950af7 Feat: auto install ca in registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
b852605193 Feat: enable mtls in harbor replication 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
40e67f3b14 Feat: Enable mtls for registry 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
07a1d51693 Feat: enable tls in registryctlAdd tls related code in registryctl 
Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
10753caf90 Feat: enable tls in chart 
add tls related code in chart server

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
e6bb3b0977 Feat: enable tls related thing to jobservice 
Add tls related code in jobservice

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
DQ
da359f609f Feat: enable mtls in core 
add mtls related code in core

Signed-off-by: DQ <dengq@vmware.com>
 2020-03-18 19:22:09 +08:00
Wang Yan
b4e941e961
drop table access log in migration (#11118) 
Use the audit log instead, the access log table should be dropped after migration

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-18 19:04:38 +08:00
wang yan
3deef8a7d4 simplify query string when to list artifac 
To improve the performance of loading repository page, make the query set thinner.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-18 18:53:33 +08:00
Wenkai Yin
c92d9e4034 Remove the API to listing the resources that added with the specific label 
As we introduce a new table to record the relationship between the artifacts and labels, the current way to list label's resources doesn't work anymore, and the API isn't needed by any features, remove it in 2.0

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-18 18:47:44 +08:00
Wang Yan
0422721490
Enable pull time on getting manifest (#11110) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-18 18:38:37 +08:00
Wenkai Yin
256796ea9b Remove the project manager from context 
Remove the project manager introduced when integrated with Admiral from the context

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-18 15:28:02 +08:00
Wenkai Yin(尹文开)
eb2af6095e
Merge pull request #11107 from ywk253100/filter_replication_pull 
Filter the pulling manifest request from replication service
 2020-03-18 14:36:29 +08:00
Wenkai Yin(尹文开)
798dda8604
Escapse the repository name in the link header returned in response (#11037) 
Escapse the repository name in the link header returned in response

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-18 14:29:15 +08:00
He Weiwei
7d20154db5
fix: remove old artifact model (#11112) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-18 14:20:06 +08:00
jwangyangls
2f7ce0da1c
Merge pull request #11096 from jwangyangls/fix-some-issue-oci 
[OCI] changes some show words
 2020-03-18 14:00:13 +08:00
Wang Yan
050967f95f
Use new query model to get audit logs (#11113) 
* Use new query model to get audit logs

leverage the query builder to build query, remove the old style query string

Signed-off-by: wang yan <wangyan@vmware.com>

* Switch to new API  for  project log page

Signed-off-by: AllForNothing <sshijun@vmware.com>

Co-authored-by: AllForNothing <sshijun@vmware.com>
 2020-03-18 13:46:49 +08:00
Yogi_Wang
891ef80e46 [OCI] changes some show words 
1. search result show artifact count
2. replication shows changes both to all
3. fix delete bug when delete some artifact
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-18 12:17:30 +08:00
Wang Yan
7af0bd5ed7
Fix delete scan report on deleting artifact (#11102) 
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-17 21:25:06 +08:00
Wenkai Yin(尹文开)
f02c5570a7
Merge pull request #11111 from ywk253100/200317_clean_todo 
Clean up some TODO items
 2020-03-17 20:17:22 +08:00
He Weiwei
e3c5c37668
fix(scan): assign repository pull access policy to robot account when scan artifact (#11109) 
The v2auth middleware will check whether the requestor  has the pull or
push permissions for the repository, and forbid the request when the
requestor does not have the permission.  We need to assign repository
pulling permission to the robot account for the scanner, otherwise
scanner will be failed to pull the artifact.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-17 19:30:21 +08:00
He Weiwei
f8983fe198
feat(log): track request id in the log message (#11095) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-17 19:29:59 +08:00
Wenkai Yin
3aca33acde Clean up some TODO items 
1. Remove blob fetcher and cache

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-17 19:01:38 +08:00
Wenkai Yin
e8935dd804 Filter the pulling manifest request from replication service 
Filter the pulling manifest request from replication service so that the audit log will not record the pulling action

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-17 16:32:11 +08:00
Wang Yan
ce2257dc22
update project logs api to v2.0 (#11097) 
use audit log api to get project logs

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-17 11:58:43 +08:00
Will Sun
2b6fb4abcf
Merge pull request #11073 from AllForNothing/permission 
Swith to new API for recent log page
 2020-03-17 11:25:29 +08:00
Wenkai Yin(尹文开)
411c73bd79
Merge pull request #11046 from ywk253100/200312_replication 
Replicate tag deletion between Harbor instances
 2020-03-17 10:58:06 +08:00
jwangyangls
89cdd7a9f9
Merge pull request #11089 from jwangyangls/clear-dead-code 
[OCI] Remove dead code
 2020-03-17 10:03:18 +08:00
jwangyangls
580b74035d
Merge pull request #11007 from jwangyangls/artifact-filter-changes 
[OCI] Artifact filter params changes in ui
 2020-03-17 10:02:50 +08:00
Wenkai Yin
5925e0862d Replicate tag deletion between Harbor instances 
This commit introduces the tag deletion as a new capability for registry adapters, and currently only Harbor supports it

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-17 09:27:02 +08:00
Yogi_Wang
61fa461e91 [OCI] Remove dead code 
1. remove tag service / tag model
2. remove retag service
3. remove artifact service some function
4. remove repository service / repository model  /repositoryItem model
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-16 17:24:21 +08:00
Wang Yan
fbb3226e85
move notification handles and events metadata into api (#11085) 
1, enable audit logs for notifications
2, move the handler and meatadata into API
3, use the notification middleware to send out notification

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-16 16:56:34 +08:00
Wenkai Yin(尹文开)
a83c78c1a5
Merge pull request #11064 from ywk253100/200313_cached_token 
Check expired or not when getting token from cache
 2020-03-16 16:27:18 +08:00
Wenkai Yin(尹文开)
89eeeb29ca
Change tag count to artifact count in search result (#11068) 
Change tag count to artifact count in search result

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-16 14:28:59 +08:00
Wenkai Yin(尹文开)
d250e6998e
Fix bug when reading the readme.md of helm chart (#11059) 
Fix bug when reading the readme.md of helm chart

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-16 14:26:50 +08:00
jwangyangls
46fe1182b7
Merge pull request #11043 from jwangyangls/add-OPENPOLICYAGENT 
[OCI] Add the icon of openpolicyagent artifact
 2020-03-16 13:39:10 +08:00
Daniel Jiang
2615339f4c
Merge pull request #11076 from reasonerjt/csrf-secure-flag 
make Secure flag of CSRF cookie adapt to config
 2020-03-16 11:47:51 +08:00
He Weiwei
60f8595034
refactor(quota): implement internal quota APIs by quota controller (#11058) 
1. Use quota controller to implement the internal quota APIs.
2. The internal quota APIs can exceed the quota limitations.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-16 10:20:17 +08:00
Daniel Jiang
cbd2619035 make Secure flag of CSRF cookie adapt to config 
fixes #11074

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-16 04:03:49 +08:00
Wang Yan
9cc6e88a65
add notification middleware (#11072) 
the notification is for send out the event after DB transaction complete.
It's safe to send hook as this middleware is after transaction in the response path.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-14 22:34:36 +08:00
He Weiwei
ec31a87884
fix(blob,quota): process blobs already in registry no but associated with project (#11071) 
1. Before put manifest request, ensure that the requested size resource
include the blobs which are referenced by the manifest but not
associated with project.
2. After put manifest request, associate the blobs which are referenced
by the manifest but not associated with project.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-14 21:39:44 +08:00
Wenkai Yin
c6940e8184 United error response format for management APIs (legacy and v2.0 APIs) 
United error response format for management APIs (legacy and v2.0 APIs)

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-13 22:00:08 +08:00
AllForNothing
05431a149d Swith to new API for recent log page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-13 19:56:44 +08:00
Wenkai Yin(尹文开)
4a97cd270d
Merge pull request #11038 from ywk253100/200312_upgrade 
Repair the count usage during the upgrading
 2020-03-13 16:19:12 +08:00
He Weiwei
37e6fa5c92
fix(transaction): change to use value in the ctx to decide whether commit tx (#11062) 
Type assertion not work when the ctx in the request changed in the next
handler, so change to use value in the ctx to decide whether to commit
tx.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-13 15:19:13 +08:00
Wenkai Yin
05255a7ea7 Check expired or not when getting token from cache 
Check expired or not when getting token from cache

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-13 14:57:30 +08:00
Wenkai Yin
a4a1913598 Repair the count usage during the upgrading 
As the count quota is against artifact rather than tag in 2.0, the count usage should be recalculated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-13 13:59:48 +08:00
Yogi_Wang
b32a8db114 [OCI] Add the icon of openpolicyagent artifact 
1. add image
2. fix show ui clearly when dark
3. fix chinese words of replication name filter tooltip
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-13 12:01:00 +08:00
Daniel Jiang
2e7eb8872e
Move ArtifactInfo to internal package (#11055) 
To avoid depedency loop, this commit moves the model of ArtifactInfo to
internal pacakge, so that a controller can it from context when needed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-13 11:16:13 +08:00
He Weiwei
2a243ef7a2
refactor(rbac): refactor rbac impl to improve performance (#9988) 
1. Introduce `Evaluator` interface which do the permission checking.
2. `admin`, `lazy`, `rbac`, `namespace` and `evaluartor` set are implemented the
`Evaluator` interface.
3. Move project rbac implemention from `project` to `rbac` pkg to reduce
the name  conflict with project instance of model.
4. Do permission checking in security context by `Evaluator`.
5. Cache the regexp in rbac evaluator for casbin.
6. Cache evaluator in namespace evaluator to improve performance.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-12 23:42:53 +08:00
Ziming Zhang
8ffa79801b feature(tag_retention) add checkbox for user to control whether remove untagged image 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-12 22:50:21 +08:00
He Weiwei
12f16c8cec
feat(scan): support to scan image index (#11001) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-12 19:30:12 +08:00
Wenkai Yin
289f04d301 Restructure the packages of artifact 
1. Introduce a new interface Processor to replace Abstractor and Descriptor
2. Provide the base processors for manifest and index to reduce the duplicate code
3. Move the child artifacts checking out of processor

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-12 16:03:13 +08:00
Ziming Zhang
25b5c3796b enhance(replication) update healthy status immediately 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-12 14:55:50 +08:00
Steven Zou
b546d9167a
Merge pull request #11019 from bitsf/replication_adapter_sort 
feat(replication) sort the adapters shown on UI
 2020-03-12 14:53:34 +08:00
Ziming
b597d9d59a
feat(ci) enhance govet check performance (#11008) 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-12 11:03:24 +08:00
Wenkai Yin(尹文开)
dcaccbc757
Merge pull request #10945 from ywk253100/200305_swagger_model 
Remove the "x-go-type" for artifact definition in swagger
 2020-03-12 10:47:00 +08:00
Wenkai Yin
4ccc3da99b Remove the "x-go-type" for artifact definition in swagger 
Using "x-go-type" may cause the inconsistence between the swagger definition and the real data model

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-12 10:06:22 +08:00
Will Sun
1831aeb45c
Merge pull request #11023 from AllForNothing/robot 
Improve UI for add robot page
 2020-03-12 09:46:20 +08:00
Will Sun
878c004d9e
Merge pull request #11025 from jwangyangls/upgrade-clr 
[feat] Upgrade clarity to 2.3.8
 2020-03-12 09:45:42 +08:00
He Weiwei
89dfe24f19
feat(quota): add Request and Refresh middlewares for APIs (#10907) 
1. Introduce ReqquestMiddleware and RefereshMiddleware.
2. Add request middlware to copy artifact, mount blob, put blob upload,
put manifest, upload chart verson APIs.
3. Add refresh project middleware to delete manifest, delete artifact,
delete chart version, delete repository APIs.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-11 22:25:40 +08:00
Will Sun
aa73f16a20
Merge pull request #11027 from AllForNothing/webhook 
Improve Webhook UI
 2020-03-11 18:35:29 +08:00
Wenkai Yin(尹文开)
c2826d0368
Merge pull request #11030 from mmpei/webhook-dev-slack 
add support slack in webhook
 2020-03-11 18:20:58 +08:00
Ted Guan
4ac31c6d46
Add API for query supported event types and notify types; Return policy name in last trigger info; Remove project_id unique constraint in table notification_policy (#11029) 
Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2020-03-11 18:06:58 +08:00
Ziming Zhang
d1d0601841 feat(replication) sort the adapters shown on UI 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-11 18:06:29 +08:00
peimingming
3a6d1d75d0 add support slack in webhook 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2020-03-11 17:19:38 +08:00
AllForNothing
a19900e96e Improve webhook UI 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-11 16:51:21 +08:00
Wenkai Yin(尹文开)
8452100148
Merge pull request #10942 from ywk253100/200305_reference 
Persistent the URLs and annotations of artifact references in database
 2020-03-11 16:20:18 +08:00
Will Sun
fd3997678b
Merge pull request #10993 from AllForNothing/gc-ui 
Add new parameter for GC page
 2020-03-11 15:49:15 +08:00
AllForNothing
2fdb01ef1a Improve UI for add robot page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-11 15:42:33 +08:00
Yogi_Wang
dccf125016 [feat] Upgrade clarity to 2.3.8 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-11 15:41:33 +08:00
Wenkai Yin(尹文开)
d644d23b25
Merge pull request #10370 from kofj/fix/aliacr 
FIX: AliACR Provider.
 2020-03-11 15:36:34 +08:00
Ziming Zhang
7d53a61a92 feat(replication) sort the adapters shown on UI 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-11 15:03:10 +08:00
Wenkai Yin(尹文开)
b02cab434f
Fire event when create/delete resources (#11010) 
1. Create/delete project
2. Create/delete repository
3. Push/pull/delete artifact
4. Create/delete tag

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-11 14:39:01 +08:00
He Weiwei
41edfaf3a6
fix(api): escape path paramters before APIs and unescape them in the Prepare of operations (#11013) 
1. Escape the path paramters before the APIs.
2. Unescape the path paramters in the Prepare stage of the swagger
operations.

Closes #10860

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-11 12:18:40 +08:00
stonezdj(Daojun Zhang)
c7fd3bdfc5
Refactor event model (#10876) 
Move src/pkg/notification/model/const.go to src/pkg/notifier/model/const.go
Add auditlog handler to log project event, repo event, artifact event and tag event.

Signed-off-by: stonezdj <stonezdj@gmail.com>
 2020-03-11 11:51:28 +08:00
Yogi_Wang
fe047a2ad3 [OCI] Artifact filter params changes in ui 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-11 11:21:08 +08:00
Ziming Zhang
5622a20058 feat(pkg) move artifactselector to src/internal 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-10 17:01:25 +08:00
Steven Zou
b2434945ff fix[js_ut]:update stop job case of js 
remove the status checking loop

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-03-10 15:54:10 +08:00
Ziming
890200ea19
feature(tag_retention) add webhook for deleted artifacts (#10982) 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-03-10 10:46:58 +08:00
Will Sun
cb370f8dd9
Merge pull request #10980 from AllForNothing/scan-bug 
Fix  a bug for scanning
 2020-03-10 09:51:10 +08:00
Wenkai Yin(尹文开)
307dbc6fba
Accept the pagination information in the separated query string (#10991) 
Accept the pagination information in the separated query string

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-09 22:19:42 +08:00
AllForNothing
de009f49fb Add new parameter for GC page 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-09 17:51:00 +08:00
Yogi_Wang
a8a7975522 Csrf change to v2.0 in ui 
1.delete personal xsrf service
2.change to direactive get token
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-09 15:08:50 +08:00
Wenkai Yin(尹文开)
30896f3a10
Merge pull request #10968 from ywk253100/200306_artifact_query 
Add support for querying artifact by labels and tags
 2020-03-09 14:10:19 +08:00
Daniel Jiang
6d89553c4d
Merge pull request #10937 from reasonerjt/csrf-2.0 
Update CSRF mechanism
 2020-03-09 12:31:08 +08:00
Wang Yan
073d95b89f
add scanner pull check in policy checker middleware (#10971) 
Scanner uses the robot account to pull image and scan, the policy checker should bypass the
pull action even the policy enabled, otherwise the scan job will fail.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-09 12:11:54 +08:00
Daniel Jiang
7897fd752b
Merge pull request #10969 from reasonerjt/rm-travis 
Remove dependency on travis-ci
 2020-03-09 12:06:57 +08:00
Wenkai Yin(尹文开)
c0542776e0
Merge pull request #10961 from ywk253100/200306_replication 
Make replication work with new artifact(phase 2)
 2020-03-09 11:58:26 +08:00
Wenkai Yin(尹文开)
52c6d354d1
Merge pull request #10967 from ywk253100/200307_auth_header 
Only set "Www-Authenticate" header for registry API
 2020-03-09 11:58:09 +08:00
Wang Yan
7b8aca6cd2
Merge pull request #10973 from wy65701436/inlimited-robot 
add no expriation limited robot account
 2020-03-09 11:30:21 +08:00
Wang Yan
4c167b7a33
Merge pull request #10972 from wy65701436/conformance-fix 
fix Conformance testing failure
 2020-03-09 11:28:45 +08:00