Upstream/harbor
1
0
Fork 0
mirror of https://github.com/goharbor/harbor.git synced 2024-07-01 09:15:08 +02:00
Code Issues Projects Releases Wiki Activity
9,506 Commits 46 Branches 304 Tags 361 MiB
c000608d55
Commit Graph

4252 Commits

Author SHA1 Message Date
He Weiwei
b1c9d452ce feat(quota,webhook): send quota webhook for put and mount blob 
Closes #11712

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-28 00:59:16 +00:00
Daniel Jiang
87f006d3a9
Merge pull request #11755 from reasonerjt/token-service-core-url 
v2 auth middleware handles the ping request from internal
 2020-04-27 15:04:10 +08:00
stonezdj(Daojun Zhang)
fb90bc23f2
Merge pull request #11757 from heww/fix-issue-11692 
feat(scan): support to scan artifact automatic after it pushed
 2020-04-27 12:08:06 +08:00
Daniel Jiang
ef008fd4cf
Merge pull request #11751 from wy65701436/fixes-11744 
fix return code on getting non exist manifest
 2020-04-27 12:06:37 +08:00
Daniel Jiang
fe587d0cc8 v2 auth middleware handles the ping request from internal 
When scanner like trivy handles the auth flow to pull image, it pings
the /v2 and access the token service url in response body, by default it
will be external endpoint of Harbor.
There will be problem when Harbor is deployed on a single node with hairpinning not
supported.

This commit makes sure the address of token service in the challenge is
internal url of core component when the request is from internal.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-27 00:54:49 +08:00
He Weiwei
bc1f7b8079 feat(scan): support to scan artifact automatic after it pushed 
Closes #11692

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-26 15:56:58 +00:00
wang yan
c4c279089a fix return code on getting non exist manifest 
It's found by conformance test, it should be 404 instead of 500 when to get a non exist manifest

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-26 23:31:11 +08:00
AllForNothing
fd65520fa0 dESTInation namespace should support slash 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-26 17:24:15 +08:00
Steven Zou
06d955c6e3
Merge pull request #11737 from steven-zou/fix/issue_#11720 
fix(jobservice):fix job stats NOT_FOUND issue
 2020-04-26 16:56:25 +08:00
Wang Yan
8d5fc4c752
Fix delete repository event issue (#11722) 
In the delete repository scenario, the repository has already been removed from the DB, the repository cannot be retrived.
Just let the creation time as empty.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-26 16:51:13 +08:00
Steven Zou
0ccea49c18 fix(jobservice):fix job stats NOT_FOUND issue 
Signed-off-by: Steven Zou <szou@vmware.com>
 2020-04-26 16:23:10 +08:00
Daniel Jiang
c28f6bd6ff
Merge pull request #11738 from bitsf/retention_not_retry 
fix(retention) not retry retention job
 2020-04-26 15:34:34 +08:00
Ziming Zhang
97a7a6dc35 fix(retention) fix retention repository with slash 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-04-26 15:07:25 +08:00
Ziming Zhang
601ce08660 fix(retention) not retry retention job 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-04-26 14:33:21 +08:00
Ziming Zhang
257924856c fix(retention) fix empty pull time log 
Signed-off-by: Ziming Zhang <zziming@vmware.com>
 2020-04-26 11:56:26 +08:00
jwangyangls
0460e40a28
Merge pull request #11680 from jwangyangls/200421-fix-issue-2.0 
[Fixed] Fix issue #11334 #11252 #11191
 2020-04-24 17:46:22 +08:00
AllForNothing
432f5512d8 Fix bugs for testing kick 3 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-24 17:11:34 +08:00
Yogi_Wang
7bb7c2c868 [Fixed] Fix issue #11334 #11252 #11191 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-24 16:38:59 +08:00
Daniel Jiang
447ec13527 Allow sys admin to call /c/UserExists 
The commit fix a regression introduced by #11672 which impacts admin
adding new users.
When admin is creating new users, /c/UserExists is called by UI. We must
allow it called by admin when self-registration is turned off.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-23 11:40:54 +08:00
Will Sun
7d784ef02f
Merge pull request #11674 from AllForNothing/more-ut 
Add more UT
 2020-04-22 17:13:20 +08:00
Daniel Jiang
4c41ac81d9
Merge pull request #11672 from reasonerjt/restrict-userexists 
Enable userExists only when self-registration is turned on.
 2020-04-21 13:19:20 +08:00
mmpei
4b6196a00d
fix slack rate limit issue (#11623) 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2020-04-21 11:44:58 +08:00
AllForNothing
0c5d3cc899 Add more UT 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-21 09:54:32 +08:00
Daniel Jiang
201955c2fb Enable userExists only when self-registration is turned on. 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-21 00:17:12 +08:00
Wang Yan
ff2a7e61c9
fix catalog api issue (#11666) 
The v2 catalog API needs to filter out the empty repository and the repository which artifacts are all with no tags.

1，In v2.0.0, Harbor does not delete repository even there is no artifact, it's different with v1.10.0
2, Compares with docker distribution, it doesn't return the respository with untagged images.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-20 23:37:16 +08:00
DQ
e0b98685f3 Add comments for new tls transport 
To explain why use this to avoid replication hang forever issue

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-20 19:19:15 +08:00
Yogi_Wang
cd98a7a9b6 [Fixed] Fix issue when delete signed tag 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-17 16:04:16 +08:00
Steven Zou
3b0b9f3c7d
Merge pull request #11645 from ninjadq/avoid_default_transport_https_bug 
Fix: Default Transport HTTP2 related hang issue
 2020-04-17 15:44:52 +08:00
jwangyangls
981cae9f11
Merge pull request #11648 from jwangyangls/add-sign-pop-when-delete 
[feat] Add pop up command when delete signed tag
 2020-04-17 13:14:19 +08:00
Yogi_Wang
6f54262889 [feat] Add pop up command when delete signed tag 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-17 12:45:57 +08:00
DQ
53111d1d16 Fix: Default Transport HTTP2 related hang issue 
Create a transport the same as default except forceattempthttp2

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-17 11:36:02 +08:00
Wenkai Yin(尹文开)
e3cbfac0cc
Fix bug for replicating chart triggered by event (#11578) 
Use the new event model to fix bug for replicating chart triggered by event

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-16 16:15:21 +08:00
Wang Yan
8a0e8627ff
replace pkg errors with lib errors (#11605) 
Fixes #9704

As we do want to unify error handling, so just decreprates pkg errors, use lib/errors instead for Harbor internal used errors model.

1, The lib/errors can cover all of funcs of pkg/errors, and also it has code attribute to define the http return value.
2, lib/errors can give a OCI standard error format, like {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}

If you'd like to use pkg/errors, use lib/errors instead. If it cannot meet your request, enhance it.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-15 22:41:45 +08:00
Will Sun
205e3a969f
Merge pull request #11621 from AllForNothing/gc-ui 
Modify gc ui
 2020-04-15 22:01:59 +08:00
He Weiwei
385aaac00d
Merge pull request #11620 from heww/fix-issue-11524 
feat(scanner): make Clair and Trivy as reserved name for scanners
 2020-04-15 15:21:35 +08:00
Wang Yan
9bc96dd97a
add MarshalJSON func for lib errs (#11614) 
Customize the json output with message with err.Error(). Otherwise, the wrappged message will be lost
in the final errors object.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-15 11:30:14 +08:00
AllForNothing
a337dcf517 Modify gc ui 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-15 11:06:17 +08:00
He Weiwei
f5487479dd feat(scanner): make Clair and Trivy as reserved name for scanners 
Closes #11524

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-15 02:26:02 +00:00
Wang Yan
c6860ac35f
handle stop signal in GC job (#11612) 
Stop GC job in the init if receives the stop signal

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-14 18:59:03 +08:00
Wenkai Yin(尹文开)
2d618370ae
Merge pull request #11588 from wy65701436/trace-error 
update trace level to error
 2020-04-14 18:30:21 +08:00
Qian Deng
95d7c9382b
Merge pull request #11592 from ninjadq/min_version_tls_to_12 
Min version tls to 12
 2020-04-14 18:12:55 +08:00
Will Sun
ad60bdddf8
Merge pull request #11591 from AllForNothing/nightly-untag 
Add new nightly case for GC untagged images
 2020-04-14 17:34:58 +08:00
Will Sun
faf554f642
Merge pull request #11587 from AllForNothing/sort-name 
Fix repo list sorting and filtering bug
 2020-04-14 17:34:39 +08:00
wang yan
ff2a6c7a01 add warning to registry binary name 
Fixes #11606

As we DO NOT want to user to execute GC in the container, rename it and append the warning message.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-14 15:16:50 +08:00
wang yan
7622c17817 update trace level to error 
Use level error instead to log trace

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-14 15:01:52 +08:00
Steven Zou
adb305e4e1
feat(job):enable priority option (#11608) 
- priority option is supported when doing job registration
- the priority is defined by a unique priority sampler
- the default priority is 1000 (max is 10000)

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-04-14 14:54:44 +08:00
jwangyangls
fb3da503ea
Merge pull request #11584 from jwangyangls/add-docker-pull-push-command 
[feat] Add pull/push command in repo
 2020-04-14 10:05:39 +08:00
AllForNothing
46320641b4 Add new nightly case for GC untagged images 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-14 09:25:06 +08:00
Wenkai Yin(尹文开)
f972f2989c
Close the reponse body after reading data (#11594) 
Close the reponse body after reading data

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-14 01:19:40 +08:00
Wenkai Yin(尹文开)
4b4091b217
Merge pull request #11599 from wy65701436/revise-base-error 
update internal error output format
 2020-04-13 20:07:05 +08:00
wang yan
269d0b9f9d update internal error output format 
1, remove the code from output
2, output format is same as pkg/errors, it's easy to migrate
3, add UT

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-13 19:29:55 +08:00
DQ
b3db293091 TLS update min version and cipher suits 
min version set to tls 1.2
suit only use ecdhe and strenth above 256

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-13 18:13:30 +08:00
Steven Zou
3ad5b2ba06
fix(job currency):introduce max corrency of job (#11589) 
- update Job interface to introdcue MaxCurrency method for declaring the max currency of the specified job
- change the downstream jobs to implement the new interface method
  - GC and sample jobs are set to 1
  - other jobs are set to 0 that means unlimited
- add max currency optiot when doing job registration
- resolve issue #11586
  - probably resolve issue #11281
  - resolve issue #11570

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-04-13 18:07:54 +08:00
Wenkai Yin
7553845b4d Remove the duplicated const definition 
Remove the duplicated const definition for artifact type

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-13 17:02:23 +08:00
He Weiwei
49c9e4f696
Merge pull request #11585 from heww/cleanup-quota 
refactor(quota): cleanup code for quota
 2020-04-13 15:11:17 +08:00
Yogi_Wang
9e1bdc88e6 [feat] Add pull/push command in repo 
1.add pull/push command in repo;
2.move annotations from artifact list to artifact summary

Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-13 13:11:33 +08:00
AllForNothing
e5e39c03c5 Fix repo list sorting and filtering bug 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-13 12:24:46 +08:00
Will Sun
009662a317
Merge pull request #11426 from AllForNothing/nightly-tag 
Add new nightly cases for tag CRUD
 2020-04-13 12:15:18 +08:00
He Weiwei
0b87eaf039
Merge pull request #11505 from heww/revert-registry-authorization-type-support 
feat(scan): revert bearer token support for scanner
 2020-04-13 11:19:02 +08:00
He Weiwei
c0349da812 refactor(quota): cleanup code for quota 
1. Remove `common/quota` package.
2. Remove functions about quota in `common/dao` package.
3. Move `Quota` and `QuotaUsage` models from `common/models` to
`pkg/quota/dao`.
4. Add `Count` and `List` methods to `quota.Controller`.
5. Use `quota.Controller` to implement quota APIs.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-12 16:16:06 +00:00
Daniel Jiang
93f316ccfe
Merge pull request #11582 from heww/fix-issue-11564 
fix(blob): delete project blob with project_id
 2020-04-12 18:00:54 +08:00
He Weiwei
c585e22d18 fix(blob): delete project blob with project_id 
Closes #11564

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-12 04:23:18 +00:00
Wang Yan
740b1e46b4
fix artifact trash filter issue (#11575) 
fixes #11533

GC jobs will use the filter results to call registry API to delete manifest.

In the current imple, the filter function in some case does not return the deleted artifact as it's using digest as the filter condition.

Like: If one artifact is deleted, but there is another project/repo has a image with same digest with the deleted one, filter func will
not mark the deleted artifact as candidate. It results in, GC job does not call API to remove the manifest.

To fix it, update the filter to use both digest and repository name to filter candidate.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-12 03:11:49 +08:00
Daniel Jiang
8822e6cdba
Merge pull request #11572 from ywk253100/200410_replication_insecure 
Use insecure transport when creating the adapter for local instance in replication
 2020-04-12 00:34:14 +08:00
He Weiwei
1ce0a76bd1
Merge pull request #11555 from reasonerjt/exclude-deleted-projects 
Exclude deleted groups when counting groups associated with group ID
 2020-04-11 16:52:35 +08:00
He Weiwei
4623cec1e5 feat(scan): revert bearer token support for scanner 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-11 08:45:29 +00:00
Daniel Jiang
32ae0a6fa6 Exclude deleted projects when counting projects associated with group ID 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-10 19:07:08 +08:00
Wenkai Yin(尹文开)
81a4e7d468
Merge pull request #11569 from ywk253100/200410_replication 
Add "delete" into the action map if the action in token is "*"
 2020-04-10 18:06:47 +08:00
jwangyangls
e28b5811f7
Merge pull request #11176 from jwangyangls/change-helm-version 
Separate swagger to get v2.0 swagger and chart swagger
 2020-04-10 17:12:00 +08:00
jwangyangls
5d76f1ea4b
Merge pull request #11530 from jwangyangls/200408-fix-bug-round2 
[Fixed] fix part issue of round2
 2020-04-10 17:11:43 +08:00
Wenkai Yin
df1f52dba4 Use insecure transport when creating the adapter for local instance in replication 
Use insecure transport when creating the adapter for local instance in replication

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-10 16:45:57 +08:00
Yogi_Wang
33ed4fb67e Separate swagger to get v2.0 swagger and chart swagger 
1. Partial helm api version number clear
2. Separate swagger to get v2.0 swagger and chart swagger
3. router add chart swagger

Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-10 16:25:30 +08:00
Yogi_Wang
3c771670ca [Fix] Fix part issue of round2 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-10 16:15:42 +08:00
Wenkai Yin
847a513cea Add "delete" into the action map if the action in token is "*" 
Fixes #11563, add "delete" into the action map if the action in token is "*"

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-10 16:11:44 +08:00
He Weiwei
f4821e96b3
Merge pull request #11527 from heww/refresh-quota-ignore-limitation 
feat(quota): ignore limitation support for quota RefreshMiddleware
 2020-04-10 15:29:41 +08:00
Daniel Jiang
56b404bfb7
Get digest in content trust middleware (#11554) 
This commit removes the EnsureArtifactDigest as its implementation is
problematic: the artifactinfo in context is immutable.
When the content trust middleware needs the digest it will retrieve it
via artifact controller.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-10 14:11:56 +08:00
Steven Zou
6986af6e74
Merge pull request #11558 from danielpacak/fix/issue_11310/preserve_default_scanner 
fix(scanner): Do not override the default scanner on init
 2020-04-10 12:42:39 +08:00
Steven Zou
5661ff937b
Merge pull request #11493 from steven-zou/fix/js_ut_case_failure 
fix(js UT cases):fix the cron spec bug
 2020-04-10 10:16:05 +08:00
Daniel Pacak
dcbf0726e5 fix(scanner): Do not override the default scanner on init 
Resolves: #11310

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-04-09 17:56:07 +02:00
Wenkai Yin(尹文开)
340027a882
Merge pull request #11484 from ywk253100/200407_replication_hairpin 
Fix replication failure in kubernetes with hairpin mode disabled
 2020-04-09 16:50:46 +08:00
Wenkai Yin
8ee3421176 Fix replication failure in kubernetes with hairpin mode disabled 
Fixes #11480
Fixes #11461
Fix replication failure in kubernetes with hairpin mode disabled

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-09 15:10:25 +08:00
Will Sun
71821a1c9a
Merge pull request #11497 from AllForNothing/css 
Fix css bugs of label 'target2.0'
 2020-04-09 14:20:30 +08:00
Wenkai Yin(尹文开)
8fb46d4bc7
Just log the signature not found error in debug mode (#11529) 
Fixes #11510, just log the signature not found error in debug mode when deleting artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-09 14:17:40 +08:00
AllForNothing
a411879196 Fix css bugs of label 'target2.0' 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-09 10:32:07 +08:00
He Weiwei
0b26b36737 feat(quota): ignore limitation support for quota RefreshMiddleware 
1. Ignore limitation when refresh quota for project.
2. Return 403 when quota errors occurred.
3. Add test for Refresh method of quota controller.

Closes #11512

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-08 19:17:41 +00:00
He Weiwei
20115b92c7 fix(vulnerable): fix the wrong count of vulnerabilities in message 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-08 09:06:31 +00:00
Steven Zou
59f14dff98 fix(js UT cases):fix the cron spec bug 
- eliminate the time of cron spec  overflow issue in the UT case

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-04-08 15:59:05 +08:00
Daniel Jiang
6ad855f0ee
Merge pull request #11475 from reasonerjt/rm-token-claims-registry 
Remove the registry claim pacakge
 2020-04-08 12:19:56 +08:00
AllForNothing
e342b4ef0b Add new nightly cases for tag CRUD 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-08 09:56:27 +08:00
Steven Zou
76625eab54
Merge pull request #11473 from steven-zou/fix/issue_#11466 
fix[jobservice]:enqueue job with UTC
 2020-04-08 08:44:43 +08:00
Wenkai Yin(尹文开)
e1ba985b7c
Merge pull request #11436 from ywk253100/200404_repo_encode 
Update APIs to only accept encoded repository name that contains slash
 2020-04-07 23:14:44 +08:00
Wenkai Yin(尹文开)
bf6c4ff1f1
Merge pull request #11479 from ywk253100/200407_replication 
Fix replication bug
 2020-04-07 22:41:19 +08:00
Yogi_Wang
3775509ce5 Update APIs to only accept encoded repository name 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-07 22:40:39 +08:00
Wenkai Yin
2171634000 Fix replication bug 
Remove the URL replacing logic temporarily to make replication work and will introduce a new solution for the hairpin issue

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-07 21:27:06 +08:00
Wenkai Yin
7188e01569 Update APIs to only accept encoded repository name that contains slash 
Update APIs to only accept encoded repository name that contains slash

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-07 20:57:50 +08:00
Wenkai Yin
0a372a85eb Remove "GetMyProjects" and "GetProjectRoles" in the interface "security.Context" 
Fixes #11125, remove "GetMyProjects" and "GetProjectRoles" in the interface "security.Context"

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-07 19:53:38 +08:00
Steven Zou
0fc7629865 fix[jobservice]:enqueue job with UTC 
- schedule the periodical jobs following the UTC timezone
  - e.g: 5 10 10 * * * means run jobs at UTC time 10:10:05 everyday
- fix issue #11466

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-04-07 17:10:19 +08:00
He Weiwei
3f567514b5
Merge pull request #11468 from wy65701436/remove-count-quota-code 
remove the chart handling in quota
 2020-04-07 16:51:07 +08:00
Daniel Jiang
c303dcf617 Remove the registry claim pacakge 
This commit removes `src/pkg/token/claims/registry` that is not referenced by other
packages.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-07 16:43:28 +08:00
Wenkai Yin(尹文开)
88ae9c458c
Update tags related APIs (#11435) 
* Update tags related APIs

1. Remove API for listing tags of repository
2. Add API for listing tags of artifact
3. Support filter artifact by tag name

Signed-off-by: Wenkai Yin <yinw@vmware.com>

* [OCI] modify artifact tag name check
1. switch api get tag list
2. modify artifact tag name check
Signed-off-by: Yogi_Wang <yawang@vmware.com>

Co-authored-by: Yogi_Wang <yawang@vmware.com>
 2020-04-07 16:28:51 +08:00
wang yan
a96d2f3746 remove the chart handling in quota 
1, remove the chartmuseum controller
2, doesn't handle chartrepo url in v2 middleware

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-07 15:26:34 +08:00
He Weiwei
dd95866e6a
Merge pull request #11467 from heww/fix-issue-11131 
fix(scan): dump nil slice of vulnerabilities as empty slice in report
 2020-04-07 15:01:17 +08:00
Wenkai Yin(尹文开)
5d55bd1d0c
Merge pull request #11463 from ywk253100/200407_copy 
Update the logic of copy artifact
 2020-04-07 14:36:27 +08:00
He Weiwei
e1ab30dadf fix(scan): dump nil slice of vulnerabilities as empty slice in report 
Closes #11131

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-07 06:20:25 +00:00
He Weiwei
8ddfe2d0a5
Merge pull request #11460 from heww/merge-scan-report 
feat(scan): merge reports for image index
 2020-04-07 13:07:52 +08:00
He Weiwei
43df7b2577
Merge pull request #11459 from heww/scan-cleanup 
refactor: cleanup unused code about scan
 2020-04-07 12:00:48 +08:00
Wenkai Yin
9bfabff4d2 Update the logic of copy artifact 
1. Copy artifact will not return 409 anymore.
2. Make sure the tags of source artifact exist in the target artifact

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-07 10:55:55 +08:00
He Weiwei
6b066bade5 feat(scan): merge reports for image index 
1. Merge the scanning reports of referenced artifacts for image index.
2. Add artifact info for report.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-07 02:54:01 +00:00
Wenkai Yin(尹文开)
b819e7ae83
Merge pull request #11438 from ywk253100/200402_replication 
Support replication between Harbor 2.0 and 1.x
 2020-04-07 10:24:59 +08:00
Daniel Jiang
db10720e80
Merge pull request #11406 from reasonerjt/reenable-token-auth-for-cli-new 
Reenable token auth for cli
 2020-04-07 08:55:25 +08:00
He Weiwei
69ca7a0dae refactor: cleanup unused code about scan 
1. Cleanup unused code about clair.
2. Cleanup unused definitions in legacy_swagger.yaml about scan.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-06 17:34:25 +00:00
He Weiwei
80027c3b86
Merge pull request #11397 from ywk253100/200402_chart_api 
Add a seperated swagger file for chart API
 2020-04-07 00:05:49 +08:00
Wenkai Yin(尹文开)
e6f96e2a8b
Merge pull request #11427 from wy65701436/fixes-11280 
Add trace information into internal error
 2020-04-06 17:51:40 +08:00
wang yan
44825e819e deprecate quota count on artifact 
Fixes #11241

1, remove count quota from quota manager
2, remove count in DB scheme
3, remove UI relates on quota
4, update UT, API test and UI UT.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-06 16:56:11 +08:00
Wenkai Yin
8f8b4d5e8d Add a seperated swagger file for chart API 
Add a seperated swagger file for chart API as these APIs have no version

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-06 16:30:26 +08:00
Daniel Jiang
e8f98259dd Make sure middleware handle scanner-pull claim for v2token 
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-05 01:10:45 +08:00
Wenkai Yin
8f11cb7ff0 Support replication between Harbor 2.0 and 1.x 
Fixes #11374, fixes #11302, support replication between Harbor 2.0 and 1.x by providing versioning adapter

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-04 22:08:11 +08:00
Wenkai Yin(尹文开)
9ca87b85a5
Merge pull request #11389 from wy65701436/fix-dao-ut 
fix artifact dao UT issue
 2020-04-04 10:32:03 +08:00
wang yan
8bd2dc6394 Add trace information into internal error 
Fixes #10839
Add a StackTrace func in to Error, and log it when Harbor gets a internal

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-04 01:38:36 +08:00
He Weiwei
bd6c2f8870
fix(vulnerable,middleware): improve vulnerable middleware (#11407) 
1. Prevent the pull action when scan report status is not successfuly.
2. Bypass the checking when no vulnerabilities not found.
3. Improve the returned message when prevented the pull action.

Closes #11202

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-04 00:54:57 +08:00
Daniel Jiang
08f9ffa000 Reenable token auth for cli 
Docker CLI fails if it's not logged in upon seeing "basic" realm challenging while pinging the "/v2" endpoint. (#11266)
Some CLI will send HEAD to artifact endpoint before pushing (#11188)(#11271)

To fix such problems, this commit re-introduce the token auth flow to the CLIs.

For a HEAD request to "/v2/xxx" with no "Authoirzation" header, the v2_auth middleware populates the
"Www-Authenticate" header to redirect it to token endpoint with proper
requested scope.

It also adds security context to based on the content of the JWT which has the claims of the registry.
So a request from CLI carrying a token signed by the "/service/token" will have proper permissions.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-04-04 00:05:58 +08:00
danfengliu
616c2d9e0f
Merge pull request #11408 from jwangyangls/nightly-case-trivy-3 
[Test Case] Add nightly case for CVE
 2020-04-03 19:23:26 +08:00
Yogi_Wang
2610fe530f [Test Case] Add nightly case for CVE 
1. add nightly case for cve
2. change translate words
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-03 16:59:45 +08:00
He Weiwei
e9543a1e3c
Fix merge scan summary (#11392) 
* fix(scan): fix ScanStatus when merge NativeReportSummary

1. Running and success status is high priority when merge ScanStatus of
NativeReportSummary, otherwise chose the bigger status.
2. Merge scan logs of referenced artifacts when get the scan logs of
image index.

Closes #11265

Signed-off-by: He Weiwei <hweiwei@vmware.com>

* fix(portal): fix the annotation for the scan completed percent in scan overview

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-03 16:21:36 +08:00
Wenkai Yin(尹文开)
2783fd0950
Merge pull request #11276 from mmpei/offical-webhook-slack-fix 
fix webhook slack test error
 2020-04-03 10:37:21 +08:00
He Weiwei
c0246e2130
Merge pull request #11400 from heww/fix-issue-11391 
fix(log): correct file and line when use logger
 2020-04-02 22:23:15 +08:00
peimingming
5924658092 fix webhook slack test error 
Signed-off-by: peimingming <peimingming@corp.netease.com>
 2020-04-02 20:02:27 +08:00
He Weiwei
207463e91e fix(log): correct file and line when use logger 
1. When use the helper functions of log pkg, the depth is 4 to get the
correct file and line.
2. Whe use the default logger of log pkg, the depth is 3 to get the
correct file and line.

Closes #11391

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-04-02 10:49:38 +00:00
Will Sun
2c3552904d
Merge pull request #11393 from AllForNothing/improve-webhook 
Improve webhook UI according to the UX
 2020-04-02 17:51:13 +08:00
wang yan
7104461716 fix artifact dao UT issue 
The update column should be PullTime instead of PushTime

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-02 14:45:43 +08:00
AllForNothing
ba5fd67b08 Improve webhook UI according to the UX 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-02 14:30:21 +08:00
wang yan
a11a70d941 move logger from common to lib 
The logger is the fundamental library, so move it into lib folder
Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-02 14:09:03 +08:00
jwangyangls
4ea7b13215
Merge pull request #11385 from jwangyangls/nightly-case-trivy-2 
[Nightly] Project Level Image Serverity Policy
 2020-04-02 11:14:21 +08:00
Yogi_Wang
01f8291bb7 [Nightly] Project Level Image Serverity Policy 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-02 10:16:57 +08:00
danfengliu
f11e713ada
Merge pull request #11380 from jwangyangls/nightly-case-trivy-1 
[Nightly] add case about trivy
 2020-04-01 22:45:40 +08:00
Steven Zou
1f6301267c
Merge pull request #11369 from steven-zou/fix/issue_#11361 
fix[lua_scripts]:add default values for tonumber
 2020-04-01 19:01:02 +08:00
Qian Deng
b1284da96b
Merge pull request #11360 from ninjadq/rever_chart_api_change 
Rever chart api change
 2020-04-01 18:58:57 +08:00
Wenkai Yin(尹文开)
d187a8e69e
Merge pull request #11333 from ywk253100/200325_copy 
Update the existence checking logic when copying artifact
 2020-04-01 18:09:20 +08:00
Wang Yan
4594d58ba8
add clean untagged blobs in gc job (#11248) 
Fixes #11190, delete all of non-referenced blobs of each project in GC job, thun the quota
can be released.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-01 18:01:46 +08:00
Will Sun
b0d0b292cd
Merge pull request #11288 from AllForNothing/nightly-webhook 
Fix nightly cases for webhook
 2020-04-01 17:37:11 +08:00
Wenkai Yin(尹文开)
9f4f3be00d
Merge pull request #11364 from ywk253100/200331_replication 
Some tiny improvement for replication
 2020-04-01 17:29:08 +08:00
Wenkai Yin
e4d42deb75 Make sure the tag filter have the same behavior for empty value and * 
Fixes #11233, make sure the tag filter have the same behavior for empty value and *

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-04-01 17:15:23 +08:00
Yogi_Wang
24b57715ab [Night] add case about trivy 
`
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-04-01 17:06:34 +08:00
AllForNothing
e6e3f0a6af Fix nightly cases for webhook 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-04-01 16:27:40 +08:00
DQ
6216073d2a Add ui change 
using api/chartreport for ui

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 11:55:22 +08:00
DQ
9ff7d46e8f Rever chart version to original 
Because chart version should consistent with previous version

Signed-off-by: DQ <dengq@vmware.com>
 2020-04-01 11:55:22 +08:00
Wang Yan
f6c0608e22
fix GC jobs upgrade issue (#11365) 
Fixes #11313
Fixes #11275

1, Add more details log in GC job
2, Add type assertion for the upgrading case, the delete_untagged parameter is introduced from v2.0
3, Add UT

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-04-01 11:53:41 +08:00
Wenkai Yin(尹文开)
c2c9fa28eb
Merge pull request #11368 from heww/fix-ongoing-of-metrics 
fix(scan): ongoing is true for schedule scan all only when job is running
 2020-03-31 20:10:40 +08:00
He Weiwei
1a7cad3a14
Merge pull request #11370 from heww/fix-issue-11198 
fix(scan): add scanner name as prefix for name of the robot when submit scan job
 2020-03-31 19:23:28 +08:00
Steven Zou
b5fceae734 fix[lua_scripts]:add default values for tonumber 
- add default values for the integer vars converted by tonumber()

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-03-31 18:20:52 +08:00
Wang Yan
d6261d9456
Does not throw err in the notification job (#11363) 
Fixes #11280, no error return but just log.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-31 18:12:12 +08:00
He Weiwei
120be36fec fix(scan): ongoing is true for schedule scan all only when job is running 
Closes #11289

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-31 10:08:22 +00:00
AllForNothing
0275108cb2 Fix bugs for round 1 testing 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-31 16:21:37 +08:00
jwangyangls
ceded08507
Merge pull request #11362 from jwangyangls/refact-artifact-tag 
[OCI] Refact artifact tag
 2020-03-31 15:46:45 +08:00
Yogi_Wang
a6e986df62 [OCI] Refact artifact tag 
1.get artifact tag from another api
2.add refresh button  in artifact tag
3.fix permission change
4.some ui style
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-31 14:57:39 +08:00
Wenkai Yin
d9a5c71289 Some tiny improvement for replication 
1. Add timeout when transter artifacts
2. Check 404 error when unschedule the policy
3. Add line to mark the job failure in job log

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-31 14:50:37 +08:00
Wang Yan
34d05dae58
fix content trust middleware bypass scanner pull (#11321) 
Fixes #11206
1, fix middleware doesn't work for docker pull without auth
2, fix middleware doesn't bypass scanner pull

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-31 11:12:21 +08:00
jwangyangls
3c6f5cce54
Merge pull request #11352 from jwangyangls/fix-bug-2.0-2 
[Fix] some harbor 2.0 UI bug
 2020-03-31 11:02:06 +08:00
He Weiwei
86d446ce81
fix(log): change log level from warning to debug when unescape path params (#11359) 
Closes #11186

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-31 10:33:18 +08:00
Daniel Jiang
fdb82ae4fa
Merge pull request #11349 from reasonerjt/fix-10602 
Not checking for registry credentials in v2auth
 2020-03-31 10:26:33 +08:00
Daniel Jiang
37f9d650bd Not checking for registry credentials in v2auth 
That was added to support core process sending request to `/v2/xxx`.
It's no longer needed after reworking the flow.
This commit removes this.

Fixes #10602, as it's not a case we need to support for now.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
 2020-03-31 01:08:37 +08:00
He Weiwei
f4d96d85f8 fix(scan): add scanner name as prefix for name of the robot when submit scan job 
Closes #11198

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-30 16:52:04 +00:00
Steven Zou
3d6c65f53b
fix[UT]:improve the UT cases of enqueuer (#11358) 
- change the test cron spec
- use cretaed timer for timeout

Signed-off-by: Steven Zou <szou@vmware.com>
 2020-03-31 00:09:20 +08:00
Steven Zou
f2beed577f
fix[logger]:update log ID validation logic (#11351) 
Signed-off-by: Steven Zou <szou@vmware.com>
 2020-03-31 00:08:55 +08:00
Yogi_Wang
661867240d [Fix] 2.0 UI bug 
1.fix #11312
2.fix #11235
3.fix #11230
4.fix #11209
5.fix #11199
6.fix #11034
7.fix #9926
Signed-off-by: Yogi_Wang <yawang@vmware.com>
 2020-03-30 17:49:42 +08:00
Wenkai Yin(尹文开)
4faff18b2d
Merge pull request #11339 from ywk253100/200328_limit_offset 
Add "order by" clause to avoid the duplicat rows
 2020-03-30 17:14:44 +08:00
Wenkai Yin
fb975d902c Add "order by" clause to avoid the duplicat rows 
Add "order by" clause to avoid the duplicat rows: https://www.postgresql.org/docs/9.6/queries-limit.html

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-30 16:42:43 +08:00
Wenkai Yin(尹文开)
6815e8dc4d
Merge pull request #11348 from ywk253100/200330_tag_filter 
Make sure the tag filter have the same behavior for empty value and *
 2020-03-30 14:02:37 +08:00
He Weiwei
fbae9f0c25
Merge pull request #11347 from heww/refactor-errors 
Refactor errors
 2020-03-30 13:06:07 +08:00
Wenkai Yin
7ec5595bd8 Make sure the tag filter have the same behavior for empty value and * 
Fixes #11233, make sure the tag filter have the same behavior for empty value and *

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-30 10:10:12 +08:00
Will Sun
9a205ddbc3
Merge pull request #11332 from AllForNothing/scan-result-modify 
Modify UI for scanning result
 2020-03-30 09:47:45 +08:00
He Weiwei
1bf142c33b refactor: use lib/errors to instead of scan/errs 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-29 05:19:06 +00:00
He Weiwei
9c06c79ff4 refactor(errors): rename pkglib/error to lib/errors 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-29 05:19:05 +00:00
Wenkai Yin(尹文开)
98759642b7
Add API to list tags under the specific repository (#11336) 
Add API to list tags under the specific repository

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-29 12:19:54 +08:00
Steven Zou
1bbd3585e5
Merge pull request #11296 from ywk253100/200326_replication 
Support replicate images with media type application/vnd.docker.distribution.manifest.v1+json
 2020-03-27 18:12:12 +08:00
Steven Zou
36552ba18b
Merge pull request #11318 from ywk253100/200326_remove_pagination_default 
Iterate the link header when listing artifact
 2020-03-27 18:07:03 +08:00
Wang Yan
eccb8aa708
append pull permission for push policy (#11303) 
Fixes #11225
As registry changes to basic auth, the push action lost the pull permission.
Add it in the robot security context.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-27 17:10:04 +08:00
AllForNothing
3c51e37702 Modify UI for scanning result 
Signed-off-by: AllForNothing <sshijun@vmware.com>
 2020-03-27 16:41:38 +08:00
He Weiwei
033d6dac6b
fix(quota): allowed to put blob which size is zero (#11314) 
Closes #11239

Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-27 14:39:25 +08:00
Wenkai Yin(尹文开)
f4ad0fbf00
Use the same logic to parse the registry URL (#11320) 
Use the same logic to parse the registry URL to fix #11274

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-27 10:33:20 +08:00
Wenkai Yin(尹文开)
e8cc84738a
Merge pull request #11309 from ywk253100/200326_error 
Fix bugs of replication
 2020-03-27 10:31:03 +08:00
Wang Yan
a5c1eae81a
give the username to anonymous when to pull public resource without authN (#11306) 
For pull a public resource, there is no need to login, give the access name to anonymous in the audit logs

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-27 10:24:59 +08:00
Wenkai Yin
9a92b9e725 Fix bugs of replication 
1. Bump up the version of API used in replicatoin scheduler job
2. Check the error message to determine whether the job exists or not in jobservice when unschedule a job

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 23:12:02 +08:00
Wenkai Yin
afdfedcb49 Iterate the link header when listing artifact 
Fixes #11315
When specify no pagination in listing artifact request, the go-swagger will set the default value for them, so we need to iterate the link header to get all of artifacts

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 21:26:09 +08:00
Daniel Pacak
48df949c30
feat(trivy): Return Trivy DB update timestamp in /api/v1/metadata response (#11285) 
Resolves: #11284

Signed-off-by: Daniel Pacak <pacak.daniel@gmail.com>
 2020-03-26 19:22:28 +08:00
Wenkai Yin
64e4651b3f Support replicate images with media type application/vnd.docker.distribution.manifest.v1+json 
Fixes #11272, support replicate images with media type application/vnd.docker.distribution.manifest.v1+json

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 19:09:48 +08:00
Wenkai Yin
213c534e8a Return 404 rather than 500 error when getting registry info 
In Harbor 2.0, the replication isn't supported between instances with different versions, this commit returns the 404 error when trying to get the registry info whose version is different with the current one

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-26 18:07:11 +08:00
He Weiwei
73f3a305ce
refactor: rename testing/api to testing/controller (#11295) 
Signed-off-by: He Weiwei <hweiwei@vmware.com>
 2020-03-26 14:00:11 +08:00
jwangyangls
96572c3c86
Merge pull request #11254 from jwangyangls/nightly-case-3 
[Fix]   Fix issue 2.0
 2020-03-26 12:14:37 +08:00
Yogi_Wang
cba4490a5a [Fixed] Fix bug for 2.0 and add case for trivy 
Signed-off-by: Yogi_Wang <yawang@vmware.com>
1.add case for trivy
2.vunerbility refresh bug
3.scan mutiple artifact
4.fix global search bug
5.disable delete tag btn when remove immutable tag
6.cancel selectRow when add label or remove label;fix #11195
7.fix cron tootip
 2020-03-26 11:39:57 +08:00
qinshaoxuan
df9c2bdc46 Fix bug when scanner is unhealthy 
The function GetRegistrationByProject should not return err when Ping
return err.  The return value 'registration' has 'Health' field which
shows the scanner health status.

Resolves: #11051
See also: #9788, #9807

Signed-off-by: qinshaoxuan <qinshaoxuan@baidu.com>
 2020-03-26 11:25:47 +08:00
Wang Yan
da8902da53
Handle empty orlist in orm query (#11270) 
Fixes #11267
When caller parse an empty orlist to orm lib, it will parse the empty vaule to beego orm.
But beego will panic if the query string is empty.

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-26 11:23:20 +08:00
Wenkai Yin(尹文开)
d05817c8a2
Update the URL checking logic of auth proxy security generator (#11180) 
As we don't support bearer token in Harbor 2.0, the URL checking logic in auth proxy security generator should be updated

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-25 01:19:11 +08:00
Wenkai Yin(尹文开)
8984979bd2
Relocate/rename some packages (#11183) 
Fixes #11016
1. src/pkg/q->src/internal/q
2. src/internal->src/lib (internal is a reserved package name of golang)
3. src/api->src/controller

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-24 20:45:45 +08:00
Ted Guan
e49a247d3d
Replication webhook support (#11179) 
* replication webhook support

Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>

* replication webhook support with ut fixed

Signed-off-by: guanxiatao <guanxiatao@corp.netease.com>
 2020-03-23 18:45:58 +08:00
Wang Yan
168637a743
Add permission check for audit logs API (#11154) 
add a base method to require system admin permission

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-23 16:59:10 +08:00
Wang Yan
15d2a93aa2
Fix orm query setter issue (#11177) 
For the Andlist, the query setter should ignore it

Signed-off-by: wang yan <wangyan@vmware.com>
 2020-03-23 16:05:20 +08:00
Wenkai Yin(尹文开)
1762bfae69
Merge pull request #11158 from ywk253100/200320_repository_api 
Add "_self" suffix for repository API to avoid conflict
 2020-03-23 14:45:52 +08:00
Wenkai Yin(尹文开)
c4af6ff824
Fix bug when deleting the repository (#11121) 
Fixes #10997 by looping the artifact candidates until all of them are deleted

Signed-off-by: Wenkai Yin <yinw@vmware.com>
 2020-03-23 10:38:47 +08:00
Wenkai Yin(尹文开)
8688f78cd2
Merge pull request #11144 from ywk253100/200319_security_middleware 
Rewrite the filters with middleware mechinism
 2020-03-23 10:12:48 +08:00