Commit Graph

3442 Commits

Author SHA1 Message Date
Daniel Jiang
06e4e124d8
Refine request handle process (#9760)
* Refine request handle process

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2019-11-07 13:02:17 +08:00
Steven Zou
ee31418e8e revoke scan permission from the developer role
Signed-off-by: Steven Zou <szou@vmware.com>
2019-11-06 17:57:48 +08:00
Steven Zou
240f771006
Merge pull request #9733 from steven-zou/fix/scan_all_enhancements
do improvements to the scan all job
2019-11-05 16:22:50 +08:00
Steven Zou
ebc5d2482b do improvements to the scan all job
- update scan all job to avoid sending too many HTTP requets
- update scan controller to support scan options
- update the db schema of the scan report to introduce requester
- introduce scan all metrics to report the overall progress of scan all job
- fix the status updating bug in scan report
- enhance the admin job status updats
- add duplicate checking before triggering generic admin job
- update the db scheme of admin job

fix #9705
fix #9722
fix #9670

Signed-off-by: Steven Zou <szou@vmware.com>
2019-11-05 15:12:07 +08:00
Yogi_Wang
143f2a13b7 Fix the quoestion of cannot soft by pull time
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-11-05 14:05:24 +08:00
jwangyangls
473b453616
Merge pull request #9703 from jwangyangls/fix-replication-name-conflict
Fix a bug that when someone edits a rule with same name ,it displays the name is already in use
2019-11-04 18:22:06 +08:00
Yogi_Wang
03e54729c3 Fix bug of UI;
1.One of the request is cancelled when trying to do the multiple deletion for tags
2.when using other languages, switch languages, English button has default border
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-11-04 15:11:21 +08:00
jwangyangls
8cf5dba39a
Merge pull request #9696 from jwangyangls/fix-webhook-wrong-text
Modify the tootip words of verify remote cert.
2019-11-04 11:45:47 +08:00
Yogi_Wang
dd9a786b49 Modify the tootip of verify remote cert.
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-11-04 10:40:24 +08:00
Yogi_Wang
92cac6fd35 Fix a bug that when someone edits a rule with same name ,it displays the name is already in use
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-11-01 12:37:19 +08:00
Steven Zou
a928928a43
Merge pull request #9686 from heww/fix-project-severity-mapping
fix(policy-checker): add func to transform project severity to vuln.Severity
2019-11-01 11:12:13 +08:00
jwangyangls
a21349fc66
Merge pull request #9676 from jwangyangls/modify-ui-style-quota
Improve the style of quota ui
2019-11-01 11:10:49 +08:00
Daniel Jiang
f2beee16b1
Merge pull request #9673 from steven-zou/fix/issue_#9668_status_conflicts
return more clear error message for scan related API
2019-11-01 11:08:43 +08:00
Wenkai Yin(尹文开)
2101d7125c
Merge pull request #9680 from heww/install-clair-adapter-imporvement
fix(scanner): imporve clair adapter initializing
2019-11-01 10:42:04 +08:00
Will Sun
1c8105e837
Merge pull request #9663 from AllForNothing/nav-modify
Modify harbor-shell UI
2019-11-01 10:26:54 +08:00
sshijun
b4fa143aa4 Modify harbor-shell UI
Signed-off-by: sshijun <sshijun@vmware.com>
2019-11-01 09:31:33 +08:00
He Weiwei
ae8931e816 fix(policy-checker): add func to transform project severity to vuln.Severity
The severity saved in db is lowercase but the severities in vuln pkg
begin with upper letter, this fix use func to transform project severity
value from db to vuln.Severity.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-31 14:11:44 +00:00
wilmardo
1f79fb085b feat: Expand password length to unlimited
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2019-10-31 15:01:27 +01:00
Steven Zou
eb8ec49f4f add UT cases for the common error pkg
Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-31 21:55:27 +08:00
Daniel Jiang
bc65609a10
Merge pull request #9657 from wy65701436/quota-sync-switcher
add a switcher for quota sync on core launch
2019-10-31 19:22:23 +08:00
Wang Yan
fa784d7514
Merge pull request #9649 from wy65701436/fix-9081
add ldflags for harbor compiler and linker
2019-10-31 19:14:16 +08:00
Yogi_Wang
bdeddd4d95 Improve the style of quota ui
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-10-31 16:38:20 +08:00
wang yan
c46d7e856a add a switcher for quota sync on core launch
As the quota sync is default called by harbor-core on every launch, and it will break the launch process if any failure throwed.

1, The commit is to provide an switcher for the system admin to bypass the quota sync.
2, In case Harbor goes into the restarting cycle.

Harbor already provides an internal API to sync quota data, in the failure case,
system admin can launch harbor and call the /api/internal/syncquota to sync quota.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-31 16:17:27 +08:00
He Weiwei
7170485a9b fix(scanner): imporve clair adapter initializing
1. Remove ping action when initialize clair adapter installed by harbor.
2. Remvoe the `IsDefault` property when initialize clair adapter that
make it switch to auto detecting.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-31 08:03:37 +00:00
jwangyangls
b0b63fdeae
Merge pull request #9666 from jwangyangls/fix-ut-warning
Fix a warning that appears when running the test
2019-10-31 15:37:21 +08:00
Yogi_Wang
7aa47f8450 Fix a warning that appears when running the test
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-10-31 14:28:59 +08:00
Steven Zou
afb46188b2 return more clear error message for scan related API
- add a common error pkg to support error with code and AsError check
- replace some errors in scan with coded errors
- fix #9668

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-31 11:35:55 +08:00
jwangyangls
cfffbb5e37
Merge pull request #9659 from jwangyangls/fix-ui-style-margin-top-0
Fix page style confusion caused by error margin-top
2019-10-31 10:03:04 +08:00
wang yan
253e87d186 inject ldflags for harbor compiler and linker
1, replace the UIVERSION file with ldflags, which is generarted by make to inject into the UI core.
2, inject additional ldflags for harbor compiler

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-30 18:31:42 +08:00
Wenkai Yin(尹文开)
5d6cbe9aa1
Merge pull request #9654 from heww/fix-qutoa-order-by
fix(quota): order by quotas only on support resources
2019-10-30 17:20:17 +08:00
Yogi_Wang
f7532ff6a1 Fix page style confusion caused by error margin-top
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-10-30 16:46:10 +08:00
jwangyangls
b964d0411b
Merge pull request #9622 from jwangyangls/fix-ui-immutable-style
Improve immutable ui style
2019-10-30 15:03:18 +08:00
Yogi_Wang
a9adc50550 Improve immutable ui style
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-10-30 12:07:48 +08:00
Will Sun
5f62b5778b
Merge pull request #9611 from AllForNothing/scanner-bug
Improve scanner UI
2019-10-30 10:43:48 +08:00
He Weiwei
3c80832341 fix(quota): order by quotas only on support resources
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-30 02:42:34 +00:00
sshijun
0c41a01f83 Improve scanner UI
Signed-off-by: sshijun <sshijun@vmware.com>
2019-10-29 18:47:57 +08:00
Steven Zou
19eb0ae7f4
Merge pull request #9637 from steven-zou/fix/bug_#9629
fix the bug of returning errors when nothing is updated
2019-10-29 17:37:06 +08:00
Steven Zou
adf780c2a5
Merge pull request #9634 from steven-zou/fix/#9632_and_#9633
fix bug found in the pluggable scanner
2019-10-29 17:36:19 +08:00
Will Sun
afd3b76001
Merge pull request #9567 from AllForNothing/scanner-css
Fix package vulnerabilities
2019-10-29 16:51:04 +08:00
jwangyangls
26453e8806
Merge pull request #9607 from jwangyangls/fix-delete-repositories-error
Fix the error when delete some images
2019-10-29 15:49:29 +08:00
Will Sun
c31b08ac3d
Merge pull request #8610 from AllForNothing/rep
change adapterInfo from by hardcode to by getting from backend API
2019-10-29 15:49:07 +08:00
Steven Zou
cb8d4d0daf fix the bug of returning errors nothing is updated
- bug details: #9629
- root cause: the preconditions for updating may not be matched

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-29 15:34:34 +08:00
Steven Zou
35d074e849 fix bug found in the pluggable scanner
- fix #9632
- fix #9633

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-29 15:32:51 +08:00
Daniel Jiang
b17711abbf
Merge pull request #9592 from qyqcswill/code_clean
remove useless code
2019-10-29 15:08:59 +08:00
sshijun
4a97cf40c6 change adapterInfo from by hardcode to by getting
from backend API

Signed-off-by: sshijun <sshijun@vmware.com>
2019-10-29 14:13:19 +08:00
Steven Zou
10c30fde3d
Merge pull request #9618 from steven-zou/fix/bug_#9608
permission grant for scanner related actions are not correctly
2019-10-28 22:12:56 +08:00
Steven Zou
5b2ab34e03 permission grant for scanner related actions are not correctly
- add new endpoint for getting scanner candidates of specified project
- adjust the permission granting functions
- fix #9608

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-28 18:20:47 +08:00
Wang Yan
e0fd4cd609
Merge pull request #9599 from wy65701436/fix-quota-migration
Fix quota sync crash on getting the unknown mainfest
2019-10-28 17:47:46 +08:00
Yogi_Wang
e7c7e7ac25 Fix the error when delete some images
Signed-off-by: Yogi_Wang <yawang@vmware.com>
fix #9604 fix #9603
2019-10-28 15:43:50 +08:00
wang yan
fc347fc4eb Fix quota sync crash on getting the unknown mainfest
1, eat the unknown manifest error, and log it. The migration process will not crashed on it.
2, enable to persist DB of sync quota API.
3, add empty project support.

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-28 15:17:08 +08:00
Steven Zou
5c4c04a122
Merge pull request #9602 from heww/upgrade-clair-adapter
Upgrade clair adapter to v1.0.0
2019-10-28 12:19:28 +08:00
jwangyangls
987da4212d
Merge pull request #9559 from jwangyangls/fix-immutable-bug
Improve Immutable tag UI
2019-10-28 11:27:38 +08:00
Wenkai Yin(尹文开)
f007a62b04
Merge pull request #9588 from stonezdj/fix_ldap_group_sql
Fix User Group Search SQL error
2019-10-28 11:22:14 +08:00
Yogi_Wang
26381c93be Improve Immutable tag UI
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-10-28 10:07:40 +08:00
jwangyangls
e8554b9d66
Merge pull request #9587 from jwangyangls/add-webhook-tests-1.10
Add webhook nightly case
2019-10-28 10:05:16 +08:00
He Weiwei
28e0c0693b Upgrade clair adapter to v1.0.0
1. Upgrade clair adapter to v1.0.0.
2. Make the clair adapter which installed by harbor immutable and using internal registry address.
3. Add support to build clair adapter image from binary.
4. Switch to ScannerPull action when make authorization for the scan request.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-26 17:25:36 +00:00
Steven Zou
eba1a01ac2
Merge pull request #9595 from steven-zou/fix/update_scanner_failure
property use_internal_addr can not be updated
2019-10-25 17:39:34 +08:00
jwangyangls
8d35d36537
Merge pull request #9552 from jwangyangls/add-ut
Add front ut in app
2019-10-25 15:45:13 +08:00
Wang Yan
20a867a13c
Merge pull request #9579 from wy65701436/fix-id
update immutable rule API
2019-10-25 15:43:08 +08:00
hao.cheng
29e905271d promote code quality
Signed-off-by: hao.cheng <hao.cheng@daocloud.io>
2019-10-25 15:37:35 +08:00
Steven Zou
9d0263fc9a property use_internal_addr can not be updated
- pick up `use_internal_addr` in the update API

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-25 15:35:43 +08:00
Yogi_Wang
988279cdcb Add webhook nightly case
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-10-25 15:25:32 +08:00
hao.cheng
94bc8c2f5c remove useless code
Signed-off-by: hao.cheng <hao.cheng@daocloud.io>
2019-10-25 15:20:25 +08:00
wang yan
f9996663d8 update immutable rule API
1, unify disable and enable
2, fix update rule error

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-25 14:11:07 +08:00
stonezdj
f402db380b Fix User Group Search SQL error
User Group Query SQL error in some cases

Signed-off-by: stonezdj <stonezdj@gmail.com>
2019-10-25 14:00:45 +08:00
Wenkai Yin(尹文开)
28ee5f7d43
Merge pull request #9302 from liuxu623/fix-fatal-error
fix fatal error
2019-10-25 13:16:54 +08:00
Wenkai Yin(尹文开)
91bc57d695
Merge pull request #9566 from qyqcswill/code_simplify
code simplify
2019-10-25 13:15:50 +08:00
jwangyangls
066dbcb658
Merge pull request #9524 from jwangyangls/modify-filter
Modiy the way of replication show label filter
2019-10-25 12:55:41 +08:00
Steven Zou
868851cc51
Merge pull request #9571 from steven-zou/fix/support_internal_addr_scanner
support using internal registry addr to perform scan
2019-10-24 20:52:27 +08:00
Wang Yan
d18678a48d
Merge pull request #9506 from wy65701436/token-sevice
Enable robot account to support scan pull case
2019-10-24 19:52:33 +08:00
Steven Zou
cb59ba3bbc support using internal registry addr to perform scan
- do changes to the sql schema
- add `UseInternalAddr` and `Immutable` properties to scanner registration
- support multiple authentication type
  - basic
  - bearer token

Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-24 18:28:35 +08:00
wang yan
71c769ec97 remvoe bypass to scanner pull
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-24 17:49:20 +08:00
hao.cheng
758e7aa06f code simplify
Signed-off-by: hao.cheng <hao.cheng@daocloud.io>
2019-10-24 17:42:42 +08:00
Ziming
13499fb60b
Merge pull request #9491 from bitsf/tag_retention_webhook
implement log for tag retention immutable tags
2019-10-24 17:06:00 +08:00
sshijun
8bab3f7cd4 Fix package vulnerabilities
Signed-off-by: sshijun <sshijun@vmware.com>
2019-10-24 16:37:52 +08:00
Ziming
eb6708ed76
Merge pull request #8174 from bitsf/refact_replication_info
refact replication info to support special endpoints and credential
2019-10-24 15:34:01 +08:00
Ziming Zhang
e757899b49 add special error/log for not delete immutable tag in tag retention job
Change-Id: I3440f3b888bf8c65afc75d04253eea41f20eef0e
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-10-24 15:25:48 +08:00
Steven Zou
956d9db1d5
Merge pull request #9528 from steven-zou/fix/pluggable_scanner_improvments
improve the scan controlling
2019-10-24 15:17:06 +08:00
Wang Yan
4baa35bc17
Merge pull request #9520 from ywk253100/191022_health_check
Remove the health checker for Clair in health check API
2019-10-24 14:50:01 +08:00
Yogi_Wang
52603de9a6 Add front ut in app
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-10-24 13:22:36 +08:00
Ziming Zhang
1801bac03d refact replication adapter
Change-Id: Ic28854089b8dcfcbc7e42065df5c19c64d5b85e7
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-10-24 12:24:55 +08:00
Steven Zou
7fad103e46 - fix API test cases failures
Signed-off-by: Steven Zou <szou@vmware.com>

- fix scan report dao bug
2019-10-23 20:44:01 +08:00
wang yan
a6ad1b2db8 update code per review comments
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-23 20:05:51 +08:00
wang yan
2fa85aefca fix per comments
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-23 18:45:30 +08:00
wang yan
5996189bb0 update per comments and fix govet error
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-23 18:45:30 +08:00
wang yan
22b4ea0f89 Enable robot account bypass policy check
1, the commit is for internal robot to bypass policy check, like vul and signature checking.
2, add a bool attribute into registry token, decode it in the harbor core and add the status into request context.
3, add a bool attribut for robot API controller, but API will not use it.y

Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-23 18:45:30 +08:00
Ziming Zhang
5419e1a844 refact replication info to support special endpoints and credential
Change-Id: I2f7a51d3aaf57bb6d1942526184f4e62ce3afeab
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2019-10-23 17:19:56 +08:00
Wenkai Yin(尹文开)
c3c8b03af5
Merge pull request #9532 from heww/issue-9151
fix(gc,quota): clean blob size cache in redis when gc
2019-10-23 17:03:22 +08:00
Will Sun
ee2de96c91
Merge pull request #9518 from AllForNothing/scanner-im-v2
Improve scanner UI
2019-10-23 17:01:26 +08:00
Wenkai Yin(尹文开)
bd28ba43c0
Merge pull request #9478 from ywk253100/191018_test
Populate public metadata into the event
2019-10-23 15:21:53 +08:00
Will Sun
2bee9d7047
Merge branch 'master' into scanner-im-v2 2019-10-23 14:12:42 +08:00
Steven Zou
ae3ff228e1 Merge branch 'master' into fix/pluggable_scanner_improvments 2019-10-23 13:31:39 +08:00
Steven Zou
38395e015c fix api test case failure
Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-23 13:27:03 +08:00
wang yan
182e557bf2 update query in the immutable delete manifest middleware
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-23 13:26:19 +08:00
Wenkai Yin
9d896d4d72 Remove the health checker for Clair in health check API
As we introduce the pluggable scanner, users can add the external scanners, so we remove the Clair from the health check API

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2019-10-23 12:55:03 +08:00
jwangyangls
cf87408e90
Merge pull request #9522 from jwangyangls/modify-repo-replicaiton
Modify the permission when add Limited Guest
2019-10-23 11:42:27 +08:00
Yogi_Wang
f38fff8eff Modify the permission when add Limited Guest
Signed-off-by: Yogi_Wang <yawang@vmware.com>
2019-10-23 10:31:31 +08:00
Steven Zou
962bafb7ce fix go imports order issues
Signed-off-by: Steven Zou <szou@vmware.com>
2019-10-23 09:34:47 +08:00
He Weiwei
cc86664730 fix(gc,quota): clean blob size cache in redis when gc
Closes #9151

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2019-10-22 14:20:04 +00:00
wang yan
3e826c4e80 update query in the immutable delete manifest middleware
Signed-off-by: wang yan <wangyan@vmware.com>
2019-10-22 19:37:20 +08:00